Commit graph

326 commits

Author SHA1 Message Date
Mark Johnston be2f92a96b makefs/zfs: Add a helper function for adding ZAP entries
Some ZAPs are used to represent sets, in which keys and values are the
same.  Add a helper function for this case.  No functional change
intended.

MFC after:	1 week
2024-06-01 11:16:18 -04:00
Ed Maste 176b9e0d72 makefs: warn that ffs sectorsize other than 512 may not work
newfs always sets sectorsize to DEV_BSIZE (512) and derives some other
values based on the number of 512-byte sectors per real sector.  Similar
logic is required in makefs.  Until that happens, emit a warning that
the image may be incorrect.

PR:		276571
2024-01-26 17:27:57 -05:00
Mark Johnston e0deb85065 makefs: Make it possible to silence warnings about duplicate paths
When generating a VM image from an installworld mtree manifest, makefs
spits out several thousand warnings about duplicate paths in the
manifest.  These are harmless and have been around for a long time (see
the phabricator revision for some more details), so let's at least have
a way to make makefs quieter.

Reviewed by:	brooks, imp, emaste
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D43513
2024-01-19 19:50:56 -05:00
Jose Luis Duran 518a1fd7d3 makefs: Fix ZFS cross-reference sections
MFC after:	1 week
Fixes:		240afd8c1f ("makefs: Add ZFS support")
Pull Request:	https://github.com/freebsd/freebsd-src/pull/979
2024-01-02 09:20:26 -05:00
Warner Losh 4d65a7c695 usr.sbin: Automated cleanup of cdefs and other formatting
Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by:		Netflix
2023-11-26 22:24:01 -07:00
Warner Losh 8a7b612037 usr.sbin: Remove ancient SCCS tags.
Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl script.

Sponsored by:		Netflix
2023-11-26 22:23:56 -07:00
John Baldwin 5fc8ed9313 makefs: Quiet a -Wdangling-pointer warning in cd9660_generate_path_table
This function temporarily stores a pointer to an on-stack variable (a
TAILQ_HEAD of a temporary list) into a global variable (*n).

Reviewed by:	imp
Differential Revision:	https://reviews.freebsd.org/D42581
2023-11-14 18:35:44 -08:00
John Baldwin c06962057b makefs cd9660: Unexpand a few TAILQ macros
Reviewed by:	imp, emaste
Differential Revision:	https://reviews.freebsd.org/D42580
2023-11-14 18:35:27 -08:00
Mark Johnston 7e5002e3d6 makefs/zfs: Add a regression test which checks file access permissions
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2023-11-08 07:18:58 -05:00
Mark Johnston 50565cf514 makefs/zfs: Don't set ZFS_NO_EXECS_DENIED in file flags
This flag was leftover from testing and should have been removed.

PR:		274938
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2023-11-08 07:04:12 -05:00
Mark Johnston ef067b5799 makefs/zfs: Update dd_used_breakdown after writing the MOS
Fixes:	96092bf910 ("makefs/zfs: Fill out dd_used_breakdown fields in DSL directories")
Sponsored by:	The FreeBSD Foundation
2023-10-23 12:29:20 -04:00
Mark Johnston 9f9d18d61e makefs/zfs: Add a regression test which checks the 'used*' properties
PR:		274613
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2023-10-23 11:15:00 -04:00
Mark Johnston 96092bf910 makefs/zfs: Fill out dd_used_breakdown fields in DSL directories
This is required for the "used", "usedds" and "usedchild" dataset
properties to be displayed.

PR:		274613
Reported by:	Mike Tancsa <mike@sentex.net>
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2023-10-23 11:12:14 -04:00
Dimitry Andric 8a77bc5e1b makefs/zfs: fix build with gcc 12 after b5a2bf512d
Due to integer promotion rules, dn_nlevels (uint8_t) gets promoted to a
plain int, resulting in -Werror warnings with gcc 12:

  In file included from /workspace/src/usr.sbin/makefs/zfs.c:35:
  /workspace/src/usr.sbin/makefs/zfs.c: In function '_dnode_cursor_flush':
  /workspace/src/usr.sbin/makefs/zfs.c:684:23: error: comparison of integer expressions of different signedness: 'unsigned int' and 'int' [-Werror=sign-compare]
    684 |         assert(levels <= c->dnode->dn_nlevels - 1);
        |                       ^~
  /workspace/src/usr.sbin/makefs/zfs.c:691:27: error: comparison of integer expressions of different signedness: 'unsigned int' and 'int' [-Werror=sign-compare]
    691 |                 if (level == c->dnode->dn_nlevels - 1) {
        |                           ^~
  /workspace/src/usr.sbin/makefs/zfs.c: In function 'dnode_cursor_next':
  /workspace/src/usr.sbin/makefs/zfs.c:739:41: error: comparison of integer expressions of different signedness: 'unsigned int' and 'int' [-Werror=sign-compare]
    739 |                 for (levels = 0; levels < c->dnode->dn_nlevels - 1; levels++) {
        |                                         ^

Fix it by subtracting 1U instead of 1.

Fixes:		b5a2bf512d
MFC after:	3 days
2023-09-30 20:27:20 +02:00
Mark Johnston b6da7f2fd1 makefs/zfs tests: Add a test case which appends to a file after import
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2023-09-28 16:36:12 -04:00
Mark Johnston ef20cd33d3 makefs/zfs: Ensure that the last block of a file has the right size
When copying a file's contents into the pool, a loop copies blocks of
the maximum size (128KB), allocating space from the vdev for each block.
The space allocator rounds up to the nearest block size, but for files
larger than the maximum size, this can result in the last block having a
smaller logical size than the rest of the blocks belonging to that
dnode.  This violates some ZFS invariants when appending to that file.

Modify fs_populate_file() to use the same block size for the final
block.

Reported by:	cperciva
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2023-09-28 16:16:17 -04:00
Mark Johnston 60c95af88f makefs/zfs: Remove a nonsensical comment
When populating files, makefs needs to copy their contents into
userspace in order to compute a checksum, so copy_file_range(2) is out
of the question.  Though, it could possibly be used when building other
types of filesystems.

MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2023-09-27 19:03:26 -04:00
Mark Johnston 87534f95dd makefs/zfs tests: Run zdb prior to importing makefs-created pools
zdb contains quite a few assertions that can help catch bugs.

MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2023-09-27 12:00:00 -04:00
Mark Johnston 46402fd216 makefs/zfs: Set the objset block pointer fill field correctly
It is supposed to contain the number of objects allocated from the set,
excluding the meta dnode.

Reported by:	mav
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2023-09-27 12:00:00 -04:00
Mark Johnston b5a2bf512d makefs/zfs: Use unsigned integers for indirect block level counts
No functional change intended.

MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2023-09-27 12:00:00 -04:00
Greg Lehey 915af88322 Explain the -d option. 2023-09-26 15:05:16 +10:00
Stéphane Rochoy d26c1a0f8b makefs: Handle missing link keyword in mtree spec
PR:		241178
Reviewed by:	emaste
MFC after:	3 days
Pull Request:	https://github.com/freebsd/freebsd-src/pull/850
2023-09-21 12:49:11 -04:00
Ed Maste 84696cf10c makefs: remove ISO9660 Archimedes extension support
We have no need to create ISO images with RISC OS specific metadata.

Reviewed by:	imp
Relnotes:	yes
Obtained from:	OpenBSD a435da44c07f
Sponsored by:	The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39344
2023-09-20 07:37:19 -04:00
Ed Maste 90aad5d3e4 makefs: clean up some leftovers after $FreeBSD$ tag removal
Fixes: 2a63c3be15
2023-08-16 16:24:37 -04:00
Warner Losh fa9896e082 Remove $FreeBSD$: two-line nroff pattern
Remove /^\.\\"\n\.\\"\s*\$FreeBSD\$$\n/
2023-08-16 11:55:10 -06:00
Warner Losh d0b2dbfa0e Remove $FreeBSD$: one-line sh pattern
Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/
2023-08-16 11:55:03 -06:00
Warner Losh 1d386b48a5 Remove $FreeBSD$: one-line .c pattern
Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/
2023-08-16 11:54:42 -06:00
Warner Losh 2a63c3be15 Remove $FreeBSD$: one-line .c comment pattern
Remove /^/[*/]\s*\$FreeBSD\$.*\n/
2023-08-16 11:54:29 -06:00
Warner Losh 42b388439b Remove $FreeBSD$: one-line .h pattern
Remove /^\s*\*+\s*\$FreeBSD\$.*$\n/
2023-08-16 11:54:23 -06:00
Warner Losh b3e7694832 Remove $FreeBSD$: two-line .h pattern
Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/
2023-08-16 11:54:16 -06:00
Elyes Haouas b152ff4c1e makefs: Fix typos
Signed-off-by: Elyes Haouas <ehaouas@noos.fr>
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/653
2023-06-01 20:33:41 -06:00
Chuck Silvers d464a7698d ffs: restore backward compatibility of newfs and makefs with older binaries
The previous change to CGSIZE had the unintended side-effect of allowing
newfs and makefs to create file systems that would fail validation when
examined by older commands and kernels, by allowing newfs/makefs to pack
slightly more blocks into a CG than those older binaries think is valid.
Fix this by having newfs/makefs artificially restrict the number of blocks
in a CG to the slightly smaller value that those older binaries will accept.
The validation code will continue to accept the slightly larger value
that the current newfs/makefs (before this change) could create.

Fixes:		0a6e34e950
Reviewed by:	mckusick
MFC after:	3 days
Sponsored by:	Netflix
2023-05-29 19:26:28 -07:00
Mark Johnston 14c5cf3a16 makefs/zfs: Avoid generating a GUID of zero 2023-05-26 15:38:08 -04:00
Kirk McKusick 0a6e34e950 Fix size differences between architectures of the UFS/FFS CGSIZE macro value.
The cylinder group header structure ended with `u_int8_t cg_space[1]'
representing the beginning of the inode bitmap array. Some architectures
like the i386 rounded this up to a 4-byte boundry while other
architectures like the amd64 rounded it up to an 8-byte boundry.
Thus sizeof(struct cg) was four bytes bigger on an amd64 machine
than on an i386 machine. If a filesystem created on an i386 machine
was moved to an amd64 machine, the size of the cylinder group
calculated by the CGSIZE macro would appear to grow by four bytes.
Filesystems whose cylinder groups were exactly equal to the block
size on an i386 machine would appear to have a cylinder group that
was four bytes too big when moved to an amd64 machine. Note that
although the structure appears to be too big, it in fact is fine.
It is just the calaculation of its size that is in error.

The fix is to remove the cg_space element from the cylinder-group
structure so that the calculated size of the structure is the same
size on all architectures.

Reported by:  Tijl Coosemans
Tested by:    Tijl Coosemans and Peter Holm
MFC after:    1 week
Sponsored by: The FreeBSD Foundation
2023-05-15 12:57:15 -07:00
Warner Losh b61a573019 spdx: The BSD-2-Clause-NetBSD identifier is obsolete, drop -NetBSD
The SPDX folks have obsoleted the BSD-2-Clause-NetBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with:		pfg
MFC After:		3 days
Sponsored by:		Netflix
2023-05-12 10:44:04 -06:00
Warner Losh 4d846d260e spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD
The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with:		pfg
MFC After:		3 days
Sponsored by:		Netflix
2023-05-12 10:44:03 -06:00
Simon J. Gerraty 188bf8876f Build host tools on non-FreeBSD host with DIRDEPS_BUILD
Add Makefile.depend.options to libnetbsd, mkimg and makefs
to ensure libegacy is build if needed.

targets/pseudo/stage/Makefile avoid the need for mtree
the staging process creates target dirs as needed anyway.

Reviewed by:	stevek
Sponsored by:	Juniper Networks, Inc.
Differential Revision:	https://reviews.freebsd.org/D39757
2023-04-22 13:13:22 -07:00
Ed Maste dd9059b3e9 makefs: set cd9660 Rock Ridge timestamps for . and ..
DOT and DOTDOT entries have special handling, and previously only Rock
Ridge PX (POSIX attributes) entries were attached.  Add TF (timestamp)
entries as well.

PR:		203531
Reported by:	Thomas Schmitt <scdbackup@gmx.net>
Reviewed by:	markj
Sponsored by:	The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39662
2023-04-20 12:20:56 -04:00
Ed Maste 79ead08497 makefs: remove unused variable
Reported by:	Clang
Sponsored by:	The FreeBSD Foundation
2023-04-17 08:22:12 -04:00
Ed Maste 10854bf517 makefs: use unsigned and %u for rock_ridge_move_count
For diff reduction against NetBSD's makefs.  Based on NetBSD git mirror
commit 00991aee8248.

With this change our makefs cd9660 support should be in sync up to
NetBSD commit bdae6c9dc792 ("makefs(8): Nix trailing whitespace.").

Sponsored by:	The FreeBSD Foundation
2023-04-17 08:21:28 -04:00
Ed Maste e5551216d8 makefs: call brelse from bread
This matches NetBSD and rationalizes makefs with the kernel API.

This reverts commit 370e009188.

Reviewed by:	mckusick
Sponsored by:	The FreeBSD Foundation
Obtained from:	NetBSD 0a62dad69f62, 0c4125e1a19f, cb6a5a3575fd
Differential Revision:	https://reviews.freebsd.org/D39070
2023-04-16 15:45:17 -04:00
Ed Maste c753f49f70 makefs: remove unused cd9660 options
Makefs defined "follow-symlinks" and "help" options, but they did
nothing.  Remove them.

Obtained from:	OpenBSD a8f1645688c2
Sponsored by:	The FreeBSD Foundation
2023-04-16 09:38:02 -04:00
Ed Maste af7624ed31 makefs: use %s and __func__ in printf messages
For diff reduction against NetBSD.

Obtained from:	NetBSD 0c4125e1a19f
Sponsored by:	The FreeBSD Foundation
2023-04-11 11:21:36 -04:00
Ed Maste 22db5e5d03 makefs: set the validation entry system type
Obtained from:	NetBSD c3cb4dcc9c43
Sponsored by:	The FreeBSD Foundation
2023-04-10 19:07:14 -04:00
Ed Maste 1d1ffa2efd makefs: use size_t or ssize_t where appropriate
Obtained from:	NetBSD af7bc97830ac
2023-04-10 18:50:27 -04:00
Ed Maste c02e6ca7c2 makefs: simplify filename conversion handling
Obtained from:	OpenBSD 4095a2e01d18
Sponsored by:	The FreeBSD Foundation
2023-04-10 12:03:06 -04:00
Ed Maste d73acb32d3 makefs: remove some unused cd9660 types and defines
Obtained from:	OpenBSD
Sponsored by:	The FreeBSD Foundation
2023-04-10 11:15:34 -04:00
Ed Maste 7bc4ccf3ae makefs: avoid uninitialized memory in root directory date
Move date setting into cd9660_populate_iso_dir_record so there is no
path that leaves it unset.

PR:		203531
Reported by:	Thomas Schmitt <scdbackup@gmx.net>
Reviewed by:	markj
Sponsored by:	The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39258
2023-04-06 13:47:35 -04:00
Christos Margiolis f21faa67ab makefs(8): do not print comma after the last super-block
Reviewed by:	markj, emaste
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D39430
2023-04-05 18:25:19 -04:00
Ed Maste 2f11df6337 makefs: remove CD9660MAXPATH #define
It was used only in constructing the host path that contains file
content, which is not related to anything CD9660-specific.  PATH_MAX is
the appropriate limit.  See OpenBSD commit 299d8950f319.

Obtained from:	OpenBSD
Sponsored by:	The FreeBSD Foundation
2023-04-05 14:05:49 -04:00