Commit graph

382 commits

Author SHA1 Message Date
John Baldwin 0163de282e libsa: Disable -Wdangling-pointer for zfs.c.
GCC 12 warns about a dangling pointer to 'objid' in
zfs_bootenv_initial().  However, this appears to be a false positive
as the pointer to 'objid' is only passed to zfs_lookup_dataset() but
not saved anywhere that outlives the lifetime of the
zfs_bootenv_initial() function.

Reviewed by:	imp, emaste
Differential Revision:	https://reviews.freebsd.org/D37533
2022-12-04 16:27:22 -08:00
Warner Losh a07cef5a73 stand: Add dv_match
On OpenFirmware, and possibly kboot, we use full path names for the
objects that are the 'device'. kboot uses a hack of knowing that all
disk device nodes start with '/dev', but this generalizes it for
OpenFirmware where both 'block' and 'network' devices live in the same
namespace and one must ask the OF node its type to know if this device
type matches.

For drivers that don't specify, the current convention of using
strncmp() is retained. This is done only in devparse(), but everything
uses it directly (or will soon).

Sponsored by:		Netflix
Differential Revision:	https://reviews.freebsd.org/D37554
2022-11-30 15:30:33 -07:00
Warner Losh 33bbe5ddcb stand: parsedev API change: devspec now points to start of full device name
To support more flexible device matching, we now pass in the full
devspec to the parsedev routines. For everything execpt uboot, this is
just a drop in (since everything except uboot and openfirmware always
uses disk...: and/or zfs:, but openfirmware isn't really affected).

uboot we kludge around it by subtracting 4 from where the rest of the
device name starts. This is unforunate, and can compute the address one
before the string. But we never dereference that address. uboot needs
more work, and this is an acceptable UB until that other work happens.

OFW doesn't really use the parsedev routines these days (since none of
the supported device uses this... yet). It too needs more work, but it
needs device matching support first.

Sponsored by:		Netflix
Reviewed by:		delphij
Differential Revision:	https://reviews.freebsd.org/D37553
2022-11-30 15:30:33 -07:00
Warner Losh 66012c8fc4 stand: create devinit
devinit() marches through all the devices, calling the inint routines if
any exist. Replace all the identical copies of this code.

Sponsored by:		Netflix
Differential Revision:	https://reviews.freebsd.org/D37349
2022-11-30 15:30:33 -07:00
Warner Losh bb9f61da17 zfs: Remove devicename_stubs
We no longer need the zfs stubs since we're no longer referencing these
functions outside of zfs.c.

Sponsored by:		Netflix
Differential Revision:	https://reviews.freebsd.org/D37345
2022-11-30 15:30:32 -07:00
Warner Losh 90412431fe stand: make zfs_parsedev static
It's now unreferenced outside of zfs.c.

Sponsored by:		Netflix
Differential Revision:	https://reviews.freebsd.org/D37344
2022-11-30 15:30:32 -07:00
Warner Losh 8337ab69ba stand: For all disk drivers, connect dv_parsedev to disk_parsedev
Sponsored by:		Netflix
Differential Revision:	https://reviews.freebsd.org/D37339
2022-11-30 15:30:32 -07:00
Warner Losh ca0654bad6 stand/zfs: Connect dv_parsedev to zfs_parsedev
Sponsored by:		Netflix
Differential Revision:	https://reviews.freebsd.org/D37348
2022-11-30 15:30:32 -07:00
Warner Losh 781ca0afcd stand: Introduce devparse to parse device / path strings
devparse is now the preferred interface to use to parse device
strings or device:/path strings. It parses the passed in string,
mallocs the device's particular devdesc string and returns the
'remainder' of the device:/path for further processing.

Sponsored by:		Netflix
Differential Revision:	https://reviews.freebsd.org/D37338
2022-11-30 15:30:31 -07:00
Warner Losh a0aad69f95 stand: Introduce new dv_parsedev routine
Allow device classes to define a parsing routine. Most device classes
already have these routines, but there's much duplication in their
use. Define an interface for a common routine to parse an individual
device. By convetion, files have the form "[device:]/path/to/file"
where device is optional (filled in to be the value of currdev)
and it starts with the dv_name field of the device, with the rest
of the name up to the device (typically a unit number, but disks
add partition inforation, and other devices may do artibtrary
otehr things).

Sponsored by:		Netflix
Differential Revision:	https://reviews.freebsd.org/D37337
2022-11-30 15:30:31 -07:00
Warner Losh ba11bc368e stand: Change zfs_parsedev() API
Change the first argument to zfs_parsedev() to be a pointer to a struct
devdesc *. This now gets filled in with a malloc'd structure that's
returned to the caller that the caller is repsonsible for freeing. Most
nplaces in the tree passed in a malloc'd pointer anyway, and this moves
knowledge of zfs_devdesc more firmly into the zfs.c code.

Sponsored by:		Netflix
Differential Revision:	https://reviews.freebsd.org/D37336
2022-11-30 15:30:31 -07:00
Warner Losh 983a18021d stand/zlib: Zlib still uses K&R function definitions
So add ${NO_WDEPRECATED_NON_PROTOTYPE} to the CFLAGS of those
files. This can be removed when we import a zlib that's free of this
anachronism.

Sponsored by:		Netflix
Reviewed by:		jhb
Differential Revision:	https://reviews.freebsd.org/D37516
2022-11-30 11:08:44 -07:00
John Baldwin 69f6399c37 libsa: Add missing GNU-stack annotations to _setjmp.S.
ld.bfd marks the stack as executable for the crt objects due to the
missing annotations which raises a fatal warning starting with version
2.39.
2022-11-22 08:36:40 -08:00
John Baldwin 03bbe1845d stand/libsa: Remove MIPS setjmp() and longjmp(). 2022-11-22 08:36:28 -08:00
Alexander Leidinger f993fff689 Sort list of supported features for more easy handling
in the future.

Reviewed by:	imp
2022-11-10 09:47:23 +01:00
Warner Losh 269865a8ee stand: Update comment about devdesc
How devdesc is used is opaque until much code is read. Give a more
useful description of the theory behind it here.

Sponsored by:		Netflix
2022-11-04 19:46:44 -06:00
Elliott Mitchell 798ea06f07 stand: Nuke double-semicolons
A distinct number of double-semicolons have ended up in FreeBSD.  Take a
pass at getting rid of many of these harmless typos.

Reviewed by: emaste, imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/609
Differential Revision: https://reviews.freebsd.org/D31717
2022-11-02 09:34:54 -06:00
Warner Losh bb3230e40b geli: Move check for DEVT_DISK into geli_probe_and_attach
We only work on DEVT_DISK disks, so move that into the probe to drive
the point home better.

Sponsored by:		Netflix
2022-10-22 19:47:24 -06:00
Warner Losh 787df454c8 stabd/geli: Bail out if you can't get the disks size
If the DIOCGMEDIASIZE ioctl fails, assume the disk doesn't have geli
encryption. While all disks should implement this, fail safe for disks /
partitions that do not.

Sponsored by:		Netflix
2022-10-21 17:39:34 -06:00
Michał Grzelak 7b54d275ef libsa/netif.c: Replace #if with #ifdef
Follow the convention with *_DEBUG macros in stand/libsa/* and replace
"#if" with "#ifdef".

Reviewed by:	kd
Obtained from:	Semihalf
Differential Revision: https://reviews.freebsd.org/D36740
2022-09-29 12:52:02 +02:00
Michał Grzelak c66c6da4f0 libsa/arp.c: Change printf format string
Change printf format string to avoid compilation failure when
ARP_DEBUG macro is defined.

Reviewed by:	imp
Obtained from:	Semihalf
Differential Revision: https://reviews.freebsd.org/D36735
2022-09-29 12:51:14 +02:00
Michał Grzelak ee0d06faa0 libsa/rarp.c: Change casted type and printf format
Change the casted type and printf format string to avoid compilation
failure when RARP_DEBUG macro is defined.

Reviewed by:	imp
Obtained from:	Semihalf
Differential Revision: https://reviews.freebsd.org/D36738
2022-09-29 12:51:14 +02:00
Michał Grzelak 1ed7916188 libsa/rarp.c: Change printf format string
Change printf format string to avoid compilation failure when
RARP_DEBUG macro is defined.

Reviewed by:	imp
Obtained from:	Semihalf
Differential Revision: https://reviews.freebsd.org/D36739
2022-09-29 12:51:14 +02:00
Martin Matuska c7046f76c2 zfs: merge openzfs/zfs@c629f0bf6
Notable upstream pull request merges:
  #13725 Fix BLAKE3 tuneable and module loading on Linux and FreeBSD
  #13756 FreeBSD: Organize sysctls
  #13773 FreeBSD: add kqfilter support for zvol cdev
  #13781 Importing from cachefile can trip assertion
  #13794 Apply arc_shrink_shift to ARC above arc_c_min
  #13798 Improve too large physical ashift handling
  #13799 Revert "Avoid panic with recordsize > 128k, raw sending and
         no large_blocks"
  #13802 Add zfs.sync.snapshot_rename
  #13831 zfs_enter rework
  #13855 zfs recv hangs if max recordsize is less than received
         recordsize

Obtained from:	OpenZFS
OpenZFS commit:	c629f0bf62
2022-09-21 14:17:13 +02:00
Warner Losh 45ad955714 stand: Add driver interface docs
Add some rather bare-bones driver interface docs.

Sponsored by:		Netflix
Suggestions by:		rpokala
Reviewed by:		pauamma
Differential Revision:	https://reviews.freebsd.org/D35912
2022-09-09 17:25:59 -06:00
Michael Gmelin 0eb736c0f6 stand: Unbreak FAT32 in loader
This corrects an issue introduced in b4cb3fe0e3, where a freshly
allocated `DOS_FS` structure would not be initialized properly before
use in `dos_open`.

In case of FAT32 file systems, this would leave `fs->dirents`
uninitialized and - depending on its content and due to checks in
`parsebs` - prevent mounting the file system successfully.

This particularily impacted the EFI loader, as it was sometimes not
able to read files from a FAT32-formatted EFI partition, including
LoaderEnv (`/efi/freebsd/loader.env`).

Accepted by:	imp
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D36482
2022-09-08 02:16:34 +02:00
Kornel Dulęba 66c73af7ea stand: Parse BIOS revision from SMBIOS
Add a smbios.bios.revision kenv, which contains the system BIOS revision
as defined in SMBIOS specification, section 3.3.1.
Since the revision is stored in two separate byte fields,
the smbios_setenv helper can't be used.
Read and construct the kenv manually instead.

Approved by:	mw(mentor)
Sponsored by:	Stormshield
Obtained from:	Semihalf
Differential Revision:	https://reviews.freebsd.org/D36413
2022-09-02 13:10:32 +02:00
Warner Losh 6d645da0d4 stand: Search less agressively for UFS super block
The boot loader should look in the standard places for the UFS
superblock, but not go too far into the speculative realm. Supress
errors about hash being invalid, which will allow us to boot, even when
the superblock looks good, but fails the hash test. This defers any
policy decisions about booting and/or recovery to userland. This also
has the side effect of eliminating some rather spammy messages when UFS
searches devices with filesystems that are not UFS...

Sponsored by:		Netflix
Reviewed by:		mckusick
Differential Revision:	https://reviews.freebsd.org/D36253
2022-08-18 06:41:11 -06:00
Toomas Soome d98de74405 loader: zfs reader should only store devdesc in f_devdata
Use d_opendata for device specific data.

PR:		265825
Reviewed by:	imp
Differential Revision:	https://reviews.freebsd.org/D36202
2022-08-15 21:07:23 +03:00
Kirk McKusick e688661642 Move the ability to search for alternate UFS superblocks from fsck_ffs(8)
into ffs_sbsearch() to allow use by other parts of the system.

Historically only fsck_ffs(8), the UFS filesystem checker, had code
to track down and use alternate UFS superblocks. Since fsdb(8) used
much of the fsck_ffs(8) implementation it had some ability to track
down alternate superblocks.

This change extracts the code to track down alternate superblocks
from fsck_ffs(8) and puts it into a new function ffs_sbsearch() in
sys/ufs/ffs/ffs_subr.c. Like ffs_sbget() and ffs_sbput() also found
in ffs_subr.c, these functions can be used directly by the kernel
subsystems. Additionally they are exported to the UFS library,
libufs(8) so that they can be used by user-level programs. The new
functions added to libufs(8) are sbfind(3) that is an alternative
to sbread(3) and sbsearch(3) that is an alternative to sbget(3).
See their manual pages for further details.

The utilities that have been changed to search for superblocks are
dumpfs(8), fsdb(8), ffsinfo(8), and fsck_ffs(8). Also, the prtblknos(8)
tool found in tools/diag/prtblknos searches for superblocks.

The UFS specific mount code uses the superblock search interface
when mounting the root filesystem and when the administrator doing
a mount(8) command specifies the force flag (-f). The standalone UFS
boot code (found in stand/libsa/ufs.c) uses the superblock search
code in the hope of being able to get the system up and running so
that fsck_ffs(8) can be used to get the filesystem cleaned up.

The following utilities have not been changed to search for
superblocks: clri(8), tunefs(8), snapinfo(8), fstyp(8), quot(8),
dump(8), fsirand(8), growfs(8), quotacheck(8), gjournal(8), and
glabel(8). When these utilities fail, they do report the cause of
the failure. The one exception is the tasting code used to try and
figure what a given disk contains. The tasting code will remain
silent so as not to put out a slew of messages as it trying to taste
every new mass storage device that shows up.

Reviewed by: kib
Reviewed by: Warner Losh
Tested by:   Peter Holm
Differential Revision: https://reviews.freebsd.org/D36053
Sponsored by: The FreeBSD Foundation
2022-08-13 12:43:40 -07:00
Warner Losh 4f0c9b76cf stand: Only compile decompression routines
We don't need the compress rotuines, nor zstd_opt.c. Remove them.
Expand the number of places we omit code for IN_LIBSA (which are FreeBSD
specific). Due to the agressive optimization, though, this doesn't
reduce the size of the loader. It does reduce the number of 'false
positives' for places to omit to reduce the size as well as reducing the
build time slightly.

Sponsored by:		Netflix
Reviewed by:		tsoome, delphij
Differential Revision:	https://reviews.freebsd.org/D36145
2022-08-12 21:48:18 -06:00
Warner Losh 1482113008 stand: Compile out the extensive superblock diagnostic messages for BIOS loader
The BIOS loader operates in a very constrained environment. The messages
for the super block integrity tests take up about 12k of space. Compile
them out for the BIOS loader, while leaving it intact for all other
loaders that aren't space constrained. These aren't used in the 'super
tiny' *boot* programs, so no adjustment is needed there.

We reply on the fact that (a) i386 doesn't support 32-bit UEFI booting
and (b) LIBSA_CPUARCH is "i386" when building on both i386 and when
we're building the 32-bit libsa32 library.

This saves about 12k of space for this constrained envrionment and will
take a bit of the pressure off some machines where the loader has grown
too big for their BIOS (see comments in i386/loader/Makefile for
details).

Sponsored by:		Netflix
Reviewed by:		mckusick
Differential Revision:	https://reviews.freebsd.org/D36175
2022-08-12 21:48:17 -06:00
Warner Losh edb26097cb stand: Replace zfs_fmtdev with generic devformat()
Sponsored by:		Netflix
Differential Revision:	https://reviews.freebsd.org/D35973
2022-08-11 10:27:16 -06:00
Warner Losh 654b7837a6 stand: For zfs, set dv_fmtdev to zfs_fmtdev
Add a generic way to get the string representation of a zfs device / mount.

Sponsored by:		Netflix
Reviewed by:		tsoome (prior version)
Differential Revision:	https://reviews.freebsd.org/D35923
2022-08-11 10:27:16 -06:00
Warner Losh 0b3a4a588f stand: Use devformat instead of disk_devfmt
Use devformat instead of disk_devfmt. This allows us to avoid knowing
the details of the device that's underneath us. Remove disk.h include
and the -I${LDRSRC} from the build of ufs.c since they are no longer
needed.

Sponsored by:		Netflix
Differential Revision:	https://reviews.freebsd.org/D35922
2022-08-11 10:27:16 -06:00
Warner Losh ec9f3e776f stand: Use devformat rather than disk_devfmt
Fix layering violation and use devformat to get the string
representation of the device to see if we're mounted yet or not. Remove
added include to pickup disk.h.

Sponsored by:		Netflix
Reviewed by:		tsoome (prior version)
Differential Revision:	https://reviews.freebsd.org/D35919
2022-08-11 10:27:16 -06:00
Warner Losh ad759c7352 stand: Add disk_fmtdev for dv_fmtdev for all the disk devices
All of the archsw fmtdev functions treat DEVT_DISK as a call to
disk_fmtdev. Set all disks' dv_fmtdev to disk_fmtdev so devformat
will return the same thing.

Sponsored by:		Netflix
Reviewed by:		tsoome (prior version)
Differential Revision:	https://reviews.freebsd.org/D35917
2022-08-11 10:27:16 -06:00
Warner Losh dc472f6702 stand: Add devformat to return formatted string for a device
Use dv_fmtdev to return a formatted string for a device. If this is a
null pointer, return the device name and unit followed by a colon (eg
disk3:).

Sponsored by:		Netflix
Reviewed by:		tsoome (prior version)
Differential Revision:	https://reviews.freebsd.org/D35916
2022-08-11 10:27:16 -06:00
Warner Losh 4d4b1a298c stand: Add dv_devfmt to return a string represenation of the device
Add a new pointer, dv_devfmt, to allow devices to format themselves. We
will use this to simplify many of the fmtdev functions in the tree as
they are all almost the same, or all are isomorphic to each other.

Sponsored by:		Netflix
Reviewed by:		tsoome (prior version)
Differential Revision:	https://reviews.freebsd.org/D35915
2022-08-11 10:27:16 -06:00
Warner Losh d2d4e1271b stand: Change zfs_fmtdev to take a struct devdesc *
Sponsored by:		Netflix
Differential Revision:	https://reviews.freebsd.org/D35974
2022-08-11 10:27:15 -06:00
Warner Losh c32dde3166 stand: Change disk_fmtdev to take a struct devdesc *
We do a number of games with ploymorphism for different types struct
*devdesc. Adjust one place that this affects to take the address of the
base class (most others have void * at the moment). This is more type
safe than a bare void *.

Sponsored by:		Netflix
Differential Revision:	https://reviews.freebsd.org/D35914
2022-08-11 10:27:15 -06:00
Warner Losh f863970a82 stand: Reduce number of files that need -I${LDRSRC}
geliboot.c and geliboot_crypto.c don't need anything from stand/common,
so remove them from the list of things to add it.

Sponsored by:		Netflix
Differential Revision:	https://reviews.freebsd.org/D35921
2022-08-03 11:24:38 -06:00
Warner Losh 4333168b1d stand: Remove unneeded include from geli
bootstrap.h isn't needed at all by geil, so remove it.

Sponsored by:		Netflix
Reviewed by:		tsoome (earlier version)
Differential Revision:	https://reviews.freebsd.org/D35920
2022-08-03 11:24:38 -06:00
Warner Losh a23c26b2fe stand: use snprintf here
This code was written prior to snprintf being in the then libstand (now
libsa). Since we have it, use it for extra safety. The code already
tries to be safe, but since we have snprintf as well, the added layer of
protection will suffice. The current code reserves 16 bytes (plus a NUL)
at the end for worst case of inet_ntoa, which is still a little
pessimal, but safe from overflow.

Sponsored by:		Netflix
Reviewed by:		tsoome
Differential Revision:	https://reviews.freebsd.org/D35102
2022-08-03 11:24:38 -06:00
Kirk McKusick b21582ee03 Add a flags parameter to the ffs_sbget() function that reads UFS superblocks.
Rather than trying to shoehorn flags into the requested superblock
address, create a separate flags parameter to the ffs_sbget()
function in sys/ufs/ffs/ffs_subr.c. The ffs_sbget() function is
used both in the kernel and in user-level utilities through export
to the sbget() function in the libufs(3) library (see sbget(3)
for details). The kernel uses ffs_sbget() when mounting UFS
filesystems, in the glabel(8) and gjournal(8) GEOM utilities,
and in the standalone library used when booting the system
from a UFS root filesystem.

The ffs_sbget() function reads the superblock located at the byte
offset specified by its sblockloc parameter. The value UFS_STDSB
may be specified for sblockloc to request that the standard
location for the superblock be read.

The two existing options are now flags:

UFS_NOHASHFAIL will note if the check hash is wrong but will still
   return the superblock. This is used by the bootstrap code to
   give the system a chance to come up so that fsck can be run to
   correct the problem.

UFS_NOMSG indicates that superblock inconsistency error messages
   should not be printed. It is used by programs like fsck that
   want to print their own error message and programs like glabel(8)
   that just want to know if a UFS filesystem exists on a partition.

One additional flag is added:

UFS_NOCSUM causes only the superblock itself to be returned, but does
   not read in any auxiliary data structures like the cylinder group
   summary information. It is used by clients like glabel(8) that
   just want to check for possible filesystem types. Using UFS_NOCSUM
   skips the superblock checks for csum data which allows superblocks
   that have corrupted csum data to be read and used.

The validate_sblock() function checks that the superblock has not
been corrupted in a way that can crash or hang the system. Unless
the UFS_NOMSG flag is specified, it will print out any errors that
it finds. Prior to this commit, validate_sblock() returned as soon
as it found an inconsistency so would print at most one message.
It now does all its checks so when UFS_NOMSG has not been specified
will print out everything that it finds inconsistent.

Sponsored by: The FreeBSD Foundation
2022-07-30 22:51:38 -07:00
Warner Losh 60cb4f9a8e stand: tftp.c doesn't need bootinfo.h
tftp.c includes bootinfo.h, but doesn't need it. Remove it, and remove
the -Istand/common from CFALGS since that's the only reason we had it.

Sponsored by:		Netflix
Reviewed by:		tsoome
Differential Revision:	https://reviews.freebsd.org/D35928
2022-07-27 09:04:13 -06:00
Warner Losh e98f952c82 stand: Make sure nobody has a NULL pointer for dv_cleanup
dv_cleanup is specified almost everywhere. Use nullsys instead of NULL
to indicate 'do nothing'. Also, be consistent in trailing commas that
were missing before.

Sponsored by:		Netflix
Reviewed by:		tsoome
Differential Revision:	https://reviews.freebsd.org/D35913
2022-07-27 09:04:13 -06:00
Warner Losh 079f02e8c0 stand: Small comment correction
While in theory, once upon a time, dv_type was arch specific, that's
never been the case in FreeBSD (and certaintly isn't in the surviving
drivers). Remove that notation.

Sponsored by:		Netflix
Reviewed by:		tsoome
Differential Revision:	https://reviews.freebsd.org/D35911
2022-07-27 09:04:12 -06:00
Warner Losh 4932a6e41d stand: Create DEV_NAMLEN for the length of a device name
Rather than hard coding 8 for the device name length, create a #define for it.

Sponsored by:		Netflix
Reviewed by:		tsoome
Differential Revision:	https://reviews.freebsd.org/D35910
2022-07-27 09:04:12 -06:00
Warner Losh 16ebddc9a7 stand/zfs: Update comment about blake3_impl_hack.c
Fix a typo by reworking the comment for blake3_impl_hack.c and amplify
the nature of the hack and its temporary reason for existing.

Sponsored by:		Netflix
Reviewed by:		delphij
Differential Revision:	https://reviews.freebsd.org/D35895
2022-07-24 16:53:36 -06:00
Warner Losh 1306a5dc07 stand/libsa: zfs use standard ZFS_EARLY stuff
Now that the minor issues preventing zfs.c from using CFLAGS_EARLY have
been fixed, use that mechanism like everything else that needs the
OpenZFS spl headers. This simplifies things somewhat. Update comments to
document why zfs.c is still special, though in different ways.

Note: We also use the fact that NEED_SOLARIS_BOOLEAN is only defined in
an environment where the solaris compat boolean stuff will be defined
prior to this point (eg, when we're building zfs.c in libsa), but not in
other environments (like when we're building mkimage and stand-alone
boot loaders that don't use libsa). These latter uses should be changed
to use the same ZFS compile env, but aren't as part of this commit.
This has to be done in the same change as the ZFS_EARLY change to not
break zfs.c building for one commit affecting bisectabiltiy.

Sponsored by:		Netflix
Reviewed by:		tsoome, delphij
Differential Revision:	https://reviews.freebsd.org/D35894
2022-07-24 16:53:36 -06:00
Warner Losh 09ace5cefb stand/zfs: Limit flags further for ZFS
Constrain CFLAGS for ZFS: don't add anything globally. Add the includes
to only the files that need them. Add -DHAS_ZSTD_ZFS to zfs.c (which
includes zfsimpl.c which includes zfssubr.c both of which need this
defined). Also add it to efi/boot1/Makefile since zfs_module.c also
includes zfsimple.c.

Sponsored by:		Netflix
Reviewed by:		tsoome
Differential Revision:	https://reviews.freebsd.org/D35887
2022-07-24 16:53:35 -06:00
Warner Losh b7625c2c2c stand: Use c99 structure initialization for gzipfs_fsops
Sponsored by:		Netflix
2022-07-24 16:53:35 -06:00
Warner Losh 9450d9c4e5 stand: Use c99 structure initialization for splitfs_fsops
Sponsored by:		Netflix
2022-07-24 16:53:35 -06:00
Warner Losh e7d045aa2b stand: Use c99 structure initialization for pkgfs_fsops
Sponsored by:		Netflix
2022-07-24 16:53:35 -06:00
Warner Losh 90a7e556a4 stand: Use c99 structure initialization for nfs_fsops
Sponsored by:		Netflix
2022-07-24 16:53:35 -06:00
Warner Losh 0ad8a113b7 stand: Use c99 structure initialization for bzipfs_fsops
Sponsored by:		Netflix
2022-07-24 16:53:35 -06:00
Warner Losh 12a41918a2 stand: Use c99 structure initialization for ext2fs_fsops
Sponsored by:		Netflix
2022-07-24 16:53:35 -06:00
John Baldwin bb7ddd077c stand geli: Restore include path to LDRSRC.
Various GELI sources need bootstrap.h and disk.h. In theory they
shouldn't need anything outside of libsa, but disk.h and bootstrap.h are
currently required.

This fixes the build with MK_LOADER_ZFS=no.

Obtained from: CheriBSD
Fixes: eaf7aabddc stand: geli CFLAGS tightening
Sponsored by: DARPA
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D35861
2022-07-22 10:08:46 -06:00
John Baldwin d1fb0e1dc8 stand libsa: Restore include path to LDRSRC for disk.h for filesystems.
In theory they shouldn't need anything outside of libsa, but disk.h and
bootstrap.h are currently required. Future work wil address this issue.

This fixes the build with MK_LOADER_ZFS=no. ZFS's Makefile.inc adds
these flags globally to CFLAGS when it should not. This masked the
problem because the tools/boot/universe.sh didn't build MK_LOADER_ZFS=no
as part of its regressions. Future work will also fix this.

Obtained from: CheriBSD
Fixes: 84bf2bbbec stand: constrain zlib/gzip CFLAGS better
Sponsored by: DARPA
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D35860
2022-07-22 10:08:40 -06:00
Warner Losh 4773d3425e stand: Add comments on ZFS build
Add comments describing the weird nesting things we have to do to live
inside the ZFS world. Also fix a stale comment by moving nvlist.c to an
early user.

Sponsored by:		Netflix
2022-07-08 23:01:59 -06:00
Warner Losh 75ad24775b stand: Add blake3 support to boot loader
Add the necessary glue to get blake3 building for the boot loaded as
well as connected to the ZFS system so it is useful.

On some platforms, we create references to blake3_sse2_impl and
blake3_sse41_impl ops structs to utilize SIMD. These aren't present on
x86 (since we dind't ask for them), but are on aarch64 with no
implementation. Since we don't want SIMD in the boot loader, have these
all return 'unsupported' always. This should be fixed upstream to allow
more flexibility in this selection, but for now we use this hack to not
modify the sys/contrib/openzfs with difficult to maintain hacks while
an upstreamable solution is found.

tsoome@ did the implementation bits in sys/cddl/boot, and I did the
Makefile work and the aweful blake3_impl_hack.c.

Co-author:		tsoome@freebsd.org
Sponsored by:		Netflix
Reviewed by:		kevans (earlier version)
Differential Revision:	https://reviews.freebsd.org/D35750
2022-07-08 22:57:59 -06:00
Warner Losh df76778ad5 stand: delete zstd_shim.c
We no longer need to use this to get zfs_zstd.c compiling, so delete it.

Sponsored by:		Netflix
Reviewed by:		kevans
Differential Revision:	https://reviews.freebsd.org/D35750
2022-07-08 22:49:56 -06:00
Warner Losh 119c786f7b stand: Compile zfs_zstd.c directly now
Now that we have the ability to work around all the issues that
zstd_stub.c worked around, compile zfs_std.c directly.

Sponsored by:		Netflix
Reviewed by:		kevans
Differential Revision:	https://reviews.freebsd.org/D35750
2022-07-08 22:49:56 -06:00
Warner Losh b0a337d501 stand: Fix inclusion of sys/blake3.h
sys/blake3.h is not safe to include in a standalone environment. It
assumes, unwisely, that there's only kernel or userland. The userland
choice is bad due to imperfections in how we handle stdlib.h in this
environment (we wind up including the host's stdlib.h, it expects a
standard FreeBSD setup which the gymnastics we've done to create a
OpenSolaris/Illumos-like environment interfere with). Thankfully, in
this case we can just pretend to be the kernel for a little bit by
defining _KERNEL and the undefing it after we exit.

Sponsored by:		Netflix
Reviewed by:		kevans
Differential Revision:	https://reviews.freebsd.org/D35750
2022-07-08 22:49:55 -06:00
Warner Losh faf25f48d6 stand: Work around upstream issues in the standalone environment
There's a number of issues with including zfs_context.h from a
standalone environment. First, sys/uio_imp.h isn't at all safe for this
environment, so define its guard #defines so that its contents are
skipped. Next, there's a problem including string.h to get the mem*
routines, so just define them here. ZFS_MODULE_PARAM_ARGS isn't defined
properly. I had wanted to define it when I was upstreaming changes to
include/os/freebsd/spl/sys/zfs_context.h, but they ran into resistance
so I'm defining that here now (it is also defined in zstd_shim.c, but
that will disappear once the issues it works around are
cleared). Finally, sys/sysmacros.h has to be included now before
sys/atomic.h, but upstream includes it after so include it here so that
the guards make the out-of-order includes in upstream irrelevant.

Sponsored by:		Netflix
Reviewed by:		kevans
Differential Revision:	https://reviews.freebsd.org/D35750
2022-07-08 22:49:55 -06:00
Warner Losh 87d3aa5397 stand: For ZFS build, add new directory
OpenZFS is based on having a number of layers of include files that
define things and include the next layer. Insert a later at the start
for files in libsa/zfs/spl so we can override issues with upstream
OpenZFS while we work to get the issues corrected upstream.

Sponsored by:		Netflix
Reviewed by:		kevans
Differential Revision:	https://reviews.freebsd.org/D35750
2022-07-08 22:49:55 -06:00
Warner Losh 37dabb06e2 stand: Add comment about CFLAGS pollution
Add a note that veriexec / bearssl builds will pollute the CFLAGS in a
way that's somewhat hard to fix, so I'm just noting it for now.

Sponsored by:		Netflix
2022-07-08 11:47:37 -06:00
Warner Losh 84bf2bbbec stand: constrain zlib/gzip CFLAGS better
Define ZLIB_CFLAGS and use it only for the sources that are in ZLIB or
that include it.

Sponsored by:		Netflix
2022-07-08 11:47:37 -06:00
Warner Losh 59a4cfe03c stand: Confine BZIP defines to bzip files
Sponsored by:		Netflix
2022-07-08 11:47:37 -06:00
Warner Losh eaf7aabddc stand: geli CFLAGS tightening
Only add -DWEAK_REFS to sha256.c and sha512.c instead of
everything. Remove redundant include that's not needed.
Minor formatting tweak.

Sponsored by:		Netflix
2022-07-08 11:47:36 -06:00
Warner Losh 451ac17a50 stans: Narrow the scope of includes and other flags
CFLAGS+= here affects *ALL* libsa files being built. However, this is
only needed for zfs.c, so define it only for this. Also, use the defines
from defs.mk. Move all the zfs.c include hacks together. Also, move the
-Wformat -Wall warnings that were added to CFLAGS+= to the individual
files instead for the same reason.

Sponsored by:		Netflix
2022-07-08 11:47:36 -06:00
Warner Losh 0e5ac0ad03 stand: Use a for loop for all the common ZSTD files
Also add comments about why we're not using the BMI instructions when
vailable.

Sponsored by:		Netflix
2022-07-08 11:47:36 -06:00
Warner Losh 285f6ab665 stand: Separate out ZSTD sources from ZFS sources
Sponsored by:		Netflix
2022-07-08 11:47:36 -06:00
Warner Losh f424b167a3 stand: Don't reuse ZFSSRC
ZFSSRC is the top level directory where the ZFS sources come from. Don't
reuse it for a list of ZFS sources. Instead, use ZFS_SRC

Sponsored by:		Netflix
2022-07-08 11:47:36 -06:00
Toomas Soome e417249016 loader: GELI encrypted disk should still use device name disk
geli_probe_and_attach() does pick geli_devsw structure for
encrypted disks, the implementation depends on device
name "disk" when device type is DEVT_DISK, but geli_devsw is
setting name field "gelidisk".

PR:		264282
Submitted by:	yamagi@yamagi.org
Reported by:	yamagi@yamagi.org
MFC after:	2 weeks
2022-06-20 10:10:14 +03:00
Gordon Bergling c44b5e090d stand: Fix a common typo in source code comments
- s/independant/independent/

MFC after:	3 days
2022-06-05 09:49:51 +02:00
Martin Matuska e3aa18ad71 zfs: merge openzfs/zfs@b9d98453f
Notable upstream pull request merges:
  #12321 Fix inflated quiesce time caused by lwb_tx during zil_commit()
  #13244 zstd early abort
  #13360 Verify BPs as part of spa_load_verify_cb()
  #13452 More speculative prefetcher improvements
  #13466 Expose zpool guids through kstats
  #13476 Refactor Log Size Limit
  #13484 FreeBSD: libspl: Add locking around statfs globals
  #13498 Cancel in-progress rebuilds when we finish removal
  #13499 zed: Take no action on scrub/resilver checksum errors
  #13513 Remove wrong assertion in log spacemap

Obtained from:	OpenZFS
OpenZFS commit:	b9d98453f9
2022-06-03 18:17:53 +02:00
Toomas Soome 9cd45772a4 libsa: mark head_errlog feature supported.
head_errlog is new format for errlog, but we do not really
use errlog, so we can just mark it supported, to enable reading
from pool.

MFC after:	1 week
2022-05-21 09:13:06 +03:00
Mark Johnston e097436cb2 libsa: Make the nvlist implementation more self-contained
Move declarations into a new nvlist.h rather than putting everything in
libzfs.h.  This makes this nvlist code easier to reuse elsewhere.  In
particular, the nvlist implementation in sys/contrib/libnv does not
provide XDR encoding, but this is needed when reading from or writing to
ZFS pools.

Also:
- Remove references to boolean_t.  It has to be a 32-bit int here, so
  just reference the underlying type.
- Add includes needed when compiling the nvlist code outside of stand/.

No functional change intended.

Reviewed by:	tsoome
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D35255
2022-05-20 10:35:19 -04:00
Mark Johnston bcc3148c70 libsa: Fix a bug in nvlist creation
When adding an entry to an nvlist, the data buffer might need to be
resized.  When this happens, the XDR encoder's notion of the buffer size
also needs to be updated, otherwise the operation may erroneously fail.

Reviewed by:	tsoome, imp
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D35254
2022-05-20 10:34:43 -04:00
Warner Losh 70b5c4ff48 stand: Install libsa.3
Turns out there is a libsa.3. It's a bit out of date, but we reference
it in a number of places so we should install it. We need to do the DO32
dance because this Makefile is included twice and we don't want it
installing twice.

Sponsored by:		Netflix
2022-04-30 12:52:19 -06:00
Warner Losh bd001d86d6 stand: s/libstand/libsa/g to catch up with rename
We renamed libstand to libsa years ago with the move from sys/boot to
stand. Catch up in the comments.

Sponsored by:		Netflix
2022-04-30 07:34:19 -06:00
Kyle Evans 914dc91d12 stand: zfs: handle holes at the tail end correctly
This mirrors dmu_read_impl(), zeroing out the tail end of the buffer and
clipping the read to what's contained by the block that exists.

This fixes an issue that arose during the 13.1 release process; in
13.1-RC1 and later, setting up GELI+ZFS will result in a failure to
boot.  The culprit is this, which causes us to fail to load geom_eli.ko
as there's a residual portion after the single datablk that should be
zeroed out.

PR:		263407
Reviewed by:	tsoome
MFC after:	3 days
Differential Revision:	https://reviews.freebsd.org/D35019
2022-04-21 14:57:24 -05:00
Gordon Bergling 746cc38ec3 libsa: Fix a typo in a panic message
- s/occured/occurred/

MFC after:	3 days
2022-04-20 12:56:52 +02:00
Gordon Bergling 9cd75b5588 stand: Fix a common typo in source code comments
- s/existance/existence/

MFC after:	3 days
2022-03-28 19:34:30 +02:00
Toomas Soome 9633c3d874 WITHOUT_BOOT build option appears to include stand, failing on libsa
building libsa needs to use -I${LDRSRC} for some files.

PR:		260083
Submitted by:	Ivan Rozhuk
MFC:		1 day
2022-03-17 19:42:05 +02:00
Martin Matuska c03c5b1c80 zfs: merge openzfs/zfs@a86e08941 (master) into main
Notable upstream pull request merges:
  #9078:  log xattr=sa create/remove/update to ZIL
  #11919: Cross-platform xattr user namespace compatibility
  #13014: Report dnodes with faulty bonuslen
  #13016: FreeBSD: Fix zvol_cdev_open locking
  #13019: spl: Don't check FreeBSD rwlocks for double initialization
  #13027: Fix clearing set-uid and set-gid bits on a file when
          replying a write
  #13031: Add enumerated vdev names to 'zpool iostat -v' and
          'zpool list -v'
  #13074: Enable encrypted raw sending to pools with greater ashift
  #13076: Receive checks should allow unencrypted child datasets
  #13098: Avoid dirtying the final TXGs when exporting a pool
  #13172: Fix ENOSPC when unlinking multiple files from full pool

Obtained from:	OpenZFS
OpenZFS commit:	a86e089415
2022-03-08 18:53:02 +01:00
Dimitry Andric 74f7afdfd2 Disable clang 14 warning about bitwise operators in one more place
Follow up 5f2aca8394, where I missed the -Werror warning still being
emitted in libsa.

Fixes:		5f2aca8394
MFC after:	3 days
2022-02-10 19:48:31 +01:00
John Baldwin c7721958ff geliboot: Use the correct IV length for AES-XTS.
- Use AES_XTS_IV_LEN instead of the key length as the IV length.
- Use G_ELI_IVKEYLEN as the size of the zeroed iv[] array in
  g_eli_crypto_cipher() to match geli_io().

PR:		261172
Reported by:	Malcolm Matalka <mmatalka@gmail.com>, mikael
Reviewed by:	markj
Sponsored by:	FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D33884
2022-01-13 17:19:54 -08:00
John Baldwin b156362338 geliboot: Use the multi-block functions for AES-XTS.
Reviewed by:	markj
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D33530
2022-01-11 14:18:12 -08:00
Emmanuel Vadot 01cad73192 loader: tslog: Add more log for module loading
This helps mesuring what's happening when we load the kernel/modules/mfsroot.

This also adds TSENTER2 which uses the third argument of TSRAW, same
as in the kernel.

MFC after:	2 weeks
Sponsored by:	Beckhoff Automation GmbH & Co. KG
Differential Revision:	https://reviews.freebsd.org/D33699
2022-01-11 09:14:10 +01:00
Jessica Clarke 5b13fa7987 ufs: Rework shortlink handling to avoid subobject overflows
Shortlinks occupy the space of both di_db and di_ib when used. However,
everywhere that wants to read or write a shortlink takes a pointer do
di_db and promptly runs off the end of it into di_ib. This is fine on
most architectures, if a little dodgy. However, on CHERI, the compiler
can optionally restrict the bounds on pointers to subobjects to just
that subobject, in order to mitigate intra-object buffer overflows, and
this is enabled in CheriBSD's pure-capability kernels.

Instead, clean this up by inserting a union such that a new di_shortlink
can be added with the right size and element type, avoiding the need to
cast and allowing the use of the DIP macro to access the field. This
also mirrors how the ext2fs code implements extents support, with the
exact same structure other than having a uint32_t i_data[] instead of a
char di_shortlink[].

Reviewed by:	mckusick, jhb
Differential Revision:	https://reviews.freebsd.org/D33650
2022-01-02 20:55:36 +00:00
Emmanuel Vadot dfc9c1d493 loader: tftp: Copy the first block into the cache
tftp_open reads the first block so copy it in the cached data.
If we have more than one block (i.e. we called tftp_read before
tftp_preload) simply just reset the transfer.

Reported by:	mmel
Reviewed by:	mmel, tsoome
MFC after:	2 weeks
Sponsored by:	Beckhoff Automation GmbH & Co. KG
Differential Revision:	https://reviews.freebsd.org/D33697
2021-12-30 16:30:13 +01:00
Emmanuel Vadot 3eb019000c loader: tftp: Add preload method
The preload method will transfer the whole file in a buffer and cache it
so read/lseek operations are faster.

Reviewed by:	imp, tsoome
MFC after:	2 weeks
Sponsored by:	Beckhoff Automation GmbH & Co. KG
Differential Revision:	https://reviews.freebsd.org/D33410
2021-12-16 11:50:41 +01:00
Emmanuel Vadot c25d9aff46 loader: Add preload operation to fs_ops
When we load an ELF file (kernel or module) we do seek(2) a lot to
parse/load the different sections of the ELF file.
Protocol like TFTP suffers a lot from this as there is no resume or
a way to start the tranfer from a specified offset in the file.
fs_preload is added to help those protocol.
Call preload just after opening the ELF file that we need to load so
the underlying method can cache the hole file and then read/lseek operations
are faster.

Reviewed by:	imp
MFC after:	2 weeks
Sponsored by:	Beckhoff Automation GmbH & Co. KG
Differential Revision:	https://reviews.freebsd.org/D33409
2021-12-16 11:50:38 +01:00
Emmanuel Vadot c5f24f5e0d loader: ip: Do not call getsecs so much
getsecs is very costly, reuse the values we got before.

Fetching a ~30MB kernel with the tftp command use to take ~26 seconds
and now it's ~18 seconds.

Reviewed by:	imp, tsoome
MFC after:	2 weeks
Sponsored by:	Beckhoff Automation GmbH & Co. KG
Differential Revision:	https://reviews.freebsd.org/D33408
2021-12-16 11:50:36 +01:00
Emmanuel Vadot 4f36ed513c loader: tftp: Don't let tftp timeout
When we load a kernel or module we open/close it a few times.
Since we're using the same port number each time and that we requested
the same file the ACK that we send are valid on the server side and the
server send us the file multiple times.
This makes tftp loading time very inconsistant due to the UDP "flood" that
we have to process.

Reviewed by:	imp, tsoome
MFC after:	2 weeks
Sponsored by:	Beckhoff Automation GmbH & Co. KG
Differential Revision:	https://reviews.freebsd.org/D33407
2021-12-16 11:50:34 +01:00
Emmanuel Vadot 70661eaafa loader: Add a readtest command
readtest will simply load the file in memory, useful for timing
loading on some filesystems.

Reviewed by:	tsoome
MFC after:	2 weeks
Sponsored by:	Beckhoff Automation GmbH & Co. KG
Differential Revision:	https://reviews.freebsd.org/D33411
2021-12-16 11:50:31 +01:00
Emmanuel Vadot bf07f2f862 loader: tftp: Don't error on tftp error 0
tftp-hpa sends NAK with tftp error set to 0 when trying to get
a directory and this is the first thing that loader tries to do
and this make it hangs.

Reviewed by:	imp, tsoome
MFC after:	2 weeks
Sponsored by:	Beckhoff Automation GmbH & Co. KG
Differential Revision:	https://reviews.freebsd.org/D33406
2021-12-16 11:50:29 +01:00