As part of decommissioning mips support, remove the boot loader
support. Do this in advance of other boot loader work to limit the
amount of work that will be thrown away.
Sponsored by: Netflix
Reviewed by: jrtc27
Differential Revision: https://reviews.freebsd.org/D33377
The STDSB macro is passed to the ffs_sbget() routine to fetch a
UFS/FFS superblock "from the stadard place". It was identically defined
in lib/libufs/libufs.h, stand/libsa/ufs.c, sys/ufs/ffs/ffs_extern.h,
and sys/ufs/ffs/ffs_subr.c. Delete it from these four files and
define it instead in sys/ufs/ffs/fs.h. All existing uses of this macro
already include sys/ufs/ffs/fs.h so no include changes need to be made.
No functional change intended.
Sponsored by: Netflix
net_open() does replace f_devdata with pointer to netdev_sock,
this will cause memory leak when device is closed, but also does
alter the devopen() logic.
We should store &netdev_sock to dev->d_opendata instead, this
would preserve and follow the devopen() logic.
Fixes network boot on aarch64 (tested by bz).
Reviewed-by: imp
MFC After: 2 weeks
Differential Revision: https://reviews.freebsd.org/D32227
Pass the ivlen along through, and just drop this KASSERT() if we're
building _STANDALONE for the time being.
Fixes: 1833d6042c ("crypto: Permit variable-sized IVs ...")
Booting FreeBSD on an EC2 c5.xlarge instance, the loader "twiddles"
810 times over the course of 510 ms, a rate of 1.59 kHz. Even accepting
that many systems are slower than this particular VM and will take
longer to boot (especially if using spinning-rust disks), this seems
like an unhelpfully large amount of twiddling when compared to the
~60 Hz frame rate of many displays; printing the twiddles also consumes
roughly 10% of the boot time on the aforementioned VM.
Setting the default globaldiv to 16 dramatically reduces the time spent
printing twiddles to the console while still twiddling at roughly 100
Hz; this should be ample even for systems which take longer to boot and
consequently twiddle slower.
Note that this can adjusted via the twiddle_divisor variable in
loader.conf, but that file is not processed until nearly halfway
through the loader's runtime.
Reviewed by: allanjude, jrtc27, kevans
MFC after: 1 week
Sponsored by: https://www.patreon.com/cperciva
Differential Revision: <https://reviews.freebsd.org/D32163>
Now that the loader tslog code doesn't call printf, we can profile
printf using TSLOG. On an EC2 c5.xlarge instance, we spend roughly
45 ms here (out of roughly 500 ms), presumably due to the time spent
writing output to the console.
MFC after: 1 week
Sponsored by: https://www.patreon.com/cperciva
We want to keep our root file system open to preserve bcache segment
between file accesses, thus reducing physical disk IO.
Reviewed by: imp, allanjude, kevans (previous version)
Differential Revision: https://reviews.freebsd.org/D30848
MFC after: 1 month
Summary:
Open file list is currently created as statically allocated array (64 items).
Once this array is filled up, loader will not be able to operate with files.
In most cases, this mechanism is good enough, but the problem appears, when
we have many disks with zfs pool(s). In current loader implementation, all
discovered zfs pool configurations are kept in memory and disk devices open -
consuming the open file array. Rewrite the open file mechanism to use
dynamically allocated list.
Reviewed by: imp
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D31364
servip is set from bootp bp_siaddr (if present) and rootip is
set immediately from servip in tha sane bootp code.
However, the common/dev_net.c does only set rootip (based on
url processing etc). Therefore, we should also use rootip in tftp
reader.
Fixes hung tftp based boot when bp_siaddr is not provided.
MFC after: 1 week
In my initial testing, these are the functions which showed up as
being worth instrumenting. More may be added later.
common/console.c: cons_probe
common/gfx_fb.c: read_list, insert_font, autoload_font
common/interp.c: interact
common/interp_lua.c: interp_init, interp_run
efi/libefi/efipart.c: efipart_readwrite
i386/libi386/biosdisk.c: bd_init, bd_open, bd_edd_io, bd_chs_io, bd_io
libsa/open.c: open
libsa/read.c: read
libsa/twiddle.c: twiddle
Note that profiling interp_run may be of questionable utility as it
may depend on user behaviour (e.g. pressing keys).
Reviewed by: kevans (earlier version)
At present this only supports x86, due to the use of the rdtsc
instruction; and is inert unless a buffer is allocated and passed to
the tslog code (which will be done by a future commit).
Reviewed by: kevans
A bug in the loader's bzipfs & gzipfs filesystems caused compressed
kernel and modules not to work on EFI systems with a veriexec-enabled
loader. Since the size of files in these filesystems are not known
_a priori_ `stat` would initialize the size to -1 and the loader would
then hang in an infinite loop while trying to seek (read) to the end
of file since the loop termination condition compares the current
offset to that negative target position.
Reviewers: vangyzen, imp, Bret Ketchum (Bret.Ketchum@dell.com)
Differential Revision: https://reviews.freebsd.org/D30414
Sponsored by: Dell EMC Isilon
MFC to: stable/12, stable/13
MFC after: 1 week
LLVM12 complains if you change the symbol binding:
`error: _longjmp changed binding to STB_GLOBAL`
In this case LLVM actually ignored the weak directive and used the
later .global, but GNU as would mark the symbol as weak.
None of the other architectures mark the libsa _setjmp as weak so
just drop this directive.
Parentheses added to HASZERO macro to avoid a GCC warning, and formatted
with clang-format as we have adopted these and don't consider them
'contrib' code.
Obtained from: musl (snapshot at commit 4d0a82170a25)
Reviewed by: kib (libc integration), mjg (both earlier)
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D17630
Notable upstream changes:
778869fa1 Fix reporting of mount progress
e7adccf7f Disable use of hardware crypto offload drivers on FreeBSD
03e02e5b5 Fix checksum errors not being counted on repeated repair
64e0fe14f Restore FreeBSD resource usage accounting
11f2e9a49 Fix panic if scrubbing after removing a slog device
MFC after: 2 weeks
Caller is not interested in symlinks follow them.
Throw an error if too many links encountered.
Reviewed by: stevek
Sponsored by: Juniper Networks
--This line, and those below, will be ignored--
> Description of fields to fill in above: 76 columns --|
> PR: If a GNATS PR is affected by the change.
> Differential Revision: https://reviews.freebsd.org/D### (*full* phabric URL needed).
> Submitted by: If someone else sent in the change.
> Reviewed by: If someone else reviewed your modification.
> Approved by: If you needed approval for this commit.
> Obtained from: If the change is from a third party.
> MFC after: N [day[s]|week[s]|month[s]]. Request a reminder email.
> MFH: Ports tree branch name. Request approval for merge.
> Relnotes: Set to 'yes' for mention in release notes.
> Security: Vulnerability reference (one per line) or description.
> Sponsored by: If the change was sponsored by an organization.
> Empty fields above will be automatically removed.
Change-Id: I4ef92ff256f503c620dc5bba79ed93b32cb2330d
The integer arrays are encoded in nvlist as counted array <count, i0, i1...>,
loader xdr_array() is missing the count. This will affect the pool import when
there are hole devices in pool.
Also fix the new data add and print functions.
Follow-up to r353959 and r368070: do the same for other architectures.
arm32 already seems to use its own .fnstart/.fnend directives, which
appear to be ARM-specific variants of the same thing. Likewise, MIPS
uses .frame directives.
Reviewed by: arichardson
Differential Revision: https://reviews.freebsd.org/D27387
We don't have NEON available in the boot loader, so we have to disable
it. OpenZFS included ZSTD which used the wrong symbol to bring in neon
support. Change to use the code that's been submitted upstream as a
pull request to both.
__ARM_NEON is the proper symbol, defined in ARM C Language Extensions
Release 2.1 (https://developer.arm.com/documentation/ihi0053/d/). Some
sources suggest __ARM_NEON__, but that's the obsolete spelling from
prior versions of the standard.
OpenZFS Pull Request: https://github.com/openzfs/zfs/pull/11055
ZSTD Pull Request: https://github.com/facebook/zstd/pull/2356
We are using asize property from pool label and we do not depend
on partition data to find last two pool labels and to validate LBA for disk IO.
This does allow us to re-enable support for partitionless disk setups.
The OpenZFS code that uses the BMI instructions is broken. Forcibly
disable them to prevent their use. When enabled, the build breaks.
This fixes the build when compiled for a core with BMI instructions.
This is the same fix committed in r364777, for the same issue.
Submitted by: Jung-uk Kim
Add support to the _STANDALONE environment enough bits of the kernel
that we can compile it. We still have a small zstd_shim.c since there
were 3 items that were a bit hard to nail down and may be cleaned up
in the future. These go hand in hand with a number of commits to
sys/sys in the past weeks, should this need be MFCd.
Discussed with: mmacy (in review and on IRC/Slack)
Reviewed by: freqlabs (on openzfs repo)
Differential Revision: https://reviews.freebsd.org/D26218
OpenZFS will start using some of the kernel timekeeping bits
shortly. This implements the bare minimum of that which currently
is just the time_seconds variable.
During devswitch probe, we pick boot pool based on boot disk, if the boot
disk happens to have multiple pools in freebsd-zfs partitions, the current
code does pick last pool from boot disk as boot pool. While there is no
way at that stage to test, the more logical approach would be to pick
first matching pool.
This patch is assuming we do pass pool guid pointer with guid value 0,
this will help us to determine, if the guid value is already set or not.
The general suggestion would be not to share disk between different pools.
Reported by: Alexander Leidinger
There was a small window cp was broken. Work around this by using :>
instead of cp /dev/null. Ideally, we'd keep the cp /dev/null in the
build as a regression test, but doing so breaks people that upgraded
during the cp breakage and this is simpler than bootstrapping a
working cp since there's no good __FreeBSD_version sign posts for
that.
Suggested by: lots of people
Too stubborn for his own good: imp
bootonce feature is temporary, one time boot, activated by
"bectl activate -t BE", "bectl activate -T BE" will reset the bootonce flag.
By default, the bootonce setting is reset on attempt to boot and the next
boot will use previously active BE.
By setting zfs_bootonce_activate="YES" in rc.conf, the bootonce BE will
be set permanently active.
bootonce dataset name is recorded in boot pool labels, bootenv area.
in case of nextboot, the nextboot_enable boolean variable is recorded in
freebsd:nvstore nvlist, also stored in boot pool label bootenv area.
On boot, the loader will process /boot/nextboot.conf if nextboot_enable
is "YES", and will set nextboot_enable to "NO", preventing /boot/nextboot.conf
processing on next boot.
bootonce and nextboot features are usable in both UEFI and BIOS boot.
To use bootonce/nextboot features, the boot loader needs to be updated on disk;
if loader.efi is stored on ESP, then ESP needs to be updated and
for BIOS boot, stage2 (zfsboot or gptzfsboot) needs to be updated
(gpart or other tools).
At this time, only lua loader is updated.
Sponsored by: Netflix, Klara Inc.
Differential Revision: https://reviews.freebsd.org/D25512
Move hexdump from stand/common/misc.c to stand/libsa/hexdump.c
(svn cp)
Disable use of pager - causes linking issue for boot1
can be re-enabled by defining HEXDUMP_PAGER.
Reviewed by: stevek, imp
MFC after: 1 week
Sponsored by: Juniper Networks
Differential Revision: https://reviews.freebsd.org/D26235
Please note that neither zstd nor encryption is
supported by the loader at this instant. This
change makes it safe to use those features in
one's root pool, but not in one's root dataset.
It can be useful to know what type of machine we are running on for desktop
related thing.
It also allow us to support all the DMI variable that linux driver can fetch.
MFC after: 1 week
Sponsored by: Sponsored-by: The FreeBSD Foundation
The checkpoints are another way of keeping the state of ZFS.
During the rewind, the pool has to be exported.
This makes checkpoints unusable when using ZFS as root.
Add the option to rewind the ZFS checkpoint at the boot time.
If checkpoint exists, a new option for rewinding a checkpoint will appear in
the bootloader menu.
We fully support boot environments.
If the rewind option is selected, the boot loader will show a list of
boot environments that existed before the checkpoint.
Reviewed by: tsoome, allanjude, kevans (ok with high-level overview)
Differential Revision: https://reviews.freebsd.org/D24920
Because we need to read asize from vdev_tree. We also need to consider
different vdev type difference.
Reviewed by: allanjude
Sponsored by: Netflix, Klara Inc.
Differential Revision: https://reviews.freebsd.org/D25586
The DIOCGMEDIASIZE is calculated md->md_sectorsize * md->md_provsize, and
for boot disk, the md_sectorsize is 4k. However, the md_provsize is already
in units of bytes.
Sponsored by: Netflix, Klara Inc.
The vdev_init() does check for "known" vdev types, the [log] device removal will create "hole"
device, but vdev_init() does not allow it.
Obtained from: illumos
MFC after: 1 week
We should have nextboot feature implemented in libsa zfs code.
To get there, I have created zfs_nextboot() implementation based on
two sources, our current simple textual string based approach with added
structured boot label PAD structure from OpenZFS.
Secondly, all nvlist details are moved to separate source file and
restructured a bit. This is done to provide base support to add nvlist
add/update feature in followup updates.
And finally, the zfsboot/gptzfsboot disk access functions are swapped to use
libi386 and libsa.
Sponsored by: Netflix, Klara Inc.
Differential Revision: https://reviews.freebsd.org/D25324
fs_summary_info structure. This change was originally done
by the CheriBSD project as they need larger pointers that
do not fit in the existing superblock.
This cleanup of the superblock eases the task of the commit
that immediately follows this one.
Suggested by: brooks
Reviewed by: kib
PR: 246983
Sponsored by: Netflix
- Use enc_xform_aes_xts.setkey() directly instead of duplicating the code
now that it no longer calls malloc().
- Rather than bringing back all of xform_userland.h, add a conditional
#include of <stand.h> to xform_enc.h.
- Update calls to encrypt/decrypt callbacks in enc_xform_aes_xts for
separate input/output pointers.
Pointy hat to: jhb
Assume gcc is at least 6.4, the oldest xtoolchain in the ports tree.
Assume clang is at least 6, which was in 11.2-RELEASE. Drop conditions
for older compilers.
Reviewed by: imp (earlier version), emaste, jhb
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D24802
When reading less than sector size but from sector boundary,
the vdev_read() will read full sector into the provided buffer
and therefore corrupting memory past buffer end.
MFC after: 2 days
This code was trying to use a pointer value for st_dev, which is definitely
not a pointer. Instead, cast to uintptr_t so it becomes a non-pointer value
before casting it.
Tested: mips-gcc cross compile, mips32 build
On slow platforms, it helps to spread the hashing load
over time so that tftp does not timeout.
Also, some .4th files are too big to fit in cache of pkgfs,
so increase cache size and ensure fully populated.
Reviewed by: stevek
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D24287
This option is intended to aid development, to allow building with user malloc.
The use case would be to build userboot & test with libc (or other) malloc and
use extra malloc debug features.
We need a valid st_dev, st_ino and st_mtime
to correctly track which files have been verified
and to update our notion of time.
ve_utc_set(): ignore utc if it would jump our current time
by more than VE_UTC_MAX_JUMP (20 years).
Allow testing of install command via userboot.
Need to fix its stat implementation too.
bhyveload also needs stat fixed - due to change to userboot.h
Call ve_error_get() from vectx_close() when hash is wrong.
Track the names of files we have hashed into pcr
For the purposes of measured boot, it is important
to be able to reproduce the hash reflected in
loader.ve.pcr
so loader.ve.hashed provides a list of names in the order they
were added.
Reviewed by: imp
MFC after: 1 week
Sponsored by: Juniper Networks
Differential Revision: https://reviews.freebsd.org//D24027
we have crc32(const void *, size_t) in libsa. Unfortunately zlib has
crc32(long, const unigned char *, unsigned) and we have conflict.
Since we do build libsa with zlib, we can use zlib version instead.
Reviewed by: allanjude
Differential Revision: https://reviews.freebsd.org/D24068
smbios used to be an i386 only kinda weird quirk to the x86
architecture. But UEFI picked it up, dusted it off and now it's many
other locations. Make it base technology by moving it to libsa and
fixing up the compliation. The code has issues with unaligned access
still, but that will be addressed in a followup commit.
Differential Revision: https://reviews.freebsd.org/D23660
Create a ptov() function. It's basically the same as the btx PTOV
macro, but works everywhere. smbios needs this to translate addresses,
but the translation differs between BIOS booting and EFI booting. Make
it a function so one smbios.o can be used everywhere. Provide
definitions for it in the two loaders affected.
Differential Revision: https://reviews.freebsd.org/D23660
This uses the new layout of the upstream repository, which was recently
migrated to GitHub, and converted into a "monorepo". That is, most of
the earlier separate sub-projects with their own branches and tags were
consolidated into one top-level directory, and are now branched and
tagged together.
Updating the vendor area to match this layout is next.
In some cases the pool discovery will get stuck in infinite loop while setting
up the vdev children.
To fix, we split the vdev setup into two parts, first we create vdevs based on
configuration we do get from pool label, then, we process pool config from MOS
and update the pool config if needed.
Testing done: confirm previously hung loader is not hung any more.
MFC after: 1 week
Instead of passing NULL to fzap_name_equal and crashing, just return
ENOENT. This happened when higher bits of a hash of the searched key
(its hash prefix) matched a hash prefix of some key in the ZAP, but the
full hash value of the searched key did not match any key in the ZAP.
I observerved this problem when loader tried to look up
"features_for_read" in a particular old pool that predates pool
features.
MFC after: 2 weeks
Sponsored by: Panzura
We do allocate amount of memory (void * or char *), and then assign this
buffer to struct iso_primary_descriptor *vd. Make sure we do
allocate enough bytes.
In fact we do allocate enough, but it is good idea to make sure this really
is so.
MFC after: 1 week
o Remove All Rights Reserved from my notices
o imp@FreeBSD.org everywhere
o regularize punctiation, eliminate date ranges
o Make sure that it's clear that I don't claim All Rights reserved by listing
All Rights Reserved on same line as other copyright holders (but not
me). Other such holders are also listed last where it's clear.
illumos/illumos-gate@663207adb1663207adb1
10601 Pool allocation classes
https://www.illumos.org/issues/10601
illumos port of ZoL Pool allocation classes. Includes at least these two
commits:
441709695 Pool allocation classes misplacing small file blocks
cc99f275a Pool allocation classes
10757 Add -gLp to zpool subcommands for alt vdev names
https://www.illumos.org/issues/10757
Port from ZoL of
d2f3e292d Add -gLp to zpool subcommands for alt vdev names
Note that a subsequent ZoL commit changed -p to -P
a77f29f93 Change full path subcommand flag from -p to -P
Portions contributed by: Jerry Jelinek <jerry.jelinek@joyent.com>
Portions contributed by: Håkan Johansson <f96hajo@chalmers.se>
Portions contributed by: Richard Yao <ryao@gentoo.org>
Portions contributed by: Chunwei Chen <david.chen@nutanix.com>
Portions contributed by: loli10K <ezomori.nozomu@gmail.com>
Author: Don Brady <don.brady@delphix.com>
11541 allocation_classes feature must be enabled to add log device
illumos/illumos-gate@c1064fd7cec1064fd7cehttps://www.illumos.org/issues/11541
After the allocation_classes feature was integrated, one can no longer add a
log device to a pool unless that feature is enabled. There is an explicit check
for this, but it is unnecessary in the case of log devices, so we should handle
this better instead of forcing the feature to be enabled.
Author: Jerry Jelinek <jerry.jelinek@joyent.com>
FreeBSD notes.
I faithfully added the new -g, -L, -P flags, but only -g does something:
vdev GUIDs are displayed instead of device names. -L, resolve symlinks,
and -P, display full disk paths, do nothing at the moment.
The use of special vdevs is backward compatible for read-only access, so
root pools should be bootable, but exercise caution.
MFC after: 4 weeks
Include the server IP address when logging nfs_open(), add a few missing
"\n"s, and correct a typo.
Reviewed by: kevans
MFC after: 2 weeks
Sponsored by: Panasas
Differential Revision: https://reviews.freebsd.org/D22346
In our case the structure is more complex and simple static initializer
will upset compiler diagnostics - using memset is still better than building
more complext initializer.
Since physical device asize is calculated from psize and the asize is stored
in pool label, we can use asize to set the value of psize, which is used to
calculate the location of the pool labels.
MFC after: 1 week
Port illumos change: https://www.illumos.org/issues/11667
Move lz4.c out of zfs tree to opensolaris/common/lz4, adjust it to be
usable from kernel/stand/userland builds, so we can use just one single
source. Add lz4.h to declare lz4_compress() and lz4_decompress().
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D22037
Add ficl words for isvirtualized
and move ficl inb and outb words to ficl/x86/sysdep.c
so can be shared by i386 and amd64
Reviewed by: imp bdrewery
MFC after: 1 week
Sponsored by: Juniper Networks
Differential Revision: https://reviews.freebsd.org/D22069
When zfs probe did fail and no spa was created, but zfs_fmtdev() is called,
we will crash while dereferencing spa (NULL pointer dereference).
MFC after: 1 week
Clean up libstand zalloc* sources. Note that it is not 100% whitespace cleanup.
I also reduced block in znalloc and zfree as those were obvious simplifications
and did help to save one level of indent.
This brings the libsa/mips _setjmp implementation closer to parity with the
libc version.
Reviewed by: imp, jhb
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D21330
ufsread.c grows a dependency on __ashldi3 with llvm90. For gptboot, just
start pulling in ashldi3.c ashrdi3.c lshrdi3.c into libsa for all archs as
the number of archs requiring one or more of them keeps growing. qdivrem.c
and quad.h can be trivially kicked out of libsa if we start pulling these
from compiler-rt as qdivrem was only used to implement umoddi3, divdi3,
moddi3 (also in qdivrem.c).
Reviewed by: imp
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D21291
Other parts of stand/ that don't use libsa will need to grab bits from libc
shortly. Push LIBC_SRC up to defs.mk in advance of this so that they can use
it, and rename it to LIBCSRC to match the convention of the rest of the *SRC
variables in this file.
Reviewed by: imp
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D21282
Use quad.h from libc instead for the time being. This reduces the number of
nearly-identical-quad.h we have in tree to two with only minor changes.
Prototypes for some *sh*di3 have been added to match the copy in libkern.
The differences between the two are likely few enough that they can perhaps
be merged with little additional effort to bring us down to 1.
MFC after: 3 days
We should support removing vdev from boot pool. Update loader zfs reader
to support com.delphix:removing.
Reviewed by: allanjude
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D18901
Summary:
efi loader does not work with static network parameters. It always uses
BOOTP/DHCP and also uses RARP as a fallback. Problems with DHCP servers can
cause the loader to fail to populate network parameters.
Submitted by: Siddharth Tuli <siddharthtuli_gmail.com>
Reviewed by: imp
Sponsored by: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D20811
NANDFS has been broken for years. Remove it. The NAND drivers that
remain are for ancient parts that are no longer relevant. They are
polled, have terrible performance and just for ancient arm
hardware. NAND parts have evolved significantly from this early work
and little to none of it would be relevant should someone need to
update to support raw nand. This code has been off by default for
years and has violated the vnode protocol leading to panics since it
was committed.
Numerous posts to arch@ and other locations have found no actual users
for this software.
Relnotes: Yes
No Objection From: arch@
Differential Revision: https://reviews.freebsd.org/D20745
Add support for an HTTP "network filesystem" using the UEFI's HTTP
stack.
This also supports HTTPS, but TianoCore EDK2 implementations currently
crash while fetching loader files.
Only IPv4 is supported at the moment. IPv6 support is planned for a
follow-up changeset.
Note that we include some headers from the TianoCore EDK II project in
stand/efi/include/Protocol verbatim, including links to the license instead
of including the full text because that's their preferred way of
communicating it, despite not being normal FreeBSD project practice.
Submitted by: scottph
Reviewed by: imp, bcran
Differential Revision: https://reviews.freebsd.org/D20643
There are many new features in ZoF. Most, if not all, do not effect read only usage.
Encryption in particular is enabled at the pool level but used at the dataset level.
The loader obviously will not be able to boot if the boot dataset is encrypted, but
should not care if some other dataset in the root pool is encrypted.
Reviewed by: allanjude
MFC after: 1 week
rename the source to gsb_crc32.c.
This is a prerequisite of unifying kernel zlib instances.
PR: 229763
Submitted by: Yoshihiro Ota <ota at j.email.ne.jp>
Differential Revision: https://reviews.freebsd.org/D20193
If the file is verified - do not allow write
otherwise do not allow read.
Add O_ACCMODE to stand.h
Reviewed by: stevek, mindal_semihalf.com
MFC after: 3 days
Sponsored by: Juniper Networks
Differential Revision: https://reviews.freebsd.org/D20387
This is a prerequisite of unifying kernel zlib instances.
Submitted by: Yoshihiro Ota <ota at j.email.ne.jp>
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D20191
The values of the d_slice and d_partition fields of a disk_devdesc have a
few values with special meanings in the disk_open() routine. Through various
evolutions of the loader code over time, a d_partition value of -1 has
meant both "use the first ufs partition found in the bsd label" and "don't
open a bsd partition at all, open the raw slice."
This defines a new special value of -2 to mean open the raw slice, and it
gives symbolic names to all the special values used in d_slice and
d_partition, and adjusts all existing uses of those fields to use the new
constants.
The phab review for this timed out without being accepted, but I'm still
citing it below because there is useful commentary there.
Differential Revision: https://reviews.freebsd.org/D19262
This relies on libbearssl and libsecureboot
to verify files read by loader in a maner equivalent
to how mac_veriexec
Note: disabled by default.
Use is initially expected to be by embeded vendors
Reviewed by: emaste, imp
Sponsored by: Juniper Networks
Differential Revision: D16336
The cd9660_open() does pass whole path to dirmatch() and we need to
compare only the current path component, not full path.
Additinally, skip over duplicate / (if any) and check if the last component
in the path was meant to be directory (having trailing /). If it is in fact
a file, error out.
Loader does fail to properly match the file name in directory record and
does open file based on prefix match.
For fix, we check the name lengths first.
Reviewed by: allanjude
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D19213
This commit fixes a remaining output buffer overrun in the
single-sector case when there is a non-zero tail.
Reviewed by: allanjude, tsoome
MFC after: 3 months
MFC with: r344226
Differential Revision: https://reviews.freebsd.org/D19220
The bug occurred when a bounce buffer was used and the requested read
size was greater than the size of the bounce buffer. This commit also
rewrites the read logic so that it is easier to systematically verify
all alignment and size cases.
Reviewed by: allanjude, tsoome
MFC after: 3 months
Differential Revision: https://reviews.freebsd.org/D19140
Use recent best practices for Copyright form at the top of
the license:
1. Remove all the All Rights Reserved clauses on our stuff. Where we
piggybacked others, use a separate line to make things clear.
2. Use "Netflix, Inc." everywhere.
3. Use a single line for the copyright for grep friendliness.
4. Use date ranges in all places for our stuff.
Approved by: Netflix Legal (who gave me the form), adrian@ (pmc files)
Since the loader zfs reader does not need to read the dump zvol, we can
just enable the feature.
illumos issue #9051https://www.illumos.org/issues/9051
MFC after: 2 weeks
The reason for this change is that currently, a send/recv
takes many hours to time out.
This is suboptimal in the bootloader because it means for example
that NFS will take hours to fail before allowing subsequent access
methods such as gzip to be tried.
Setting MAXWAIT to 300 seconds (5 minutes) still allows slow
connections of 1Mb to be used to download a 30MB kernel file.
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D18544
With r342151 I did fix the BIOS version of zfs_probe_dev() from accessing
the whole disk, but the fix was not complete - we actually did not check
if the device name was really for whole disk. Since UEFI version
is only calling the zfs_probe_dev() with partitions and not with whole
disk, the UEFI loader was not able to find the zfs pools.
This update does correct the issue by calling archsw.arch_getdev() to
translate the device name back to dev_desc, and we have whole disk when both
partition and slice values are -1.
Reported by: alvisen_gmail.com
Differential Revision: https://reviews.freebsd.org/D18558
First of all, normal setups can not boot such pools as the tools
do not support installing boot programs.
Secondly, for proper pool configuration detection, we need to checks all
four label copies on disk, 2 from front and 2 from the end of the disk,
but zfs label does not contain the size of the disk - so we depend on
firmware to report the correct disk size or use information from the
partition table.
Without partition table, we only can rely on firmware to report and support
disk IO properly.
There is a specific case: 8TB disks are reported by BIOS to have 4294967295
sectors (0x00000000ffffffff), the sectors reported by OS is 15628053168
(0x00000003a3812ab0), so the reported size is less than actual but is hitting
32-bit max. Unfortuantely the real limit must be even lower because probing
this disk in this system will wnd up with hung system.
UEFI boot of this system seems not to be affected.
Reviewed by: imp
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D18558
superblock has a check-hash error, an error message noting the
superblock check-hash failure is printed and the mount fails. The
administrator then runs fsck to repair the filesystem and when
successful, the filesystem can once again be mounted.
This approach fails if the filesystem in question is a root filesystem
from which you are trying to boot. Here, the loader fails when trying
to access the filesystem to get the kernel to boot. So it is necessary
to allow the loader to ignore the superblock check-hash error and make
a best effort to read the kernel. The filesystem may be suffiently
corrupted that the read attempt fails, but there is no harm in trying
since the loader makes no attempt to write to the filesystem.
Once the kernel is loaded and starts to run, it attempts to mount its
root filesystem. Once again, failure means that it breaks to its prompt
to ask where to get its root filesystem. Unless you have an alternate
root filesystem, you are stuck.
Since the root filesystem is initially mounted read-only, it is
safe to make an attempt to mount the root filesystem with the failed
superblock check-hash. Thus, when asked to mount a root filesystem
with a failed superblock check-hash, the kernel prints a warning
message that the root filesystem superblock check-hash needs repair,
but notes that it is ignoring the error and proceeding. It does
mark the filesystem as needing an fsck which prevents it from being
enabled for writing until fsck has been run on it. The net effect
is that the reboot fails to single user, but at least at that point
the administrator has the tools at hand to fix the problem.
Reported by: Rick Macklem (rmacklem@)
Discussed with: Warner Losh (imp@)
Sponsored by: Netflix
of args data between gptboot/zfsboot and loader(8).
Despite what seems like a lot of changes here, there are no actual
changes in behavior, or in the data layout in the structures involved.
This is just eliminating identical code pasted into multiple locations.
In detail, the changes are...
- Move struct zfs_boot_args definition from libsa/zfs/libzfs.h to
i386/common/bootargs.h because it is specific to x86 booting and the
handoff between zfsboot and loader, and has no relation to the zfs
library code in general.
- The geli_boot_args and zfs_boot_args structs both contain an identical
set of member variables containing geli information. Extract this out
to a new geli_boot_data struct, and embed it in the arg-passing structs.
- Provide new routines geli_import_boot_data() and geli_export_boot_data()
that can be shared between gptboot, zfsboot, and loader instead of
pasting identical code into several different .c files.
- Remove some checks for a NULL pointer that can never be true because the
pointer being tested was set using pointer math (kargs + 1) and that can
never result in NULL in this code.
ffs_subr.c requires calculate_crc32c() from libkern. Unfortunately we
cannot just add libkern/crc32.c to libstand because crc32.o is already
compiled from contrib/zlib/crc32.c. Use the include trick to rename
the source.
Note that libstand also provides crc32.c which seems to be unused.
Reviewed by: imp
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D17677
In current tftp code we drop out-of-order packets; however, we should play
nice and re-send ACK for older data packets we are receiving. This will
hopefully stop server repeating those packets we already have received.
Note we do not answer duplicates from "previous" session (that is, session
with different port number), those will eventually time out.
Differential Revision: https://reviews.freebsd.org/D17087
The name check referred in the comment is not the only possible error source,
we need to validate the result.
Reviewed by: allanjude
Approved by: re (kib)
Differential Revision: https://reviews.freebsd.org/D17081
tftpfile is allocated just above and needs to be freed.
Reviewed by: imp
Approved by: re (kib)
Differential Revision: https://reviews.freebsd.org/D17058
It was possible in some rare circumstances for ngets to behave terribly with
bhyveload and some form of redirecting user input over a pipe.
PR: 198706
Submitted by: Ivan Krivonos <int0dster@gmail.com>
MFC after: 1 week
This moves the bulk of the geli support from lib386/biosdisk.c into a new
geli/gelidev.c which implements a devsw-type device whose dv_strategy()
function handles geli decryption. Support for all arches comes from moving
the taste-and-attach code to the devopen() function in libsa.
After opening any DEVT_DISK device, devopen() calls the new function
geli_probe_and_attach(), which will "attach" the geli code to the open_file
struct by creating a geli_devdesc instance to replace the disk_devdesc
instance in the open_file. That routes all IO for the device through the
geli code.
A new public geli_add_key() function is added, to allow arch/vendor-specific
code to add keys obtained from custom hardware or other sources.
With these changes, geli support will be compiled into all variations of
loader(8) on all arches because the default is WITH_LOADER_GELI.
Relnotes: yes
Sponsored by: Microchip Technology Inc
Differential Revision: https://reviews.freebsd.org/D15743
to it being a common name elsewhere. Rename the old kzip one
to subr_inflate.c.
This actually fixes the build issues on sparc64 that my inclusion of
.PATH ${SYSDIR}/kern created in r336244, so also revert the broken
workaround I committed in r336249.
This slipped passed me because apparently, I never did a clean build.
boot_parse_arg to parse a single arg
boot_parse_cmdline to parse a command line string
boot_parse_args to parse all the args in a vector
boot_howto_to_env Convert howto bits to env vars
boot_env_to_howto Return howto mask mased on what's set in the environment.
All these routines return an int that's the bitmask of the args
translated to RB_* flags. As a special case, the 'S' flag sets the
comconsole_speed env var. Any arg that looks like a=b will set the env
key 'a' to value 'b'. If =b is omitted, 'a' is set to '1'. This
should help us reduce the number of redundant copies of these routines
in the tree. It should also give a more uniform experience between
platforms.
Also, invent a new flag RB_PROBE that's set when 'P' is parsed. On
x86 + BIOS, this means 'probe for the keyboard, and if it's not there
set both RB_MULTIPLE and RB_SERIAL (which means show the output on
both video and serial consoles, but make serial primary). Others it
may be some similar concept of probing, but it's loader dependent
what, exactly, it means.
These routines are suitable for /boot/loader and/or the kernel,
though they may not be suitable for the tightly hand-rolled-for-space
environments like boot2.
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D16205
Move the libzfs stuff into libsa. There's no need for it to be a
separate library. The separate library adds to the issues of build
ordering that we see from time to time. Move the filesystem support
into libsa, like all the other filesystem support rather than making
zfs the odd-duck out.
Discussed with: allanjude@
Reduce by 1 the number of crazy libraries we need in stand by moving
geli into libsa (where architecturally it belonged all along). This
just moves things around without any code changes.
- jhb implemented UFS write support a little over 16 years ago.
- Update the library name while we're here.
Reviewed by: jhb, rpokala
Differential Revision: https://reviews.freebsd.org/D14476
This makes them compatible with the C standard signatures, avoiding
spurious mismatch errors in the places where the oddball requirements
of standalone code end up putting two declarations of the same function
in play.
This is part of a project for adding the ability to create hybrid CD/USB boot
images. In the BIOS case when booting from something that isn't a CD we need
some extra boot code to actually find our next stage (loader) within an
ISO9660 filesystem. This code will reside in a GPT partition (similar to
gptboot(8) from which it is derived) and looks for /boot/loader in an
ISO9660 filesystem on the image.
Reviewed by: imp
Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D14914
Followup to r313780. Also prefix ext2's and nandfs's versions with
EXT2_ and NANDFS_.
Reported by: kib
Reviewed by: kib, mckusick
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D9623
A foolish consistency is the hobgoblin of little minds, adored by
little statesmen and philosophers and divines. With consistency a
great soul has simply nothing to do. -- Ralph Waldo Emerson
Make sure { on the same line as struct for all struct *devdesc. Move
some type definitions to next to the dv_type define, since that's what
sets the d_type.
to fix the memory leak that I introduced in r328426. Instead of
trying to clear up the possible memory leak in all the clients, I
ensure that it gets cleaned up in the source (e.g., ffs_sbget ensures
that memory is always freed if it returns an error).
The original change in r328426 was a bit sparse in its description.
So I am expanding on its description here (thanks cem@ and rgrimes@
for your encouragement for my longer commit messages).
In preparation for adding check hashing to superblocks, r328426 is
a refactoring of the code to get the reading/writing of the superblock
into one place. Unlike the cylinder group reading/writing which
ends up in two places (ffs_getcg/ffs_geom_strategy in the kernel
and cgget/cgput in libufs), I have the core superblock functions
just in the kernel (ffs_sbfetch/ffs_sbput in ffs_subr.c which is
already imported into utilities like fsck_ffs as well as libufs to
implement sbget/sbput). The ffs_sbfetch and ffs_sbput functions
take a function pointer to do the actual I/O for which there are
four variants:
ffs_use_bread / ffs_use_bwrite for the in-kernel filesystem
g_use_g_read_data / g_use_g_write_data for kernel geom clients
ufs_use_sa_read for the standalone code (stand/libsa/ufs.c
but not stand/libsa/ufsread.c which is size constrained)
use_pread / use_pwrite for libufs
Uses of these interfaces are in the UFS filesystem, geoms journal &
label, libsa changes, and libufs. They also permeate out into the
filesystem utilities fsck_ffs, newfs, growfs, clri, dump, quotacheck,
fsirand, fstyp, and quot. Some of these utilities should probably be
converted to directly use libufs (like dumpfs was for example), but
there does not seem to be much win in doing so.
Tested by: Peter Holm (pho@)
There are some _write callbacks left only returning EROFS, replace them
by null_write. return EROFS from null_write().
Reviewed by: cem, imp, kan
Differential Revision: https://reviews.freebsd.org/D14523
Current timeout behavior is to progress in timeout values from MINTMO to
MAXTMO in MINTMO steps before finally timing out. This results in a fairly
long time before operations finally timeout, which may not be ideal for some
use-cases.
Add MAXWAIT that may be configured along with MINTMO/MAXTMO. If we attempt
to start our send/recv cycle over again but MAXWAIT > 0 and MAXWAIT seconds
have already passed, then go ahead and timeout.
This is intended for those that just want to say "timeout after 180 seconds"
rather than calculate and tweak MINTMO/MAXTMO to get their desired timeout.
The default is 0, or "progress from MINTMO to MAXTMO with no exception."
This has been modified since review to allow for it to be defined via CFLAGS
and doing appropriate error checking. Future work may add some Makefile foo
to respect LOADER_NET_MAXWAIT if it's specified in the environment and pass
it in as MAXWAIT accordingly.
Reviewed by: imp, sbruno, tsoome (all previous version)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D14389
Write support (even if it only works on UFS) will be needed for nextboot
functionality.
Reviewed by: cem, imp
Differential Revision: https://reviews.freebsd.org/D14478
ffs_sbget() may return a superblock buffer even if it fails, so the
caller must be prepared to free it in this case. Moreover, when tasting
alternate superblock locations in a loop, ffs_sbget()'s readfunc
callback must free the previously allocated buffer.
Reported and tested by: pho
Reviewed by: kib (previous version)
Differential Revision: https://reviews.freebsd.org/D14390
OK. We don't really need a bsd.stand.mk, and it was causing a -fPIC
for the toolchain to be added (bogusly) when building on amd64. Pull
all relevant defs back into defs.mk and delete bsd.stand.mk.
This saves about 15-20k on i386 loader and zfsloader which when
combined with Lua give us a lot more stack space in those constrained
environments.
