Commit graph

99 commits

Author SHA1 Message Date
Warner Losh 078a31136c boot/universe.sh: Add tests to keep MK_LOADER_BIOS_TEXTONLY working
Sponsored by:		Netflix
2024-02-17 23:29:07 -07:00
Warner Losh d0b2dbfa0e Remove $FreeBSD$: one-line sh pattern
Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/
2023-08-16 11:55:03 -06:00
Marius Strobl 4ef1c6f75d base: Remove support for the VTOC8 partitioning scheme
The removal of the sparc64 support in February 2020 obsoleted the
VTOC8 partitioning scheme as no other FreeBSD platform makes use
of it. Moreover, the code is bitrotting as nothing defines e. g.
LOADER_VTOC8_SUPPORT any more and, thus, should go now, too. With
this change, the following commits are reverted as far as VTOC8
is concerned and parts haven't already previously been deleted
along with prior sparc64 removals:
094fcb157d
a7d366e958
ba8d50d08b

The alignment example d9711c28ef
added to the VTOC8 section of gpart.8 is folded into the MBR one.

This should finally conclude the deorbit of sparc64-specific bits.

        We had joy, we had fun
        we ran Unix on a Sun.
        But that source and the song
        of FreeBSD have all gone.

Credits to Michael Bueker for the original "Unix on a Sun" and Rod
McKuen for the "Seasons in the Sun" lyrics.
2023-07-26 13:16:12 +02:00
Warner Losh cf6044857e stand: Testing program for smbios
Write a quick and dirty testing program to dump physical memory to help
test and debug the smbios.c code in new environments.

Sponsored by:		Netflix
Differential Revision:	https://reviews.freebsd.org/D39791
2023-05-01 15:12:19 -06:00
John Baldwin 1ca12bd927 Remove the riscv64sf architecture.
Reviewed by:	jrtc27, arichardson, br, kp, imp, emaste
Differential Revision:	https://reviews.freebsd.org/D39496
2023-04-12 11:09:27 -07:00
Ed Maste a2de35e1a4 CI: update QEMU command line to avoid deprecated option format
> warning: short-form boolean option 'readonly' deprecated
> Please use readonly=on instead

Sponsored by:	The FreeBSD Foundation
2023-03-14 08:44:50 -04:00
Warner Losh e600bcfb87 stand: Better documentation for full-test.sh
Update now that things have been pushed into the tree.

Sponsored by:		Netflix
2023-02-18 19:53:19 -07:00
Ed Maste 7b0593fdcb Cirrus-CI: use makefs if root size exceeds QEMU's virtual FAT limit
We use QEMU's virtual FAT support to avoid having to create a disk
image because it is much faster, but it has a limit of about 500MB.
Artifacts produced by the GCC 12 CI job exceeded this size.

Add support for creating a FAT partition image and MBR-partitioned disk
image and use it when the file system is too large for QEMU.

In one run the Cirrus-CI LLVM test task took 1m33s using QEMU's virtual
FAT while the GCC task took 6m48s using makefs+mkimg.

Reviewed by:	imp
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D38589
2023-02-14 15:41:01 -05:00
Warner Losh a5c0d55177 test: Add fstab to all ufs images
Ensure that we populate /etc/fstab for all the ufs images.  Tweak sizes
while I'm at it.

Note: This file could use a good refactoring... or maybe a rewrite in
python or lua.

Sponsored by:		Netflix
Reviewed by:		tsoome
Differential Revision:	https://reviews.freebsd.org/D38317
2023-02-03 08:41:41 -07:00
Warner Losh eaf5ec88c2 stand: Add testing for i386 as well to full-test.sh
Sponsored by:		Netflix
2023-01-25 08:04:59 -07:00
Warner Losh 43d82295d2 testing: Make sure to specify a name for zpools
Zpools have to have a name. Pools created w/o a name aren't legit and
trigger a number of bugs.

Sponsored by:		Netflix
2023-01-08 09:45:11 -07:00
Warner Losh 858a73f14c full-test: Start of a full testing suite.
full-test.sh aims to be a test suite generator for the boot loader. It
tries to grab artifacts from the web and then constructs minimal boot
environments from that as well as writing qemu-system-* using scripts
that facilitates testing all the ways we can boot... At least all the
ways that we an boot that qemu can emulate.

This is very much a work in progress, and likely could use a good
cleanup at some point.

Sponsored by:		Netflix
2022-12-02 11:08:11 -07:00
Warner Losh aad99ded88 Add test notes
Add notes on how to test things. This will likely need to be fleshed out
more in the future, but this is a good start.

Sponsored by:		Netflix
2022-12-02 11:07:33 -07:00
Warner Losh 8a744de27e stand: Remove i386-only support fire firewire
Remove support for booting off of firewire, and for having dcons via
firewire in the loader. Kernel support for these things is unchanged.
Discussed on arch@ and the current state is not working (and the build
was wrong to boot).

Sponsored by:		Netflix
Discussed:		https://lists.freebsd.org/archives/freebsd-arch/2022-November/000267.html
Reviewed by:		kevans, melifaro, emaste
Differential Revision:	https://reviews.freebsd.org/D37334
2022-11-18 10:43:59 -07:00
Warner Losh ef8b872301 stand: Flesh out the zfs image creation
Move the mbr non-geli zfs cases to no-priv creation with makefs / mkimg.
Add comments about the weird thing we do for MBR + ZFS + Legacy.  Add
comments about other architectures. Still need to think through how to
leverage a completed universe to do all the architectures...

Sponsored by:		Netflix
2022-08-17 11:39:53 -06:00
Warner Losh 3d3f2fb069 stand: Make makeroot.sh use makefs for ZFS
Start to use makefs for ZFS. This covers the gpt nogeli variants.  ZFS
MBR booting is tricky and complicated, so will need some additional
tweaks that makefs/mkimg isn't able to do at the moment. This means that
all gpt nogeli amd64 combinations can be built w/o root.

In addition, tweak the generated qemu.sh files to use stdio for the
console. We grep the output for SUCCESS and report each of the booting
types. Create a all.sh that will run these automatically. These all can
also run w/o root.

In the future, I'll add support for a make univers followed by this
script to create other architectures' tests and/or generate stand tests
for /usr/tests...

Sponsored by: Netflix
2022-08-16 15:36:55 -06:00
Warner Losh 729c458bf7 stand: Abstract out fstab creation
Now that all the fstabs are the same, abstract it out into a function.

Sponsored by:		Netflix
2022-08-15 12:46:31 -06:00
Warner Losh 26a241a0e2 stand: Update GELI UFS cases to use ufs labels
GELI images are created in a different manner than non-GELI
images. Update them to set the label of 'root' on the UFS partition and
use that in fstab. Drop comments about needing to do this, as well as
the 'dev' variable now that it's unused.

Sponsored by:		Netflix
2022-08-15 12:38:32 -06:00
Warner Losh 159b158a1d stand: Update testing harness to start to use UFS labels
Minor changes to the non-geli UFS targets: migrate to using ufs labels
so we don't need to know the name of the device we booted off of. This
doesn't change the GELI test cass just yet since I've not tested them.
ZFS doesn't need these changes since we don't need to encode the device
for it.

Sponsored by:		Netflix
2022-08-15 12:38:32 -06:00
Warner Losh aff95057bc stand: Update testing script for arch retirement
And catch up to changes in booting methods currently supported. Add
riscv.

Sponsored by:		Netflix
2022-08-15 12:38:32 -06:00
Warner Losh 08585e7799 tools/boot: Fix universe.sh's ZFS exclusion test
MK_LOADER_ZFS is the current spelling of enabling or disabling ZFS
builds. Use it instead of MK_ZFS.

Sponsored by:		Netflix
Reviewed by:		tsoome
Differential Revision:	https://reviews.freebsd.org/D35896
2022-07-24 16:53:34 -06:00
Warner Losh 9e9ba3cdbe test: Update boot universe build architectures
We build lua by default, so we don't need another build to build it
enabled w/o FORTH. That gives little value over the without forth
builds. Remove all mips, they are no longer relevant. Build aarch64
everywhere we build amd64 (except firewire which is x86 only). Build a
few more architectures once so we have at least one of every arch we
support in at least the default build. This should increase coverage
and still take less time than before.

Sponsored by:		Netflix
2022-07-08 11:47:37 -06:00
Jessica Clarke 17160f21ef install-boot.sh: Avoid - in function names for POSIX compatibility
FreeBSD sh supports this but other common POSIX shells do not; in
particular, dash does not, unlike bash and zsh. This allows the script
to be used on non-FreeBSD systems for release media building.

Reviewed by:	emaste, brooks
Differential Revision:	https://reviews.freebsd.org/D34000
2022-02-28 22:37:21 +00:00
Ed Maste bba96bb143 CI: add arm64 support to ci-qemu-test.sh
Reviewed by:	imp (earlier)
MFC after:	3 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D30907
2021-06-26 14:26:41 -04:00
Ed Maste 7d9794b34b CI: use amd64 EDK II firmware included with QEMU
QEMU (now) includes a prebuilt EDK II firmare in edk2-x86_64-code.fd.
Use that instead of requring a standalone uefi-edk2-qemu-x86_64 package.

Reviewed by:	imp
MFC after:	3 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D30915
2021-06-26 14:22:48 -04:00
Kyle Evans 8eeeee38f4 tools: boot: use four jobs for building stand
Parallel builds of stand should be assumed both possible and safe as of
7012461c9b, so let's start using some jobs to speed up lualoader test
harness builds.
2021-01-31 10:11:41 -06:00
Kyle Evans 29842cb36e lualoader: fix lua-lint run
luacheck rightfully complains that i is unused in the show-module-options
loop at the end (it was used for some debugging in the process).

We've added a new pager module that's compiled in, so declare that as an
acceptable global.
2020-12-17 18:29:30 +00:00
Mitchell Horne d46a2a0013 riscv: allow building virtual machine images
RISC-V has the same booting requirements as arm64 (loader.efi, no legacy
boot options), so generated images for both architectures have the same
partition layout.

Reviewed by:	gjb
Differential Revision:	https://reviews.freebsd.org/D27044
2020-12-08 00:37:11 +00:00
Warner Losh 4c4e2b3de3 Speciy the dev in an easily changed variable
Rather than hard coding ada0 everywhere, use ${dev}. Also, set
dev=vtbd0 since both qemu and bhyve support this. More work
should be done to use labels instead for fstab.

qemu scripts likely need adjustment. And we should also
likely generate byhve scripts too.
2020-09-28 06:00:56 +00:00
Ed Maste 08d0b468f1 ci-qemu-test.sh: use pkgbase
Reviewed by:	bcran (earlier), manu (earlier), imp
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D24276
2020-09-05 19:03:34 +00:00
Matt Macy 9e5787d228 Merge OpenZFS support in to HEAD.
The primary benefit is maintaining a completely shared
code base with the community allowing FreeBSD to receive
new features sooner and with less effort.

I would advise against doing 'zpool upgrade'
or creating indispensable pools using new
features until this change has had a month+
to soak.

Work on merging FreeBSD support in to what was
at the time "ZFS on Linux" began in August 2018.
I first publicly proposed transitioning FreeBSD
to (new) OpenZFS on December 18th, 2018. FreeBSD
support in OpenZFS was finally completed in December
2019. A CFT for downstreaming OpenZFS support in
to FreeBSD was first issued on July 8th. All issues
that were reported have been addressed or, for
a couple of less critical matters there are
pull requests in progress with OpenZFS. iXsystems
has tested and dogfooded extensively internally.
The TrueNAS 12 release is based on OpenZFS with
some additional features that have not yet made
it upstream.

Improvements include:
  project quotas, encrypted datasets,
  allocation classes, vectorized raidz,
  vectorized checksums, various command line
  improvements, zstd compression.

Thanks to those who have helped along the way:
Ryan Moeller, Allan Jude, Zack Welch, and many
others.

Sponsored by:	iXsystems, Inc.
Differential Revision:	https://reviews.freebsd.org/D25872
2020-08-25 02:21:27 +00:00
Warner Losh 25dcf0e535 Spell LOADER correctly so we test lua build. 2020-05-01 17:50:26 +00:00
Warner Losh ce6fd19966 Catch up to arm/arm and sparc64 removal. 2020-05-01 17:17:01 +00:00
Warner Losh 22ea5a7250 When we have an invalid build option, don't rm -rf the current
directory.

Add a quick sanity check to objdir before using it.  It must start
with /. If there was a make error getting it, report that and continue
with the next target. If there was anything else, bail out.
2020-05-01 17:16:57 +00:00
Ed Maste f63070048b CI: print wired page count on boot
This is a first step in tracking kernel memory use over time.
2020-02-29 19:59:21 +00:00
Ed Maste 3e6b8ce30e ci-qemu-test: if firmware is not available, hint at pkg to install
uefi-edk2-qemu-x86_64 provides the firmware ci-qemu-test.sh expects to
use.

Sponsored by:	The FreeBSD Foundation
2019-09-09 14:51:25 +00:00
Matt Macy 14113f123e Use makefs -t msdos in make_esp_file
With this last piece in place, make -C /usr/src/release release.iso is
finally able to run in a jail. This was not possible before because
msdosfs cannot be mounted inside a jail.

Submitted by:	ryan@ixsystems.com
Reviewed by:	emaste@, imp@, gjb@
MFC after:	1 week
Sponsored by:	iXsystems, Inc.
Differential Revision:	https://reviews.freebsd.org/D21385
2019-09-03 18:37:55 +00:00
Ed Maste dc93026776 revert QEMU q35 platform use from r346748
It seems to be incompatible with the OVMF.fd (of unknown provenance)
in use by the Cirrus-CI config.  We will soon have a known OVMF build
via a port/package (see review D19869) and we can switch back to q35
once packages are available.

Discussed with:	bcran
2019-04-30 15:28:52 +00:00
Rebecca Cran c1ffbf2da0 Fix tools/boot/ci-qemu-test.sh and make some improvements
Update ci-qemu-test.sh

o Update the path to the OVMF file, which is now in /usr/local/share/uefi-edk2-qemu.
o Use the more modern q35, pc-q35-3.0 (Q35 + ICH9, 2009) QEMU machine
  instead of the default, obsolete pc, pc-i440fx-3.0 (i440FX + PIIX, 1996).
  For example this adds ACPI support.
o Specify the system firmware using the newer pflash drive syntax instead
  of bios.
o Remove extra, unneeded devices by passing -nodefaults.
o Change text to talk about 'firmware' instead of 'bios', since UEFI
  isn't a BIOS.

Reviewed by:	imp, emaste
Differential Revision:	https://reviews.freebsd.org/D20074
2019-04-26 14:44:46 +00:00
Ian Lepore 0c0ffba5d6 For the geli-gpt-zfs test images, both bios and uefi flavors, add a dummy
ufs partition as p2, and put the zfs partition at p3, to test the ability
of the zfs probe code to find a zfs pool on something other than the first
partition.
2019-04-25 00:08:15 +00:00
Ian Lepore 827f1973cb The zfs module has grown a dependency on zcl_nfs4.ko, so copy it into all
the test images.
2019-04-24 23:52:38 +00:00
Ian Lepore f576caac25 Complain and exit the script if the 'make install' phase fails. Also,
there is no need to install any debug files.
2019-04-24 23:51:12 +00:00
Rebecca Cran 0661cd79ca Fix install-boot.sh and rootgen.sh UEFI bugs
tools/boot/install-boot.sh was assuming that if a device was passed in,
it should operate on the current system and run efibootmgr etc. to
update the boot manager. However, rootgen.sh passes a md(4) device and
not a fixed disk.

Add a -u option to install-boot.sh to tell it to update the system
in-place and run efibootmgr etc.

Also, source install-boot.sh in rootgen.sh to allow it to find and
call make_esp_file etc. And pass the loader file to make_esp_file instead
of a directory name.

Reported by:	ian
Reviewed by:	ian,imp,tsoome
Differential Revision:	https://reviews.freebsd.org/D19992
2019-04-24 22:35:29 +00:00
Ed Maste 79b02e7017 Put QEMU CI smoke test boot log in /tmp if TMPDIR not set
Sponsored by:	The FreeBSD Foundation
2019-04-17 19:24:41 +00:00
Ed Maste 297ce4ce2d Install some entropy for QEMU CI smoke test
See r346250 and followup commits and mailing list discussion.
We currently fail to boot properly in the absense of boot-time entropy.

Sponsored by:	The FreeBSD Foundation
2019-04-17 19:16:26 +00:00
Ian Lepore 2614c4835f Allow this test script to be run from within src/tools/boot dir, and create
the temporary image in $TMPDIR.

Allow the script to be run from the src/tools/boot directory by using make
-V SRCTOP to find the top of the tree, because this script is handy for
quick smoke-testing of loader changes, as well as being useful in CI testing.

Also, use a temp directory in $TMPDIR to assemble the boot image, and write
the boot log file to $TMPDIR.  Arrange to have the temporary image clean
itself up, but leave the log file in $TMPDIR for post-mortem analysis of
failures when the script is run interactively.

Differential Revision:	https://reviews.freebsd.org/D19876
2019-04-17 16:27:43 +00:00
Ed Maste cc2e55c7ea Add a smoke test QEMU boot script for CI
Reviewed by:	imp, lwhsu (earlier)
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D19661
2019-04-10 13:41:34 +00:00
Rebecca Cran db8b561345 Rework UEFI ESP generation
Currently, the installer uses pre-created 800KB FAT12 filesystems that
it dd's onto the ESP partition.
This changeset improves that by having the installer generate a FAT32
filesystem directly onto the ESP using newfs_msdos and then copying
loader.efi into /EFI/freebsd.
For live installs it then runs efibootmgr to add a FreeBSD boot entry
in the BIOS.

Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D17947
2018-12-20 19:39:37 +00:00
Kyle Evans f98c671d07 Update lualoader test script a little bit
Use userboot.so from the test directory if possible, fall back to .OBJDIR.
This avoids a problem that we've had since userboot coexistence was added,
where userboot.so alone no longer exists in the .OBJDIR but is instead just
a link installed later.
2018-10-25 02:04:01 +00:00
Marius Strobl e94a7c27df Following r335259, don't copy boot1 from the running system for sparc64
either.
2018-08-23 22:57:42 +00:00