The vm_extra_pre_umount function in vmimage.subr served two purposes:
It removed /etc/resolv.conf and /qemu (if cross-building), and it
provided a function for cloudware to override in order to make cloud
specific changes to the filesystem before constructing a disk image.
This resulted in a number of bugs:
1. When cross-building, the emulator binary was left as /qemu in the
Azure, GCE, Openstack and Vagrant images.
2. The build host's resolv.conf was left as /etc/resolv.conf in the
basic-ci and basic-cloudinit images.
3. When building GCE images, a Google-specific resolv.conf file was
constructed, and then deleted before the disk image was created.
Move the bits needed for running code inside a VM staging directory
from vm_install_base into a new vm_emulation_setup routine, and move
the corresponding cleanup bits from vm_extra_pre_umount to a new
vm_emulation_cleanup routine.
Remove the /qemu and /etc/resolv.conf cleanups from the cloudware
configuration files (where they exist) since we will now be running
vm_emulation_cleanup to remove those even when vm_extra_pre_umount
has been overridden.
Override vm_emulation_cleanup in gce.conf since in that one case (and
*only* that one case) we don't want to clean up resolv.conf (since it
was constructed for the VM image rather than copied from the host).
releng/14.1 candidate.
MFC after: 1 week
Sponsored by: https://www.patreon.com/cperciva
Provide both zfs and ufs images which a 1MB partition reserved for the
config drive wearing a GPT Label "config-drive" to allow consumer to
know where they should push the config drive on the provided image.
2 formats available: qcow2 and raw
This has been tested on OVHCloud baremetal via "bring your own image"
Also tested on openstack
Reviewed by: emaste
Sponsored by: OVHCloud
Differential Revision: https://reviews.freebsd.org/D44369
Currently there is no support for generating armv7 vm images in the
release artifacts. In fact in terms of release artifacts and
architecture there is no good reason to have a vm release artifact for
armv7 as those are mostly used in SOCs or embedded boards. However
considering that developers actually do need an easy way to test armv7
with a vm running this is really important. As part of pre-commit ci for
developers this can be really helpful for the end developers.
Approved by: cperciva, imp, re
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D43952
Currently the partition layout is hardcoded to create an EFI/FAT
partition and a UFS root partition, with some logic to handle GPT/MBR
differences.
On RISC-V platforms we are seeing the emerging pattern that firmware
should be placed in a partition of a known type, rather than just a
known sector of the disk. Thus, some functionality is needed to
customize the layout for SD card images.
Add a hook, arm_create_partitions(), which can be overridden to insert
additional platform-specific partitions, possibly preceding the standard
EFI and UFS ones. A couple of new variables are added to track the
indices, e.g. ROOTFSPART_SUFFIX=p2.
In a couple places this de-duplicates the GPT/MBR logic.
Reviewed by: manu, karels, imp
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D43848
These days, the entries are the identical for GPT and MBR.
Reviewed by: manu, karels, imp
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D43847
When only the user (ALL) is specified explicitly, and the group is
implied, only sudo -u works. Specifying both the user and group, like
(ALL:ALL), is required to:
1. Use sudo -g by itself (with no -u user)
2. Use sudo -u and -g together, with a -g group that is different from
the -u user's primary group
Obtained from: 1d13533ea3
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/835
Most 64-bit Raspberry Pi models have a variable processor clock
speed that defaults to a slow speed (e.g. 600 MHz for a nominal
1.5 GHz clock). This results in everything running slowly unless
or until powerd is started, and FreeBSD is then thought to be slow.
Enable powerd by default in /etc/rc.conf on the arm64-aarch64-RPI
images. Tested on Raspberry Pi 3B+ and 4B so far.
PR: 256836
MFC after: 1 month
Reviewed by: rgrimes
Differential Revision: https://reviews.freebsd.org/D43296
Known issues:
1. The ec2-user user is created with a homedir of /usr/home/ec2-user
instead of /home/ec2-user; this appears to be a bug in cloud-init's
FreeBSD support.
2. Cloud-init configures IPv4 networking but not IPv6 networking.
releng/14.0 candidate.
Discussed with: gjb
Reviewed by: imp
MFC after: 5 days
Relnotes: yes
Sponsored by: https://www.patreon.com/cperciva
Differential Revision: https://reviews.freebsd.org/D41793
Split ec2-base.conf into ec2-base.conf and a reusable ec2.conf,
similar to how Vagrant flavours share a common vagrant.conf.
releng/14.0 candidate.
Discussed with: gjb
MFC after: 5 days
Sponsored by: https://www.patreon.com/cperciva
Differential Revision: https://reviews.freebsd.org/D41792
Using the recently-added "cloudware flavours" mechanism, turn the
existing EC2 AMIs into a new "base" flavour. The only user-visible
change is that AMI names now include the word "base".
releng/14.0 candidate.
Discussed with: gjb
Reviewed by: imp
MFC after: 5 days
Sponsored by: https://www.patreon.com/cperciva
Differential Revision: https://reviews.freebsd.org/D41791
We no longer install a default portsnap.conf, so the sed invocation just
generates an error.
Reviewed by: cperciva
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D42003
mergemaster was deprecated some time ago and will be removed from
FreeBSD 15.
Reviewed by: imp
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D41797
Other cloud images do not do this, and it can produce confusing results.
Reviewed by: Jose Luis Duran, delphij
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D41751
vm.subr's default vm_extra_pre_umount removes /qemu and
/etc/resolv.conf. When vm_extra_pre_umount is overridden these steps
need to be performed in the cloud-specific conf file.
PR: 271602
Reviewed by: dch, lwhsu
Event: Kitchener-Waterloo Hackathon 202305
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D40257
Change the vmimage script for zfs to create /home as a dataset
rather than /usr/home, ala change to bsdinstall's zfs script.
Reviewed by: markj
Differential Revision: <https://reviews.freebsd.org/D40111
For someone new to the release bits it's not always clear what files are
being created. Report the disk image name explicitly.
Reviewed by: gjb
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39953
When building ARM release images, enable IPv6 SLAAC by default in
addition to IPv4 DHCP.
Unlike amd64 (and other desktop/server) releases, ARM releases on SoC
setups are usually deployed by just using the installation image, so
there is no interactive network configuration. Not having IPv6
included by default is kind of an anachronism these days, given that
FreeBSD with the KAME project once pioneered IPv6 technology.
MFC after: 2 weeks
Provides an OCI (Oracle Cloud Infrastructure) release target for
Oracle's KVM-based VM implementation. Tested using 13.1-RELEASE,
primarily on Ampere CPU on A1.Flex VM shapes, but also works on
amd64 shapes.
- supports cloud-init and custom scripts
- provides a freebsd@ sudo-enabled user
- root user disabled over ssh & console
Approved by: gjb
Reviewed by: emaste
MFS after: 1 week
Sponsored by: The FreeBSD Foundation
Sponsored by: SkunkWerks, GmbH
Technical assistance from: Oracle
Differential Revision: https://reviews.freebsd.org/D34746
This will enable VM access to all ZFS feature automatically, only on a
newly installed or provisioned VM or cloud instance.
Reviewed by: markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D37283
Deprovision feature of waagent is used for preparing to capture a
running VM and turn it into a VM image. Using it in the process of
building a VM image from scratch will cause some side effects such as
the hostname of the building host getting reset.
Remove calling the deprovision command and use a simpler way to fulfill
the requirements of the Azure VM image.
Sponsored by: The FreeBSD Foundation
The change extends vmimage.subr to handle a new parameter, VMFS, which
should be equal to either "ufs" or "zfs". When it is set to ZFS, we use
makefs to create a bootable pool populated using the same dataset layout
as bsdinstall and "poudriere image" use. The pool can be grown using
the growfs rc.d script, just as in UFS images.
This will make it easy to provide VM and cloud images with ZFS as the
root filesystem. So far I did not do extensive testing of cloud images;
I merely verified that creation of ZFS-based AWS AMIs works and allows
me to create amd64 and arm64 EC2 instances with ZFS as the root
filesystem.
Reviewed by: emaste, gjb
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34426
We need to do a relative link to efi instead of an absolute link into
the build tree.
Sponsored by: Netflix
Reviewed by: gjb
Differential Revision: https://reviews.freebsd.org/D36941
I somehow introduced the typo when extracting one part of D34598.
Reported by: Jose Luis Duran <jlduran@gmail.com>
Fixes: 9871ae6aa9 ("Track kern.ipc.somaxconn -> ...")
Traditionally, we've used /boot/msdos for the MBR mount point for the SD
images that we produced. For GPT and bsdinstall, we've used
/boot/efi. Migrate to using /boot/efi for MBR as well and add a
/boot/msdos -> /boot/efi symlink for compatibility (which may disappear
before 14.0, but will remain on the stable branches).
When we first created the arm images, there was no EFI booting and the
FAT partion on an MBR image was used to hold the firmware, uboot.bin,
SoC config files and ubldr. When we transitioned to uboot with EFI, we
put the loader files in the same partition. Later we standardized on
/boot/efi at about the same time we added GPT support to the RE produced
images. We left the MRB case as /boot/msdos for legacy reasons and since
it wasn't always EFI. Later, we dropped support of non-EFI booting on
the RE produced images, so the duality of /boot/msdos diminished even
more. Since so little secondary meaning remains, putting it all in
/boot/efi standardizes the location and reflects the RE images
better as using efi-only booting.
In addition, always label the msdosfs partion 'efi'. While a small
misnomer on some systems that store other files in the ESP, it was
requested in review for more consistency for similar reasons to the
mountpoint rename. There was no way to have an 'alias' or 'second label'
here, so this breaks compatibility. Since the images are self-contained,
this was judged to be an acceptable change.
Sponsored by: Netflix
Reviewed by: manu, allanjude, emaste, gjb
Differential Revision: https://reviews.freebsd.org/D36635
We do not want CHS-based alignment for VM or SD card release images.
(Other images use makefs/mkimg, not kernel gpart and so do not depend
on this sysctl.)
Sponsored by: The FreeBSD Foundation
Approved by: re (gjb)
MFC after: 1 week
The latest version of the ec2-scripts package includes a completely
rewritten "use EC2 ephemeral disks for swap space" script. Now that
we have something which works on recent versions of FreeBSD, turn it
on since it's a great way to use the ephemeral disks.
Note that the option for controlling this, ec2_ephemeral_swap_enable,
is not the same as the option (ec2_ephemeralswap_enable) used with the
previous ephemeral-swap script; this change was deliberate to avoid
astonishment for users who upgraded their ec2-scripts package and had
a setting left behind in rc.conf.
This sshd_config keyword was replaced by KbdInteractiveAuthentication in
openssh 8.7, though ChallengeResponseAuthentication is silently accepted
as an alias. However, this means that the code in ec2.conf which
modifies a commented-out line no longer does anything. Apply a minimal
fix.
Reviewed by: cperciva, emaste
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34400
ubldr.bin was obsoleted by our uboot ports last year, so this is
completely unused in the default config (some customers still use
it, but that's not relevant to this script). Don't copy it at all
since it won't be used for re@ produced images.
Sponsored by: Netflix
Reviewed by: kevans, gjb (re@)
Differential Revision: https://reviews.freebsd.org/D34192
Disable Duplicate Address Detection in EC2 instances. The networking
configuration in EC2, with IPv6 addresses assigned by DHCPv6 and
host egress filtering, makes "duplicate addresses" impossible.
This speeds up the boot process in EC2 by 2 seconds.
Reviewed by: kp, imp, bz
MFC after: 2 weeks
Sponsored by: https://www.patreon.com/cperciva
Differential Revision: https://reviews.freebsd.org/D33091
Imports the changes for building official images on Azure Marketplace,
which fulfill the requirements of Azure and FreeBSD cloud images like
disk layout and UEFI for Gen2 VM, along with some minor improvements like
configurations to speed up booting.
"CLOUDWARE" list will be updated after some more collaborations with re
completed.
Reviewed by: re (gjb)
Sponsored by: The FreeBSD Foundation
Technical assistance from: Microsoft
Differential Revision: https://reviews.freebsd.org/D23804
This is useful for adding extra packages to the build of an AMI.
For example:
env VM_EXTRA_PACKAGES="zsh" make -C release ec2ami
Approved by: gjb
MFC after: 1 week
Sponsored by: Rubicon Communications, LLC ("Netgate")
Currently pkg(8) will fail to install any package if one is missing, so
make this a loop to prevent one missing package from preventing the rest
from installing. Seen building an AWS AMI for aarch64 on main and
ebsnvme-id is not available in the repo at the moment.
Approved by: gjb
MFC after: 1 week
Sponsored by: Rubicon Communications, LLC ("Netgate")
Per [(1), i.e. test(1), the string comparison operator should be `=`,
not `==` in sh(1) scripts.
No functional change. FreeBSD's test accepts `==` as an alias for `=`
for compatibility with gnu test's `==` extension.
Reviewed by: imp@, gjb@
Pull Request: https://github.com/freebsd/freebsd-src/pull/485
