While most "cloudware" (AWS, Azure, GCP, Vagrant) images get uploaded
to their respective clouds, we have two images -- BASIC-CI and
BASIC-CLOUDINIT -- which are published via the FTP mirrors. We have
been handling these using a manual and error-prone process of copying
them from the internal build directories into the FTP staging tree,
but they should be handled using the "make install ftp-stage"
mechanism which is used for all the other published images.
This commit 1. Adds a new 'make cloud-install' target which is run as
part of 'make install' and installs BASIC-CI and BASIC-CLOUDINIT images,
and 2. Extends 'make ftp-stage' in Makefile.mirrors to copy those bits
into the FTP staging area.
As a side benefit, this will also make BASIC-CI and BASIC-CLOUDINIT
images available for arm64; until now they have only been available
for amd64.
MFC after: 1 week
This missing dependency is only visible with -jN (which was otherwise
broken until recently) and only when building DVD images (which aren't
included in weekly snapshots and thereby slipped past testing).
MFC after: 3 days
The disc1 and bootonly directories have files distributed into them
for use in "full" and "mini" images; the former are disc1.iso and
memstick.img, and the latter is bootonly.iso and mini-memstick.img.
Unfortunately the scripts which package a directory tree into an ISO
or memory stick image also modify the directory, for example to
create an appropriate /etc/fstab file; so creating two images at the
same time breaks.
Resolve this by copying disc1 to disc1-disc1 and disc1-memstick,
and copying bootonly to bootonly-bootonly and bootonly-memstick,
before using those directories for constructing the ISO+memstick
images.
MFC after: 1 week
Place instances of "cd foo && bar" inside subshells for compatibility
with modern make(8) which uses a single shell for the duration of a
makefile target.
MFC after: 1 week
Components like base.txz and ports.txz are called distributions in the
installer, and with the introduction of pkgbase we will start dealing
with normal pkg packages in the installer. Rename EXTRA_PACKAGES to
DISTRIBUTIONS, and move base.txz and kernel.txz to that list.
This introduces no functional change but is a small cleanup in advance
of some pkgbase experimentation.
Reviewed by: cperciva
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D43544
mergemaster was deprecated some time ago and will be removed from
FreeBSD 15.
Reviewed by: imp
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D41797
This spits out errors but seemingly isn't actually fatal, so was missed.
Fixes: 6853d893c7 ("release: Automatically generate MK_LIB${LIBCOMPAT} and lib${libcompat}-dbg lists")
This makes it more obvious that the media being booted is an installer
rather than an installed system, which is otherwise hard to distinguish.
It also provides a more user-friendly, and more accurate, prompt.
Reviewed by: gjb
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D36419
This makes use of the new -N etcupdate flag and merges the resulting
METALOG into base.meta re-rooted to /var/db/etcupdate/current.
Reviewed by: gjb
Obtained from: CheriBSD
Differential Revision: https://reviews.freebsd.org/D35858
This is in preparation for non-FreeBSD builds where make is GNU make and
so etcupdate needs to know the name of or path to the bmake binary to
use for its own builds.
Reviewed by: gjb
Obtained from: CheriBSD
Differential Revision: https://reviews.freebsd.org/D35855
This is in preparation for non-FreeBSD and -DNO_ROOT builds. On
non-FreeBSD there is no host etcupdate to use, and -DNO_ROOT will
require additional flags that may not be supported by the host's
etcupdate when building on FreeBSD. Moreover, there's no guarantee
anyway that the host's etcupdate is quite right for the current tree;
upgrading from source only requires that the host's is good enough for
-p which just manually copies master.passwd and group, the rest of the
upgrade is done post-installworld. For example, should a new set of
autogenerated files be added that etcupdate is taught about, the host
won't know about them and so the bootstrapped current tree will
incorrectly contain them, leading to spurious diffs on the installed
system.
Reviewed by: gjb, delphij
Obtained from: CheriBSD
Differential Revision: https://reviews.freebsd.org/D35853
Our uses of tar rely on BSDisms, and so do not work in environments
where GNU tar is the default tar. Providing a TAR_CMD variable like
some other commands allows it to be overridden to use bsdtar in such
cases.
Reviewed by: brooks, delphij, gjb
Obtained from: CheriBSD
Differential Revision: https://reviews.freebsd.org/D35813
This requires a bunch of METALOG mangling to include the files we inject
into the tree. The mkisoimages.sh and make-memstick.sh scripts are now
called with the current directory inside the tree so that the relative
paths in the METALOG match up with the current directory. The scripts do
not require this when not using a METALOG, but for simplicity we always
do so. The Makefile mangles the real METALOG created from the install,
as those files are shared across all uses of the tree, but the shell
scripts create a temporary copy of the METALOG that they mangle as their
tree modifications are specific to that image. We also need to pass -D
to makefs to turn any duplicate METALOG entry errors into warnings, as
we have many (harmless) instances of those.
Whilst dvd1.iso should work, the !NOPKG code will need more work to
support this.
All media will also lack mergemaster and etcupdate trees, since more
work is needed to add -DNO_ROOT modes to them. Users of install media
built this way will have to manually bootstrap them.
Reviewed by: brooks, gjb
Differential Revision: https://reviews.freebsd.org/D33999
While here, fix all links to older en_US.ISO8859-1 documentation
in the src/ tree.
PR: 255026
Reported by: Michael Büker <freebsd@michael-bueker.de>
Reviewed by: dbaio
Approved by: blackend (mentor), re (gjb)
MFC after: 10 days
Differential Revision: https://reviews.freebsd.org/D30265
To support -DNO_ROOT work. The top-level installworld target creates a
new METALOG starting with `#mtree 2.0` so it needs to be first, to avoid
overwriting installkernel METALOG entries.
Reviewed by: gjb
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D29582
Following 7b1d1a1658, the structure
for the reldoc target has significantly changed as result of the
ASCIIDoctor/Hugo migration. As the release notes related files
on the installation medium are inherently out of date, purge them
entirely.
Discussed within: re, doceng
No objection: re (silence), doceng (silence)
Timeout: 2 weeks
MFC after: 1 week
MFC to: stable/13, stable/12, and stable/11 only
Sponsored by: Rubicon Communications, LLC ("Netgate")
The 'reldoc' target includes release-related documentation on
installation medium. Since the switch from XML to ASCIIDoctor,
the file locations have moved, and it will take some time to sort
out how this target should work now.
MFC after: 3 days
Sponsored by: Rubicon Communications, LLC ("Netgate")
For platforms that don't have any of the memstick, cdrom, or dvdrom
release images (i.e. riscv64), the release-install target will trip up
when invoking md5(1) on the non-existent image files. Skipping this
allows the install to complete successfully.
- New file. Adds logic to search for the git binary, as well
as determining the branch and revision, used in various
places.
Makefile:
- Remove searching for the svn{,lite} binary.
Makefile.ec2:
- Reduce duplicated code, removing searching for the svn{,lite}
binary, in addition to EC2_SVN{BRANCH,REV}.
- Rename EC2_SVN* with GIT* for consistency.
Makefile.mirrors:
- Remove the SRCBRANCH declaration, replaced with the exported
GITBRANCH variable.
- Update _SNAP_SUFFIX from SVNREVISION to GITREV, and remove
the leading 'r' from it, since it will break git hashes.
- Remove yet another instance of duplicated code to search for
the svn{,version}lite binary.
Sponsored by: Rubicon Communications, LLC (netgate.com)
autofs was introduced with FreeBSD 10.1 and is the supported method for
automounting filesystems. As of r296194 the amd man page claimed that it
is deprecated. Remove it from base now; the sysutils/am-utils port is
still available if necessary.
Discussed with: cy
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
src branch is used when generating the hardware.html page.
MFC after: 3 days
MFC before: 12.0-BETA3
MFC with: r339873
Sponsored by: The FreeBSD Foundation
This feature is disabled on install media as these generally won't have any
interesting kernels to be listed other than the default kernel, so the
potential performance penalty in these situations likely isn't worth it.
Approved by: re (kib)
target.
The doc/share/mk/doc.commands.mk sets SVN to /usr/local/bin/svn
by default, which is not necessarily installed by the documentation
project textproc/docproj port.
Ensure SVN can be evaluated properly to include the hardware pages
by iterating through /usr/local/bin and /usr/bin and looking for
both svn and svnlite binaries, and pass the SVN variable explicitly
through env(1) in the reldoc target to avoid failures if it does not
exist.
Approved by: re (rgrimes)
Sponsored by: The FreeBSD Foundation
There are several scripts and targets solely used to generate install
media, make sure DB_FROM_SRC is used in that case in order to prevent
checking the host database, which is irrelevant when generating
install binaries.
Sponsored by: Citrix Systems R&D
PR: 230459
Reviewed by: gjb
Differential revision: https://reviews.freebsd.org/D16638
All manpages in base are now compatible with mandoc(1), all roff documentation
will be relocated in the doc tree. man(1) can now use groff from the ports tree
if it needs.
Also remove checknr(1) and colcrt(1) which are only useful with groff.
Approved by: (no objections on the mailing lists)
Notably, this removes the toolchain from the CD-ROM image (it remains on DVD
images) and pushes the CD-ROM image well below the 700 MB mark.
MFC after: 10 days
indicates the 'release' target has run (in order to prevent
subsequent invocations that may clobber original build output).
As is, the 'release' target is a dummy target that does nothing
more than depend on subsequent targets. Unless 'make obj' is
invoked prior to 'make release', .OBJDIR and .CURDIR will always
be '/usr/src/release' (or wherever /usr/src is located).
When 'make release' invokes 'make real-release' (and subsequent
targets), .OBJDIR is not updated, which still leads to src/ tree
pollution.
While arguably a hack, 'make release' will invoke the original
dummy targets as originally intended, but instead of touching an
empty file (or returing @true), will call a 'release-done' target
that will trigger the behavior that was intended to prevent
a subsequent invocation.
Discussed with: hrs
MFC after: 3 days
X-MFC-With: r293173
Sponsored by: The FreeBSD Foundation
- Rework MANIFEST generation and parsing via bsdinstall(8).
- Allow selecting debugging distribution sets during install.
- Rework bsdinstall(8) to fetch remote debug distribution sets
when they are not available on the local install medium.
- Allow selecting additional non-GENERIC kernels during install.
At present, GENERIC is still required, and installed by default.
Tested with: head@r293203
Sponsored by: The FreeBSD Foundation
This was discussed during the 10.2-RELEASE cycle, however
since we were nearing the end of the cycle, we decided to
defer this change until after 10.2-RELEASE.
Reminded by: so (delphij), jmg
MFC after: 5 days
Sponsored by: The FreeBSD Foundation
extra bits from an "xtra-bits-dir". This feature is unusable
from release/Makefile. Add an XTRADIR setting to use it.
Differential Revision: https://reviews.freebsd.org/D3633
Reviewed by: kmacy
MFC after: 3 weeks
X-MFC-to: stable/10
Relnotes: yes
disk images to a directory within DESTDIR.
Add CLOUDINSTALL variable to automatically generate the
actual list of install targets. While here, rename the
cloud-specific names from 'vm-' to 'cw-' to avoid naming
collisions.
Add AZURE_DISK, GCE_DISK, and OPENSTACK_DISK output file
variables.
Add 'cloudware-install' to the 'install' target.
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
libarchive(3) doesn't support the new liblzma API yet, but this change
allows us to enable multi-threaded xz compression.
``make release'' should now finish in half the time on a machine with
several cores and fast disks (our typical build server).
This behaviour only applies when building a release and it doesn't
affect buildworld/installworld. To disable threaded xz compression,
set XZ_THREADS=1.
Reviewed by: gjb
Tested by: gjb
I was going to use __FreeBSD_version to determine if
xz(1) should be multi-threaded by default, but doing
this will cause problems if/when the changes are merged
from head.
Sponsored by: The FreeBSD Foundation
