Commit graph

1849 commits

Author SHA1 Message Date
Ed Maste 20a66ab4bf Retire CLANG_IS_CC option
A small reduction in build infrastructure complexity; when we had both
Clang and GCC in the tree it was useful to have both built, and choose
one or the other to install as /usr/bin/cc.  Now only Clang is in the
tree, and there is no point in building and installing base Clang but
not providing it as cc (and c++, cpp).

Reviewed by:	imp
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D37075
2022-11-21 14:56:13 -05:00
John Baldwin b9cb80883b Don't omit bsd.compiler.mk for the nested delete-old in buildworld.
src.opts.mk will need bsd.compiler.mk to determine if CXX is
supported.  Without this, src.opts.mk always marks CXX as broken and
attempts to delete all dependencies of MK_CXX from WORLDTMP.

Reviewed by:	imp
Differential Revision:	https://reviews.freebsd.org/D36890
2022-11-18 20:11:07 -08:00
Ed Maste 470fb726f3 build: Use rm -fv for BATCH_DELETE_OLD_FILES
It's possible to have files with odd permissions in the tmproot (or
sysroot), causing rm to prompt for each one during e.g. buildworld.
Add -f to forcibly delete these.

Reviewed by:	brooks
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D37111
2022-10-25 11:18:55 -04:00
Brooks Davis 77becb9325 Rename MACHINE_ABI and TARGET_ABI
The MACHINE_ABI and TARGET_ABI variables are used to set the middle of
the target triple (e.g., "-unknown-" or "-gnueabihf-"). They are not set
by any tool in the base system and I've only found the latter mentioned
in one review online. As such, rename them to to MACHINE_TRIPLE_ABI and
TARGET_TRIPLE_ABI to clear the way to use MACHINE_ABI as a supplement to
MACHINE_CPU, etc.

Reviewed by:	imp, jhb
Obtained from:	CheriBSD
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D36420
2022-10-05 17:27:44 +01:00
Dag-Erling Smørgrav 0aa2700123 Put OPIE to rest.
Differential Revision: https://reviews.freebsd.org/D36592
2022-10-02 03:37:29 +02:00
Jessica Clarke fcf1208158 Makefile.inc1 release bsd.own.mk: Introduce and use TAR_CMD
Our uses of tar rely on BSDisms, and so do not work in environments
where GNU tar is the default tar. Providing a TAR_CMD variable like
some other commands allows it to be overridden to use bsdtar in such
cases.

Reviewed by:	brooks, delphij, gjb
Obtained from:	CheriBSD
Differential Revision:	https://reviews.freebsd.org/D35813
2022-07-15 23:04:34 +01:00
Jessica Clarke 5e0a749cf1 Makefile.inc1: Set LC_COLLATE in distributeworld for glibc compatibility
distributeworld relies on "foo" sorting directly before "foo type=...",
but with glibc both en_US and en_GB have "fooa" sort between "foo" and
"foo z", resulting in some files (in particular, id due to "ident"
sorting before "id type=" but after "id") not being included in the meta
files and thus not included in the dist tarballs. Forcing use of the C
locale ensures this does not occur.

Reviewed by:	brooks
Obtained from:	CheriBSD
Differential Revision:	https://reviews.freebsd.org/D35812
2022-07-15 23:04:26 +01:00
Jessica Clarke 445421abc9 Makefile.inc1: Honour DB_FROM_SRC for NO_ROOT distributeworld
Currently the host's database files are used, but on non-FreeBSD these
are not necessarily sufficient; in particular, Linux does not have a
wheel group. Instead, use -N to use the in-tree database files when
creating the METALOG entries, as is done for the recursive makes via
IMAKE_MTREE.

Reviewed by:	brooks
Obtained from:	CheriBSD
Differential Revision:	https://reviews.freebsd.org/D35810
2022-07-15 23:03:49 +01:00
Dimitry Andric 93f5ab129a Bootstrap crunchgen after removing -dc from linker invocation
In ec81497cc7 crunchgen was updated to remove -dc from the linker
invocations in its generated makefile output, as this flag is no longer
necessary, and is going to be an error with lld 15.

Update the BOOTSTRAPPING conditions for copying the crunchgen binary
from the host, or actually bootstrapping it when necessary. Since
ec81497cc7 did not bump __FreeBSD_version, I have chosen the nearest
values.

Fixes:		ec81497cc7
MFC after:	3 days
2022-07-14 23:14:18 +02:00
Jessica Clarke 7e45839aca Makefile.inc1: Fix -DNO_ROOT distributeworld certctl usage
Currently for distributeworld we pass DESTDIR to certctl.sh as an
environment variable, which sets the default value in the script.
However, for -DNO_ROOT builds, CERTCTLFLAGS has METALOG_INSTALLFLAGS
which includes -D ${DESTDIR}, overriding the custom DESTDIR pointing at
the base dist directory.

Moreover, in order to ensure that the METALOG includes the base/ prefix
for all the files, we need to have certctl call install with -D set to
DESTDIR/DISTDIR without the /base suffix but also ensure the files get
installed to DESTDIR/DISTDIR/base.

Fix these by passing the custom DESTDIR to certctl via -D rather than in
the environment and to pass the /base suffix in the distributeworld case
via the newly-added -d option.

We also need to run certctl rehash before we generate the .meta files
from the METALOG, not after, otherwise they won't include the METALOG
additions, so move the certctl rehash call.

Finally, add a missing semicolon that results in no message being
printed in the missing openssl case. By not including the semicolon,
else echo "..." is treated as extra arguments to certctl, which is lax
in its argument parsing and ignores additional arguments, and the
semicolon and fi after the intended echo terminate the if statement as
normal so there's no syntax error at the shell level. This is harmless
as we weren't trying to do anything other than echo anyway, all that
happens is the echo doesn't actually get run.

Reported by:	markj (missing semicolon)
Reviewed by:	brooks, kevans
Obtained from:	CheriBSD
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D35809
2022-07-14 01:25:50 +01:00
Jessica Clarke b180713612 Makefile.inc1: Add a missing ${dist} to distributeworld for usr/include/i386
Not including ${dist} results in the following non-fatal error printed
once per extra distribution:

  mkdir  //usr/obj/usr/src/amd64.amd64/release/dist/usr/include/i386
  mkdir: //usr/obj/usr/src/amd64.amd64/release/dist/usr/include: No such file or directory
  *** Error code 1 (ignored)

Also fix a whitespace nit on this line whilst here.

Reviewed by:	brooks
Fixes:		a09ea2bbc3 ("amd64: add an i386 include directory")
2022-07-14 00:23:31 +01:00
Brooks Davis bda5d2a45c installworld: improve portability of ldd use
b3b462229f added a case statement to ignore lines containing strings
in square brackets such as "[vdso]" and "[preloaded]". On MacOS
Monterey where /bin/sh may be zsh, this fails with:

/bin/sh: -c: line 0: syntax error near unexpected token `;;'

Invoke grep in the pipeline to remove such lines instead.

Reviewed by:	emaste
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D35618
2022-07-01 08:33:16 +01:00
Brooks Davis a09ea2bbc3 amd64: add an i386 include directory
This directory will hold i386-specific headers that are needed for
-m32 support on amd64 and where the amd64 and i386 cases have too
little in common for combining them to make sense.  Files to be
installed will come in later commits.

With the currently required set of files, this could be done with
another INCGROUP in include/Makefile, but at least one file that
might want -m32 support (ieeefp.h) conflicts with a files installed
in /usr/include.

Reviewed by:	jhb, imp
2022-06-13 18:35:38 +01:00
Dimitry Andric 1838bd0f48 Merge llvm-project release/14.x llvmorg-14-init-18315-g190be5457c90
This updates llvm, clang, compiler-rt, libc++, libunwind, lld, lldb and
openmp to llvmorg-14-init-18315-g190be5457c90.

PR:		261742
MFC after:	2 weeks
2022-05-14 13:45:25 +02:00
Alfredo Dal'Ava Junior f9ee415674 build target triple variable from sys/conf/newvers.sh
Retrieve FreeBSD revision number directly from sys/conf/newvers.sh
when building the compiler target triple value, avoiding manual
intervention on other files every new release.

Reviewed by:	imp
MFC after:	2 months
Sponsored by:	Instituto de Pesquisas Eldorado (eldorado.org.br)
Differential Revision:	https://reviews.freebsd.org/D34429
2022-05-03 08:09:38 -03:00
Piotr Pawel Stefaniak 47bcbde91d bintrans: move files to a new directory
And reflect the change in various places.
2022-04-18 10:53:11 +02:00
Piotr Pawel Stefaniak 4cd4841a27 Modularize uuencode and uudecode by wrapping them in bintrans.c
The program will be installed as bintrans, uuencode, uudecode,
b64encode, and b64decode and will be responsible for running the coders
according to their historical behavior.

Additionally, bintrans will be able to take a parameter designating
the coder and accept all its options in this form:
bintrans <coder> [options]
and the behavior should be the same as if
<coder> [options]
was invoked.
This has the advantage that adding coders won't require installing them
as binaries.

Move uudecode files to uuencode since the latter is the one that
provides the manual page.

Reviewed by:	delphij (previous version)
Differential Revision:	https://reviews.freebsd.org/D32943
2022-04-18 10:46:44 +02:00
Ed Maste a8267ecc3d Speed up *-old-* make targets by using sed instead of xargs
Targets like 'list-old-files' used "xargs -n1" to produce a list with
one file per line.  Using xargs resulted in one fork+exec for each
Argument, resulting in rather long runtime.  Instead, use sed to split
the list.  On one machine `make list-old-files` took 30s wall clock time
with xargs and less than 1s with sed.

Reviewed by:	jhb
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D34741
2022-04-08 20:41:10 -04:00
Ed Maste b3b462229f installworld: handle ldd including preloaded objects
The installworld target makes a temporary copy of binaries to be used
during the install.  Libraries that they depend on are also included,
found by using `ldd`.

After commit 0913953c9e ldd started listing preloaded objects,
including [vdso], under a [preloaded] header.  Skip ldd output that is
enclosed in square brackets.

Reviewed by:	cy, kib [earlier version]
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D34734
2022-04-01 12:26:49 -04:00
Warner Losh 3b17e19f02 bootstrap: bump minimum supported version
Bump the minimum supported version to build -current from to 11.3R in
preparation of removing support for older systems.  11.4R was selected
as the most recent version to go out of support.

Sponsored by:		Netflix
Reviewed by:		delphij, emaste
Differential Revision:	https://reviews.freebsd.org/D32444
2022-03-01 22:55:28 -07:00
Warner Losh 33550b4739 bootstrap: No need for kbdcontrol bootstrap anymore
We only need kbdcontrol when bootstrapping from FreeBSD 10 or
pre-FreeBSD 11.0 current. Since we can no longer build from these
versions of FreeBSD, remove the support for bootstrapping them.

Sponsored by:		Netflix
2022-03-01 22:55:24 -07:00
Warner Losh 3e65b7d94d cleankernel: A target to delete the kernel compile file
With the meta-build, it's always a NO_CLEAN build. Provide a way to
remove so one can rebuild from scratch. 'cleankernel' will delete the
kernel and modules object directories. Document this in build(7).

Sponsored by:		Netflix
Reviewed by:		debdrup, markj
Differential Revision:	https://reviews.freebsd.org/D32978
2022-02-11 12:51:24 -07:00
Ed Maste ea9a92d210 Makefile.inc1: synthesize PKG_ABI from newvers.sh variables
Previously we inspected ${WSTAGEDIR}/usr/bin/uname to determine PKG_ABI,
but the file will not exist in some cases - for example, if building
only kernel packages.  We can instead synthesize the PKG_ABI from
information already provided by newvers.sh.

Reviewed by:	kevans, manu (both earlier rev)
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D34249
2022-02-11 11:26:07 -05:00
John Baldwin bca92be683 Remove more quotes around Makefile .error/.warn/.info strings.
Reviewed by:	imp, emaste
Differential Revision:	https://reviews.freebsd.org/D34175
2022-02-04 15:59:53 -08:00
John Baldwin 0a902e4261 Stop trimming 'hf' suffixes from MACHINE_ARCH.
This was only used for MIPS hard-float architectures.

Reviewed by:	imp
Sponsored by:	The University of Cambridge, Google Inc.
Differential Revision:	https://reviews.freebsd.org/D34018
2022-01-25 11:31:17 -08:00
Ed Maste 6f6fbfa3a8 Remove quotes around Makefile .error/.warn/.info strings
The text after .error et al is emitted verbatim.

Reviewed by:	sjg
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D33904
2022-01-22 14:03:07 -05:00
John Baldwin 77da558ceb check/delete-old: Auto-generate lib32 entries for libraries.
- Add usr/lib32/libfoo.so.N for lib/libfoo.so.N.

- Add usr/lib32/foo for usr/lib/foo.

- Treat casper libraries special since they are installed to
  /usr/lib32 instead of /usr/lib32/casper and thus map
  usr/lib/casper/foo to usr/lib32/foo.

Note that OLD_DIRS and MOVED_LIBS entries are not duplicated, only
OLD_FILES and OLD_LIBS.

Reviewed by:	imp, emaste
Sponsored by:	The University of Cambridge, Google Inc.
Differential Revision:	https://reviews.freebsd.org/D33324
2022-01-20 14:09:57 -08:00
John Baldwin 57ba3f00f5 Use MOVED_LIBS for libraries moved between /usr/lib and /lib.
Add a MOVED_LIBS variable similar to OLD_LIBS except that MOVED_LIBS
is used for the cases that a library's name doesn't change, but it
just moves between /usr/lib and /lib.  This will be used by a future
change to auto-generate lib32 old files entries for which these cases
need to be ignored (a moved library remains in /usr/lib32).

Suggested by:	emaste
Reviewed by:	emaste
Sponsored by:	The University of Cambridge, Google Inc.
Differential Revision:	https://reviews.freebsd.org/D33848
2022-01-20 12:47:43 -08:00
John Baldwin 21baf42c57 Add list-old-{dirs,files,libs} targets.
These targets generate a raw list of the candidate old files roughly
corresponding to the values of OLD_DIRS, OLD_FILES, and OLD_LIBS.
Currently list-old-files also includes uncompressed manpages in
addition to compressed manpages.

Use these targets in the implementation of check-old-* and
delete-old-* to replace duplicated logic.

Reviewed by:	imp, emaste
Sponsored by:	The University of Cambridge, Google Inc.
Differential Revision:	https://reviews.freebsd.org/D33327
2022-01-11 11:38:11 -08:00
Ed Maste adc88b9c01 Build libclang also if LLDB is enabled
LLDB depends on libclang as it uses Clang as the expression parser.
Previously setting WITHOUT_CLANG but leaving LLDB enabled (as default)
resulted in a build failure.

Users who set WITHOUT_CLANG in order to reduce build time or size
might want to set WITHOUT_LLDB in addition to WITHOUT_CLANG, or use
WITHOUT_TOOLCHAIN instead.

PR:		260993
Reported by:	eugen
Reviewed by:	dim
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2022-01-07 15:23:55 -05:00
Warner Losh b68d6892ba libsoft: Remove targets to generate libsoft
Remove the targets needed to make and install libsoft.

Sponsored by:		Netflix
2022-01-06 22:34:15 -07:00
Warner Losh 7a171e3083 test-includes: Build a library instead of a binary
We need to build the .o's, but don't need to link. Build a static
library instead of a binary to accmoplish this. This removes the need to
have all the libc and crt stuff built, which is required for a
binary. In addition, trying to build a non-standard binary runs into
trouble with undefined symbols on arm related to EABI, even when using a
simplified startup with -nostdlib -e start.

Turn back on testing includes, now that it works.

Sponsored by:		Netflix
Reviewed by:		brooks, markj (prior version)
Differential Revision:	https://reviews.freebsd.org/D33452
2021-12-15 15:35:24 -07:00
Warner Losh 6a2b72bf1a Remove test-includes build
This is making the CI red. I believe this is because we do a clean build
w/o metamode in the tinderbox, so none of the artifacts needed to build
a binary are present. However, I've not recreated the problem locally
yet to confirm. Remove this while I investigate. This partially reverts
dd55767b86. The rest of the commit causes no harm w/o the explicit
test here.

Sponsored by:		Netflix
2021-12-14 21:42:47 -07:00
Warner Losh dd55767b86 Test various header files to ensure they can be included by themselves.
A number of header files in sys/* have, going back to 7th Edition Unix
in 1979, reqiured other files (like sys/types.h) to compile. Likewise
the 4BSD networking code has had prerequisites. However, going back to
around the turn of the 21st century, other systems have made them be
independently include-able (wide-spread header include protection
post-dates 7th edition Unix by maybe 3 or so years judging from USENET
source postings). Start down the path of making them all independently
include-able by creating this test that fails buildworld when they are
not.

The file 'badfiles.inc' contains a list of the currently broken files
that cannot be included w/o any prerequisites. As files are fixed, 'make
badfiles.inc' should be re-run to remove them from the list. Note: All
files that start with an underscore are considered internal and not
tested.

Please note: once a file is removed from badfiles.inc, it must pass on
all architectures. Buildworld through at least the _includes target is
needed to ensure its working (though a buildkernel should also be done
on all architectures as well).

Sponsored by:		Netflix
Reviewed by:		brooks, markj
Differential Revision:	https://reviews.freebsd.org/D32498
2021-12-14 20:25:37 -07:00
Warner Losh b8cacb3898 mips: Remove mips as a recognized build target.
Remove mips as a recognized build target in advance of its removal from
the tree.

Sponsored by:		Netflix
2021-12-09 08:59:14 -07:00
Emmanuel Vadot 59f4abfaac pkgbase: Allow to specify PKG_ABI on the command line
This is useful for creating kernel package without having to buildworld
before as we use uname from the world stage dir to get the ABI.

Reviewed by:	emaste, imp
Differential Revision:	https://reviews.freebsd.org/D33323
Sponsored by:	Beckhoff Automation GmbH & Co. KG
2021-12-08 08:40:22 +01:00
Jessica Clarke 8f7eb330f7 Makefile.inc1: Bootstrap libcrypt on non-FreeBSD for libroken
After f0a6ab51e6, we need libroken's dependencies to be present; in
particular, libcrypt is not present on macOS, and so needs to be
bootstrapped. We include this on Linux too for consistency and to avoid
future compatibility issues, even if strictly unnecessary currently.

Fixes:		f0a6ab51e6 ("src.libnames.mk: Include dependencies when bootstrapping from non-FreeBSD"
MFC after:	1 week
2021-12-07 00:24:20 +00:00
Jessica Clarke 8d5d329553 Bootstrap libz when cross-building from non-FreeBSD
This is needed now libdwarf depends on libz.

Fixes:		dbf05458e3 ("libdwarf: Support consumption of compressed ELF sections")
MFC after:	1 week
2021-12-06 19:16:28 +00:00
Konstantin Belousov 9b135747a3 kern.pre.mk: allow to use and customize elfdump during kernel build
Reviewed by:	emaste
Discussed with:	jrtc27
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D32960
2021-12-06 20:46:49 +02:00
Juraj Lutter aaa4cd7f3c Makefile.inc1: Silence an error in make packages
Silence an error in "packages" target, like:

eval: /usr/src: Permission denied

on systems without subversion present.

Reviewed by:	emaste
Approved by:	emaste
MFC after:	1 week
Differential Revision: https://reviews.freebsd.org/D33153
2021-11-30 09:53:20 +01:00
Mark Johnston 7b3642da21 Hoist cddl prebuild lib dependency definitions out of a MK_ZFS block
The compilation of several libraries under cddl/lib is not conditional
on MK_ZFS = "yes", so their dependency on libspl is not conditional
either.  Unbreak buildworld when WITHOUT_ZFS is set.

Reported by:	bz
Fixes:		9e9c651cac ("cddl: fix missing ZFS library dependencies")
MFC after:	1 week
2021-11-26 09:39:09 -05:00
Ed Maste 483a226238 Add ncurses/form to _prebuild_libs to fix build
Reported by:	bz
2021-11-24 14:39:37 -05:00
Warner Losh 8502220db5 Revert "Bootstrap: Prune building from pre-FreeBSD 11 support"
This reverts commit a420a672bc.

kevans pointed out some potential problems here, so reverting until
I can fix them.
2021-11-19 00:10:21 -07:00
Warner Losh a420a672bc Bootstrap: Prune building from pre-FreeBSD 11 support
We don't need to bootstrap lex or md4 anymore.
Cat doesn't need to be bootstrapped (but is needed for buildkernel)
cruncgen doesn't need to be bootstrapped at all.
kbdcontrol isn't needed

Sponsored by: Netflix
Pull Request: https://github.com/freebsd/freebsd-src/pull/554
2021-11-18 23:13:44 -07:00
Warner Losh 27a04f5964 Add warning that MIPS is being removed in FreeBSD 14.0
MFC After:		3 days
Sponsored by:		Netflix
Reviewed by:	brooks, jhb
Differential Revision:	https://reviews.freebsd.org/D32853
2021-11-18 21:22:56 -07:00
Greg V 9e9c651cac cddl: fix missing ZFS library dependencies
In 9fae47666 zfsd got a libspl dependency to avoid undefined references.
However that workaround did not help external consumers of libzfs_core.

Fix all missing dependencies lld 13 and the rtld complain about.

Reviewed by:	freqlabs, markj
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D32521
2021-11-15 09:42:07 -05:00
Konstantin Belousov e5adb145f0 Remove arm/linux from sysent toplevel target
Sponsored by:	The FreeBSD Foundation
Fixes:	65e485014b
2021-11-10 21:18:53 +02:00
Mateusz Piotrowski eab5358b90 Makefile.inc1: Remove mentions of removed target "update"
This is follow-up to commits e290182bcf and 1f7d11e636.
2021-10-24 21:07:45 +02:00
Ed Maste 1a724aa972 Makefile.inc1: remove lorder from build/cross tools
As lorder is not used by the base system build there is no need to
include it in bootstrap-tools or NXBDIRS.

Fixes: 		0e1e341b48 ("Stop using lorder and ranlib when...")
Sponsored by:	The FreeBSD Foundation
2021-10-18 19:29:22 -04:00
Kyle Evans e2e2c622c3 native-xtools: avoid libllvm while populating the sysroot
Prior to 021385aba5, MK_CLANG=no was sufficient to avoid descending
into lib/clang, but the referenced change added a couple of other
enabling knobs. Turn those off, too, to continue avoiding libllvm.

With this change, we no longer end up with a libllvm using the wrong
default target triple; `poudriere jail -cx` works once again.

Reported by:	bhughes, imp, probably others
Fixes:	021385aba5 ("Add WITH_LLVM_BINUTILS to install LLVM ...")
2021-10-13 04:41:17 -05:00
Warner Losh c05b382edb Revert "bootstrap: No need to disable shared libraries for bootstrap tools"
This reverts commit e8f26e5dc8.

Although the change worked locally, it's breaking something in the CI
build for the riscv64 build (which makes no sense it would only break
that since we're building host tools to bootstrap at that point).

Sponsored by:		Netflix
2021-10-11 11:49:48 -06:00
Warner Losh e8f26e5dc8 bootstrap: No need to disable shared libraries for bootstrap tools
There's no need to disable shared libraries when building the bootstrap
tools. This was added on 2000 (commit ad879ce955) when the perl
bootstrap was added (libperl and miniperl) and saved a fair amount of
time (perl took a long time to build on 2000-era hardware).

For many years now, however, we rarely build any libraries when
bootstrapping. Even when we do, the optimization saves at most a few
seconds when upgrading since the libraries built have been small. Shared
libraries are more robust accross versions that static libraries due to
creaping dependencies (we aren't crossing versions of share libraries,
though, just using what's on the host). In addition, linux and macos
have been building like this for some time because static binaries on
those systems are difficult to impossible.

Sponsored by:		Netflix
Reviewed by:		arichardson, bapt
Differential Revision:	https://reviews.freebsd.org/D32443
2021-10-11 11:20:07 -06:00
Kyle Evans 4d7876aa70 build: slap a libtinfo dependency on libncurses
This mirrors the SUBDIR_DEPEND in lib/ncurses/Makefile.

Reported by:	jenkins (e.g., riscv64 build #23984)
Fixes:	396851c20a ("ncurses: split libtinfo from libncurses")
2021-10-06 22:03:37 -05:00
Baptiste Daroussin cbc83e378a ncurses: chase dependency changes in the source tree
Differential Revision:	https://reviews.freebsd.org/D32098
2021-10-04 11:38:24 +02:00
Konstantin Belousov cf0ee8738e Drop cloudabi
According to https://github.com/NuxiNL/cloudlibc:
CloudABI is no longer being maintained. It was an awesome experiment,
but it never got enough traction to be sustainable.

There is no reason to keep it in FreeBSD.

Approved by:	ed (private mail)
Reviewed by:	emaste
Sponsored by:	The FreeBSD Foundation
Differential revision:	https://reviews.freebsd.org/D31923
2021-09-22 00:18:44 +03:00
Doug Rabson 895545d0e6 Fix spelling of BUILDENV_SHELL 2021-09-11 13:27:11 +01:00
Alex Richardson 021385aba5 Add WITH_LLVM_BINUTILS to install LLVM binutils instead of Elftoolchain
When WITH_LLVM_BINUTILS is set, we will install the LLVM binutils as
ar/ranlib/nm/objcopy/etc. instead of the elftoolchain ones.
Having the LLVM binutils instead of the elftoolchain ones allows us to use
features such as LTO that depend on binutils that understand LLVM IR.
Another benefit will be an improved user-experience when compiling with
AddressSanitizer, since ASAN does not symbolize backtraces correctly if
addr2line is elftoolchain addr2line instead of llvm-symbolizer.
See https://lists.freebsd.org/archives/freebsd-toolchain/2021-July/000062.html
for more details.

This is currently off by default but will be turned on by default at some
point in the near future.

Reviewed By:	emaste

Differential Revision: https://reviews.freebsd.org/D31060
2021-09-06 09:49:49 +01:00
Jessica Clarke 1e4c802913 Fix bootstrapping to actually build lldb-tblgen for later use
Because MK_LLDB=no is in BSARGS, the bootstrap-tools recursive make does
not add lldb-tblgen to _clang_tblgen, causing it to not be built. This
means that the build currently always uses the host's lldb-tblgen
(which, whilst currently it appears to work, could in future break if
TableGen backends are added or altered) and, if it doesn't exist (either
because the current FreeBSD system was built with it disabled, or you're
building on macOS/Linux), fails. Linux and macOS cross-builds used to
work simply because LLDB was previously in BROKEN_OPTIONS when building
on non-FreeBSD.

Instead, move MK_LLDB=no from BSARGS to XMAKE. This ensures that the
lib/clang build in cross-tools continues to not build LLDB parts for the
bootstrap toolchain (both to save time/space on FreeBSD, and because our
vendored LLDB does not include the macOS and Linux host files so those
would fail to build).

The DIRDEPS target is updated to move MK_LLDB=no from the BSARGS block
that mirrors Makefile.inc1 to the line that disables additional
toolchain components. The DIRDEPS build likely suffers from the same
issue currently, but having never used it and not being familiar with
how it works I am leaving that as-is. If it does suffer from the same
issue it should be easily reproducible by renaming /usr/bin/lldb-tblgen
or moving it to a directory not in PATH.

Fixes:		31ba4ce889 ("Allow bootstrapping llvm-tblgen on macOS and Linux")
MFC after:	1 week
Reviewed by:	dim, emaste, imp
Differential Revision:	https://reviews.freebsd.org/D31531
2021-08-24 15:04:25 +01:00
Jessica Clarke ab3a18095f Makefile.inc1: Make sure sub-makes see MK_CLANG_BOOTSTRAP=no when XCC is a path
Currently we override MK_CLANG_BOOTSTRAP to no so we don't build a
bootstrap compiler, but subdirectories don't see that and so the hack in
bsd.sys.mk to prefer our includes over Clang's resource dir for external
toolchains is not enabled unless you use -DWITHOUT_CLANG_BOOTSTRAP
explicitly on top of XCC (which tools/build/make.py does not do),
causing duplicate definition errors when building rtld-elf due to the
use of -ffreestanding (Clang's stdint.h will use the system one when
hosted, but its own when freestanding, and only has glibc's preprocessor
guards, not FreeBSD's).

This broke when dropping CLANG_BOOTSTRAP from BROKEN_OPTIONS.

Fixes:		31ba4ce889 ("Allow bootstrapping llvm-tblgen on macOS and Linux")
MFC after:	1 week
Reviewed by:	imp, arichardson
Differential Revision:	https://reviews.freebsd.org/D31529
2021-08-24 15:04:25 +01:00
Bjoern A. Zeeb 298ee47e19 localedef: unbreak WITHOUT_LOCALES
After 0fa5403d49 ("pkgbase: move locales into their own package") we
need usr.bin/localedef as a bootstrap tool independent on where
WITHOUT_LOCALE was specified as we ALWAYS process C.UTF-8.

At the same time LOCALES= in the local Makefile is empty but
C.UTF-8 with WITHOUT_LOCALES. C.UTF-8 is excluded from FILES, and thus
after the replacement FILES= is set to only .LC_CTYPE which results in
a build failure not knowing how to build that. Tweak the substitution to
replace only non-empty words so that FILES remains harmlessly empty.

Reviewed by:	imp
Differential Revision:	https://reviews.freebsd.org/D31589
2021-08-19 12:38:17 -05:00
Bryan Drewery 4b7bf84248 bootstrap-tools: Don't copy host tools with make -n 2021-08-17 12:26:03 -07:00
Bryan Drewery a3ddd7c92b _worldtmp: Fix some sub-makes not working with make -n 2021-08-17 12:06:40 -07:00
Alex Richardson cc1345056b Add missing lex/yacc dependency for mkesdb/mkcsmapper bootstrap
This causes build failures on macOS where the build can end up invoking
an incompatible m4 binary.

Fxies:		2de949cf85 ("Remove mkcsmapper_static and mkesdb_static from build-tools")
2021-08-16 09:57:33 +01:00
Ed Maste 54247ce1f9 Makefile.inc1: Avoid hanging if pkg is not installed
For `pkg --version`, Redirect stdin from /dev/null to avoid waiting on
/usr/sbin/pkg's bootstrap prompt if the pkg package is not installed.
Also redirect stderr to /dev/null to discard the warning message in
this case.

Reported by:	mjg
Fixes:		4e224e4be7 ("pkgbase: accommodate pkg < 1.17")
Sponsored by:	The FreeBSD Foundation
2021-08-07 23:13:40 -04:00
Mitchell Horne 61ed578ee6 Prefer MK_SSP=no to SSP_CFLAGS=
It is more idiomatic. CFLAGS is only augmented with $SSP_CFLAGS when
$MK_SSP != "no".

Reviewed by:	imp
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D31401
2021-08-04 15:23:22 -03:00
Mitchell Horne bcd2f41f7d mk: format some option lists
Alphabetize and give each option its own line, ahead of making another
change to these lists. This makes future diffs easier to read.

Reviewed by:	imp, emaste
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D31399
2021-08-04 15:23:22 -03:00
Bryan Drewery b60770fceb Fix native-xtools build
Fixes		https://github.com/freebsd/poudriere/issues/894
Fixes:		d0c737e18 ("Makefile: Fix MAKEOBJDIRPREFIX command-line")
X-MFC-With:	d0c737e18
2021-08-03 10:00:28 -07:00
Bryan Drewery 2482ea42fb Revert "Fix native-xtools build"
This reverts commit 36269b8231.

This had an unintended change included.
2021-08-03 10:00:28 -07:00
Bryan Drewery 36269b8231 Fix native-xtools build
Fixes		https://github.com/freebsd/poudriere/issues/894
Fixes:		d0c737e18 ("Makefile: Fix MAKEOBJDIRPREFIX command-line")
X-MFC-With:	d0c737e18
2021-08-03 08:22:14 -07:00
Alex Richardson 2de949cf85 Remove mkcsmapper_static and mkesdb_static from build-tools
Build them as part of the bootstrap-tools phase instead.

Reviewed by:	emaste (no objections)
Differential Revision: https://reviews.freebsd.org/D28181
2021-08-02 14:33:25 +01:00
Alex Richardson 7bc797e3f3 Add build system support for ASAN+UBSAN instrumentation
This adds two new options WITH_ASAN/WITH_UBSAN that can be set to
enable instrumentation of all binaries with AddressSanitizer and/or
UndefinedBehaviourSanitizer. This current patch is almost sufficient
to get a complete buildworld with sanitizer instrumentation but in
order to actually build and boot a system it depends on a few more
follow-up commits.

Reviewed By:	brooks, kib, markj
Differential Revision: https://reviews.freebsd.org/D31043
2021-08-02 14:33:24 +01:00
Wolfram Schneider bbd16236e9 add the time(1) command to the list of install tools
Reported by:	dhw
Approved by:	dhw
Differential Revision:	https://reviews.freebsd.org/D31373
2021-08-01 13:25:00 +00:00
Wolfram Schneider 99feb137f5 `make buildworld' with time logging for each stage
PR:	 	257141
Reviewed by:	sjg,emaste
Approved by:	emaste
Differential Revision:	https://reviews.freebsd.org/D31154
2021-08-01 09:10:49 +00:00
Ed Maste 4dbac528db pkgbase: improve pkg --version parsing
In some cases `pkg --version` might produce unexpected or additional
output.  Use a regex /^[0-9.]+$/ to match only the line containing the
version number.

Reported by:	Michael Butler on freebsd-current@
Fixes:		4e224e4be7 ("pkgbase: accommodate pkg < 1.17")
Sponsored by:	The FreeBSD Foundation
2021-07-28 17:14:33 -04:00
Ed Maste 4e224e4be7 pkgbase: accommodate pkg < 1.17
6cafdee71d adapted the pkgbase build for 1.17, but broke Cirrus-CI's
use of PKG_FORMAT=tar (the quarterly package set still has pkg 1.16).
Because of this I disabled the pkgbase build and test in 2bfba2a04b.

Now, check `pkg --version` and use the old logic for < 1.17.

To be reverted once we no longer encounter pkg 1.16 in Cirrus-CI (i.e.,
via GCP cloud images) to avoid keeping this extra complexity around.

PR:		257422
Reviewed by:	manu
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D31324
2021-07-27 14:50:06 -04:00
Emmanuel Vadot 6cafdee71d pkgbase: Track pkg 1.17
With pkg 1.17 the extention is always .pkg, packing_format is also now deprecated.
2021-07-22 10:19:02 +02:00
Alex Richardson c5d1d88a35 Don't build lldb-tablegen if MK_LLDB=no
Split out from D31060.

Reviewed by:	emaste
2021-07-19 15:05:22 +01:00
John Baldwin 0333fad1b7 Remove svnlite.
Reviewed by:	bcr, imp, emaste
Differential Revision:	https://reviews.freebsd.org/D30737
2021-06-11 14:56:41 -07:00
John Baldwin e290182bcf Remove 'make update'.
In the CVS days this used be a wrapper around either CVS or CVSup and
used to support updating src, doc, and ports checkouts.  With the move
to subversion this only supported updating src and was itself a
wrapper around 'svn update'.  With Git, users are probably better off
using appropriate Git commands directly to update without needing an
explicit make target as a wrapper.

Reviewed by:	bcr, imp, emaste
Differential Revision:	https://reviews.freebsd.org/D30736
2021-06-11 14:56:28 -07:00
Mitchell Horne 689c7e7975 libpmc: always generate libpmc_events.c
The jevents build tool will create an empty table if it doesn't find any
events, so we can remove the extra $MACHINE_CPUARCH checks.

Reviewed by:	gnn, ray, emaste
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D30531
2021-05-31 17:39:05 -03:00
Marcin Wojtas c6081dea59 Add elfctl to toolchain.
Add elfctl program to toolchain to allow modifying ELF feature flags
during system build.

Submitted by: Dawid Gorecki <dgr@semihalf.com>
Obtained from: Semihalf
Sponsored by: Stormshield
Differential Revision: https://reviews.freebsd.org/D29552
2021-05-21 15:32:42 +02:00
Glen Barber 50179c5ec7 Makefile.inc1: unbreak bootstrap when kbdcontrol does not exist
Reviewed by:		arichardson
MFC after:		12 hours
MFC target:		stable/13, releng/13.0
Differential Review:	https://reviews.freebsd.org/D29200
Sponsored by:		Rubicon Communications, LLC ("Netgate")
2021-03-23 20:47:14 -04:00
Baptiste Daroussin 4a847a6ccd Revert "bootstrap: add tic to the bootstrap tools"
Since we don't build and install the terminfo db anymore this is not
needed

This reverts commit b6a51d39e3.
This reverts commit e5e4845959.
This reverts commit 0af562d7e1.
2021-03-18 16:05:20 +01:00
Alex Richardson e5e4845959 Makefile.inc1 "Avoid duplicate script for target" warning
This happens if -DBOOTSTRAP_ALL_TOOLS or when building on non-FreeBSD.
2021-03-01 13:22:15 +00:00
Baptiste Daroussin b6a51d39e3 bootstrap: fix a last edit typo
Reported by:	cy
2021-02-25 14:53:00 +01:00
Baptiste Daroussin 0af562d7e1 bootstrap: add tic to the bootstrap tools
tic is necessary early in the build to be able to build the terminfo
database later on.

Tested by:	manu, jbeich
2021-02-25 14:25:28 +01:00
Alex Richardson 5cf6f1c4bc Remove the MK_LIBCPLUSPLUS option
This option has been equivalent to any form of C++ support since libstdc++
was removed. Therefore, replace all MK_LIBCPLUSPLUS uses with MK_CXX.

Reviewed By:	emaste
Differential Revision: https://reviews.freebsd.org/D27974
2021-02-01 09:32:07 +00:00
Kyle Evans 123ae3045d build: remove LIBPTHREAD/LIBTHR build options
WITHOUT_LIBTHR has been broken for a little over five years now, since the
xz 5.2.0 update introduced a hard liblzma dependency on libthr, and building
a useful system without threading support is becoming increasingly more
difficult.

Additionally, in the five plus years that it's been broken more reverse
dependencies have cropped up in libzstd, libsqlite3, and libcrypto (among
others) that make it more and more difficult to reconcile the effort needed
to fix these options.

Remove the broken options.

PR:		252760
Reviewed by:	brooks, emaste, kib
Differential Revision:	https://reviews.freebsd.org/D28263
2021-01-22 12:33:27 -06:00
Glen Barber a53ce3fc49 Bump CURRENT to 14.0
This one goes to 14.

Approved by:	re (implicit)
Sponsored by:	Rubicon Communications, LLC ("Netgate")
2021-01-21 19:10:07 -05:00
Kyle Evans 26490d9b74 pkgbase: allow update-packages for first-run of packaging
If ${REPODIR}/${PKG_ABI} does not exist when we begin real-update-packages,
skip the comparison with the non-existent previous repository and just
finish the repo off. This allows external scripts to just assume they can
run `update-packages` rather than figuring out if they'd previously run
`packages` for this Version/Arch combo.

PKG_VERSION_FROM_DIR was added so that we could perhaps detect the three
distinct cases:

1.) If the repo has not yet been created, PKG_VERSION_FROM_DIR will be
  empty.
2.) If the repo is in some intermediate state between created and fully
  initialized, PKG_VERSION_FROM_DIR may point to the ABI directory.
3.) If the repo is fully initialized, then PKG_VERSION_FROM_DIR points to
  the latest build to compare to.

Option #2 is explicitly unhandled at the moment, but this is no different
than it was before.

Reviewed-by:	manu
Differential-Revision:	https://reviews.freebsd.org/D28229
2021-01-20 21:58:30 -06:00
Emmanuel Vadot 613fe5321f pkgbase: differentiate package versions for ALPHA/BETA/PRERELEASE/RC phases
The current postfix conversions are:

  CURRENT / STABLE / PRERELEASE, 12.x-CURRENT becomes 12.snapYYYYMMDDhhmmss
  ALPHAx -> .ax, so 11.3-ALPHA1 becomes 11.3.a1.YYYYMMDDhhmmss
  BETAx -> .bx, so 12.1-BETA2 becomes 12.1.b2.YYYYMMDDhhmmss
  RCx -> .rcx, so 13.0-RC3 becomes 13.0.rc3.YYYYMMDDhhmmss
  RELEASE -> (nothing), so 12.1-RELEASE becomes 12.1
  RELEASE-pX -> pX, so 12.1-RELEASE-p1 becomes 12.1p1

Note that for development branches we will start to drop the minor version
component entirely, which more closely matches how these branches are
physically named (stable/NN).

snap is a new prefix that was added to pkg in [0], which is simply a more
verbose version of the current ".s" used.

As noted, build timestamps are also added to ALPHA/BETA/RC versions.  This
is largely irrelevant for re@ snapshots because they will only produce one
set of snapshots for each alpha/beta/rc, but external folks may produce
multiple in that timeframe -- at least for alpha.  For them, it is
imperative that the builds have a differentiating characteristic like this
rather than multiple builds across multiple revisions being versioned
identically.

[0] https://github.com/freebsd/pkg/pull/1929

Reviewed by:	gjb, manu
Submitted by:	rene (original, original version)
Differential Revision:	https://reviews.freebsd.org/D28167
2021-01-18 22:26:22 -06:00
Kyle Evans 95ae95d413 pkgbase: limit PKG_VERSION_FROM calculation to real-update-packages
PKG_ABI is defined in some other targets that do not need to shell out and
calculate PKG_VERSION_FROM. Moreover, it produces extra errors when
bootstrapping an initial pkgbase repo, as the /latest link doesn't exist
yet.
2021-01-18 13:34:54 -06:00
Kyle Evans da8cc827ae pkgbase: collapse -PRERELEASE into STABLE/CURRENT case for version suffix
-PRERELEASE exists as a given stable/X branch while the next minor version
release is in progress. From a functional standpoint, it should be treated
as -STABLE since it'll typically be a superset of what's included in the
concurrent releng branch.

Reviewed by:	gjb
Differential Revision:	https://reviews.freebsd.org/D28166
2021-01-17 14:08:13 -06:00
Kyle Evans 9af9e7c5cc pkgbase: default to installing all specified kernels
These only get staged off to the side, not installed into the system. Let's
change the default for packages/update-packages to be less surprising.

MFC after:	1 week
Reviewed by:	manu, meena <me igalic co>
Differential Revision:	https://reviews.freebsd.org/D28190
2021-01-17 14:05:33 -06:00
Emmanuel Vadot 03cab14ea3 Revert "pkgbase: differentiate package versions for ALPHA/BETA/PRERELEASE/RC phases"
This reverts commit aef769614f.

While the package names are a bit prettier this confuse pkg about upgrading :
$ pkg version -t 13.0.s2021011313063 13.0.a1
>
$ pkg version -t 13.0.s2021011313063 13.0_ALPHA1
<

Note that the current scheme isn't good when bumping from ALPHA to BETA or
even BETA to RC:
$ pkg version -t 13.0_ALPHA1 13.0_BETA1
=
$ pkg version -t 13.0_BETA1 13.0_RC1
=

But more thoughts have to be put into this renaming.
2021-01-14 14:15:05 +01:00
Emmanuel Vadot aef769614f pkgbase: differentiate package versions for ALPHA/BETA/PRERELEASE/RC phases
The current postfix conversions are:

    CURRENT / STABLE -> .sYYYYMMDDhhmmss
    ALPHAx -> .ax, so 11.3-ALPHA1 becomes 11.3.a1
    BETAx -> .bx, so 12.1-BETA2 becomes 12.1.b2
    RCx -> .rcx, so 13.0-RC3 becomes 13.0.rc3
    PRERELEASE -> .p, so 11.3-PRERELEASE becomes 11.3.p
    RELEASE -> (nothing), so 12.1-RELEASE becomes 12.1

Submitted by: rene (original version)
Differential Revision: https://reviews.freebsd.org/D20734
2021-01-14 14:00:04 +01:00
Bryan Drewery 44b8b2a00d Makefile.inc1: Avoid using release/Makefile for VERSION.
release/Makefile.inc1 has git executions that were being ran for each of
these lookups.  The results were not needed so just lookup what we want
directly instead.

Reviewed by:	gjb, rlibby, emaste (maybe)
Sponsored by:	Dell EMC
Differential Revision:	https://reviews.freebsd.org/D27643
2021-01-08 14:33:35 -08:00
Alex Richardson 7fa2f2a62f Rename NO_WERROR -> MK_WERROR=no
As suggested in D27598. This also supports MK_WERROR.clang=no and
MK_WERROR.gcc=no to support the existing NO_WERROR.<compiler> uses.

Reviewed By:	brooks
Differential Revision: https://reviews.freebsd.org/D27601
2021-01-07 09:31:03 +00:00
Alex Richardson 94ffff68c8 Stop passing MK_WARNS=no for bootstrap tools
I got annoyed by the number of warnings that the CheriBSD build was
emitting. It turns out that we are emitting lots of warnings during
bootstrap because bootstrap tools are built with the default compiler
flags and ignore the warnings flags that are set in bsd.sys.mk.

Looking at git blame, it appears that MK_WARNS=no has been passed since
rS112869, replacing the -DNO_WERROR option that was added in rS87775.

This commit changes MK_WARNS=no back to -DNO_WERROR. We need to pass
-DNO_WERROR, since the system compiler might have new warnings that we
don't know about yet, and we shouldn't fail the build in that case.

Reviewed By:	imp, brooks
Differential Revision: https://reviews.freebsd.org/D27598
2021-01-07 09:31:03 +00:00
Alex Richardson 0cace0bb0a Add mkimg to the list of bootstrap tools
Having this as part of the bootstrap tools is useful to build disk images
to boot in QEMU (especially when building on a Linux/macOS host where
mkimg is not available). We have been bootstrapping mkimg in CheriBSD for
a long time (using LOCAL_XTOOL_DIRS) but I believe this is also useful
upstream.

Reviewed By:	emaste, brooks
Differential Revision: https://reviews.freebsd.org/D27602
2021-01-07 09:26:21 +00:00