This also adds arm and riscv to the GCC tinderbox.
Reviewed by: emaste (older version)
Differential Revision: https://reviews.freebsd.org/D26202
(cherry picked from commit 9dfbda254c)
Allow the build process to find host binaries during the host-symlinks target when
cross-building on non-FreeBSD systems. Whilst most non-FreeBSD systems have all
the needed tools in /sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin (the final
path added by host-symlinks itself), Homebrew for macOS on Arm defaults to
/opt/homebrew/bin, other more niche systems may also deviate and users may
expect tools in a customised PATH to be picked up, unlike on FreeBSD where we
want to ensure everything comes from base. In particular, (un)xz are needed
from Homebrew on macOS, and thus cannot be found on Arm without this.
Note that non-FreeBSD builds enforce BUILD_WITH_STRICT_TMPPATH, and so the
actual main build steps will still use a sanitised PATH.
Reviewed by: jrtc27, arichardson
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D37991
(cherry picked from commit 16fbf01912)
These targets generate a raw list of the candidate old files roughly
corresponding to the values of OLD_DIRS, OLD_FILES, and OLD_LIBS.
Currently list-old-files also includes uncompressed manpages in
addition to compressed manpages.
Use these targets in the implementation of check-old-* and
delete-old-* to replace duplicated logic.
Reviewed by: imp, emaste
Sponsored by: The University of Cambridge, Google Inc.
Differential Revision: https://reviews.freebsd.org/D33327
(cherry picked from commit 21baf42c57)
In the CVS days this used be a wrapper around either CVS or CVSup and
used to support updating src, doc, and ports checkouts. With the move
to subversion this only supported updating src and was itself a
wrapper around 'svn update'. With Git, users are probably better off
using appropriate Git commands directly to update without needing an
explicit make target as a wrapper.
Reviewed by: bcr, imp, emaste
Differential Revision: https://reviews.freebsd.org/D30736
(cherry picked from commit e290182bcf)
Unlike the old fmake, running make FOO=bar when using bmake doesn't put
FOO=bar in .MAKEFLAGS at the top level, it instead just puts FOO in
.MAKEOVERRIDES and the full MAKEFLAGS will be formed for sub-makes.
Moreover, this only applies for sub-makes in rules, so this doesn't
apply to those in shell assignments. This means that the current check
does not catch make MAKEOBJDIRPREFIX=..., only those defined in config
files. Thus we must also check .MAKEOVERRIDES explicitly.
Reviewed by: sjg
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D31015
(cherry picked from commit d0c737e184)
As suggested in D27598. This also supports MK_WERROR.clang=no and
MK_WERROR.gcc=no to support the existing NO_WERROR.<compiler> uses.
Reviewed By: brooks
Differential Revision: https://reviews.freebsd.org/D27601
The new version only includes a specific version once, and uses the one
that's currently advised by tinderbox: -gcc6.
It also advises just installing the pkg, but mentions in a side-note at the
end where to find the source in the ports tree.
Reviewed by: jrtc27
Suggested by: jhb (use default from tinderbox)
Differential Revision: https://reviews.freebsd.org/D26820
This adds a new target update-packages which will create the new packages
compared to the last run.
This is how to use it:
At this point we cut a release
$ make buildworld ...
$ make buildkernel
$ make packages
There is now a PKG_VERSION directory with latest link pointing to it
Distribute the packages to server
$ something something that update the source tree
$ make buildworld ...
$ make buildkernel
$ make update-packages
You know have a PKG_VERSION directory in the REPODIR and latest link pointing to it.
In PKG_VERSION dir only the packages which differs from the latest run are
named PKG_VERSION, otherwise the old packages are there.
The process is :
Build the new packages in the PKG_VERSION directory
Compare the internal data with the PKG_VERSION_FROM version. The comparison is done
by checking the internal hash of the packages.
By default PKG_VERSION_FROM is set to what the latest link points to.
If the old and new version matches, we rm the new package and cp the old one.
Differential Revision: https://reviews.freebsd.org/D25984
LINT config files are about to be checked in directly. Eliminate
building them by hand here from NOTES files.
Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D26540
This is the initial set up for PowerPC64LE.
The current plan is for this arch to remain experimental for FreeBSD 13.
This started as a weekend learning project for me and kinda snowballed from
there.
(More to follow momentarily.)
Reviewed by: imp (earlier version), emaste
Sponsored by: Tag1 Consulting, Inc.
Differential Revision: https://reviews.freebsd.org/D26399
${TARGET_ARCH} is empty here which results in empy MAKE_PARAMS being
passed to the buildkernel phase. This breaks the build when using the
strict TMPPATH since cc will not be included in $PATH.
Reviewed By: jhb
This uses GCC toolchains instead of LLVM on architectures supported by
GCC. Currently this uses GCC 6 on aarch64, amd64, i386, and mips.
Although this does also try to use GCC 6 on powerpc, it is always
skipped for now since a powerpc-gcc6 package is not available and the
structure of make universe makes it hard to skip a subset of arches
for a target. This should be short-lived as freebsd-gcc9 does include
a powerpc-gcc9 package so powerpc should work once this switches to
GCC 9.
Discussed with: emaste
Differential Revision: https://reviews.freebsd.org/D25732
This is a temporary hack to aid with config(8) changing in r360443.
It will not work for all cases.
env PATH is used because universe-toolchain is only built when worlds
are built, and then only if clang is needed, so it may not exist.
universe-toolchain needs to be expanded to always be built, inspected to
remove non-cross-build-safe tools, used for buildworld/buildkernel,
and potentially incremental build support.
Sponsored by: Dell EMC
Currently, powerpcspe is broken with clang. Add it to the EXTRA_TARGETS until
that's fixed.
Reviewed by: brooks, bdrewery, emaste (LGTM)
MFC After: 3 days
Differential Revision: https://reviews.freebsd.org/D24212
make kernels was originally documented (in commit r295099) as the same as make
universe -DJUST_BUILD_KERNELS. However, it used the UNIVERSE_TARGET=buildkernel
which is subtly different in that it also builds the toolchains and doesn't
build the LINT modules unless they happened to already exist in the tree. This
unbreaks POLA and just builds the kernels, including LINT, now rather than all
that other stuff as well. When the commit originally happened, the two just
differed by the LINT bug. r335711 introduced the universe dependency on the
toolchain that wasn't present before, which diverged the two further. This
restores the original intent of r295099.
Reviewed by: brooks, bdrewery, emaste (LGTM)
MFC After: 3 days
Differential Revision: https://reviews.freebsd.org/D24212
Add 'WITHOUT_WORLDS' and 'WITHOUT_KERNELS' as aliases for the inconsistently
named MAKE_JUST_KERNELS and MAKE_JUST_WORLDS respectively. I always forget the
MAKE_ part (or is it BUILD_), and it's inconsistent with everything
else. Document the new things, but leave speculation of any eventual MAKE_JUST_*
deprecation out of the manuals and comments.
Reviewed by: brooks, bdrewery, emaste (LGTM)
MFC After: 3 days
Differential Revision: https://reviews.freebsd.org/D24212
If EXTRA_TARGETS is defined, build all supported architecture
variants. By default, build architecture variants needed to provide
code coverage or that are commonly used.
Use this to disable building of all the hard-float and little-endian
MIPS architecture variants along with n32 by default.
Reviewed by: rpokala
Discussed with: imp, emaste
Differential Revision: https://reviews.freebsd.org/D24178
Now that we have updated the in-tree version of LLVM to 10.0, we have all the
necessary LLVM changes to use Clang+LLD as the default toolchain for MIPS.
Relnotes: yes
Reviewed By: emaste, jhb, brooks, kevans
Differential Revision: https://reviews.freebsd.org/D23204
The sparc64 architecture is being removed from FreeBSD 13, starting
now. This removes it from the top level only. It is the only
architecture that didn't see substantial work after the call to get
things working with the external toolchain.
- Enable clang and lld as system toolchains.
- Don't use external GCC for universe by default.
- Re-enable riscv64sf since it builds fine with clang + lld.
Reviewed by: emaste, mhorne
Relnotes: yes
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D23089
This fixes a regression in r356418 where the entire universe would
fail early due to an undefined make target when a given TARGET_ARCH
had no associated kernel configs. This is true for all of the
hard-float mips TARGET_ARCHes currently.
Pointy hat to: me
Reviewed by: emaste
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D23071
powerpcspe is disabled for now until clang/llvm issues with spe
have been fixed.
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D23031
Previously, all of the kernels for a given TARGET were built if that
target was enabled. This was implemented by having each kernel built
via a universe_kernconf_<KERNEL> target that was depended on by a
universe_kernconfs target. However, this meant that if one did a
build with a limited set of TARGET_ARCH values for a given TARGET,
kernels could be built for which we hadn't built a world or toolchain.
For example, 'make TARGETS=mips TARGET_ARCHES_mips=mips64' would build
mips32 kernels.
Fix this by adding an extra layer of indirection in the kernel make
targets. universe_kernconf_<KERNEL> is now a dependency of a new
universe_kernconfs_<TARGET_ARCH>. universe_kernconfs in turn depends
on a list of universe_kernconfs_<target_arch> values, but only the
values enabled in TARGET_ARCHES_<TARGET>.
Reviewed by: imp
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D23031
TARGET=arm now defaults to TARGET_ARCH=armv7
TARGET_ARCH=arm is no longer valid.
Bump __FreeBSD_version to 1300073
Tested with make universe. Any stale LINT-V5 config files remaining in the tree
will fail the universe build. However, LINT-V5 was removed in r355119.
This retirement has been planned since last summer. The armv5 port is fragile:
it works OK for some peeople, and fails badly for others. There's a number of
subtle bugs in busdma, pmap and other MD parts of thee system that present
themselves under load or in unusual circumstances (like fsck after a
crash). stable/8, branched 10 years ago, was the last reliable release. Since
the support burden is larger then the benefit, the consensus view is armv5
should be removed from the tree.
Discussed with: arm@ mailing list and arm developer community.
It's on the chopping block in two months, the CI tinderbox doesn't bother with
it anymore either, and buildworld fails today due to an issue linking clang.
It's not worth investigating and it just eats up CPU cycles running universe
builds.
The wrong toolchain was set in MAKE_PARAMS_powerpc, however, there are
some other issues preventing powerpc from building in tinderbox:
1) There is no powerpc-gcc and powerpc-xtoolchain-gcc port that
provides an external 32-bit powerpc GCC toolchain.
2) On other targets, the same toolchain can build all of the
architectures for a given target. MIPS achieves this by always
setting -mabi and -EB/-EL explicitly instead of relying on the
compiler's default architecture. PowerPC might be able to do the
same thing, but as of today, powerpc-gcc would be required for
powerpc and powerpcspe and powerpc64-gcc would be required for
powerpc64. Our existing logic for make universe does not permit
per-MACHINE_ARCH toolchains.
I tried hacking TARGETS_powerpc to only include powerpc64 when
powerpc64-gcc was present, and while that skipped the 32-bit worlds,
it tried to build all the kernels.
Reported by: jeff
Discussed with: imp
Simplify expressions as suggested by jhb. The extra indirection made
sense in earlier versions of this patch, but not the final one.
While here, apply suggestion from emaste for wording of universe.
Also wordsmith awkwardly worded comment about when we effectively
neuter the universe build for an architecture.
Once llvm 9.0 has been vetted for mips and powerpc, I'll take them out
of these lists.
Only compile clang supporting architectures of amd64, arm, arm64,
i386, and riscv as part of universe. Compile the other architectures
if MAKE_OBSOLETE_GCC is defined. In all cases, explicit lists of
architectures in TARGETS= on the command line override.
For mips, powerpc and sparc64, do the same thing we do for risvc when
MAKE_OBSOLETE_GCC isn't defined and short-circuit their universe build
with an echo saying to install the xtoolchain port or pkg.
PR: 241134
Discussed on: arch@ (https://lists.freebsd.org/pipermail/freebsd-arch/2019-August/019674.html)
Differential Revision: https://reviews.freebsd.org/D21942
It doesn't make sense to limit to -j12 anymore, build scalability
is better than it used to be. Fold the hint into the description
of the universe target.
Reviewed by: imp
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D20342
The path is incorrect for the linker in the error message. It should have been
/usr/bin/ld, not /usr/bin/cc .
Reviewed by: emaste
Approved by: emaste (mentor)
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D19852
The sysent target is useful when changing makesyscalls.sh, when
making paired changes to syscalls.master files, or in a future where
freebsd32 sysent entries are built from the default syscalls.master.
Reviewed by: bdrewery
Obtained from: CheriBSD
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D17899
Inheriting $PATH during the build phase can cause the build to fail when
compiling on a different system due to missing build tools or incompatible
versions somewhere in $PATH. This has cause build failures for us before
due to the jenkins slaves still running FreeBSD 10.
Listing the tools we depend on explicitly instead of just using whatever
happens to be in $PATH allows us to check that we don't accidentally add a
new build dependency.
All tools that do no need to be bootstrapped will now be symlinked to
${WORLDTMP}/legacy/bin and during the build phase $PATH will only contain
${WORLDTMP}. There is also a new variable "BOOTSTRAP_ALL_TOOLS" which can
be set to force compiling almost all bootstrap tools instead of symlinking
them. This will not bootstrap tools such as cp,mv, etc. since they may be
used during the build and for those we should really only be using POSIX
compatible options.
Furthermore, this change is required in order to be able to build on
non-FreeBSD hosts. While the same binaries may exist on Linux/MacOS they
often accept different flags or produce incompatible output.
Approved By: brooks (mentor)
Differential Revision: https://reviews.freebsd.org/D16815
riscv64sf has been broken due to duplicate symbols for months and
degrades the quality of universe builds. Remove it until this is
resolved leaving a comment to it is not re-added.
PR: 232085
Reviewed by: emaste
Approved by: re (gjb, kib)
Sponsored by: DARPA, AFRL
This fixes errors from the MK_CLANG_BOOTSTRAP/MK_LLD_BOOTSTRAP lookups
to not force using XCC/XLD but to rather just build them as normal by
allowing their own bootstrap logic to work.
MFC after: 3 weeks
X-MFC-with: r335711 r335769
Sponsored by: Dell EMC
This fixes a warning for each RISCV target during universe by passing in
the required CROSS_TOOLCHAIN setting which will in turn set
CROSS_BINUTILS_PREFIX correctly. It also ensures that a tinderbox build
uses the correct compiler for riscv. Previously it was using the shared
clang compiler instead of riscv64-gcc.
Reviewed by: bdrewery
Sponsored by: DARPA / AFRL
Differential Revision: https://reviews.freebsd.org/D16049
Need to handle LLD_BOOTSTRAP separately (for archs like i386).
This would be much better off with an off-by-default option like
SHARED_TOOLCHAIN that universe force-enabled. Then a normal buildworld
would store the toolchain there if enabled and otherwise in WORLDTMP
with only the 1 arch selected.
MFC after: 3 weeks
Sponsored by: Dell EMC
This is a bit noisy now but it was silent before leading to
wondering if it was doing anything.
MFC after: 1 week
Suggested by: rpokala
Sponsored by: Dell EMC
This works similar to WITH_SYSTEM_COMPILER added in r300354. It only
supports lld via WITH_LLD_BOOTSTRAP.
When both SYSTEM_COMPILER and SYSTEM_LINKER logic passes then libclang
will not build in cross-tools. If either check fails though then
libclang is built.
The .info is reworked to notify when libclang will be built since if
either clang or lld needs to be rebuilt, but not the other, the
notification can lead to confusion on why "clang is building".
-fuse-ld= is not used with this method so some combinations of compiler
and linker are expected to fail.
A new 'make test-system-linker' target is added to see the logic results.
Makefile.inc1:
CROSS_BINUTILS_PREFIX support had to be moved higher up so that XLD
could be set and MK_LLD_BOOTSTRAP disabled before checking SYSTEM_LINKER
logic as done with SYSTEM_COMPILER. This also required moving where
bsd.linker.mk was read since XLD needs to be set before parsing it. This
creates a situation where src.opts.mk can not test LINKER_FEATURES or
add LLD_BOOTSTAP to BROKEN_OPTIONS.
Reviewed by: emaste (earlier version)
Sponsored by: Dell EMC
Differential Revision: https://reviews.freebsd.org/D15894
