The migration to LLVM's lld linker has been in progress for quite some
time - I opened an LLVM tracking bug (23214) in April 2015 to track
issues using lld as FreeBSD's linker, and requested the first exp-run
using lld as /usr/bin/ld in November 2016.
In 12.0 LLD is the system linker on amd64, arm64, and armv7. i386 was
not switched initially as there were additional ports failures not found
on amd64. Those have largely been addressed now, although there are a
small number of issues that are still being worked on. In some of these
cases having lld as the system linker makes it easier for developers and
third parties to investigate failures.
Thanks to antoine@ for handling the exp-runs and to everyone in the
FreeBSD and LLVM communites who have fixed issues with lld to get us to
this point.
PR: 214864
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
to shrink libllvm.a.
This is a workaround for "relocation truncated to fit" errors with BFD
ld 2.17.50 on arm and armv6, when linking executables against it.
The required range extensions are not yet supported by this very old
version of BFD ld. When arm and armv6 userland can be successfully
linked by lld, this workaround can be removed.
It has passed an exp run on amd64 and i386, and has testing on arm64. On
other architectures it is expected to run, however it can be disabled by
building world with -DWITHOUT_BSD_CRTBEGIN.
Sponsored by: DARPA, AFRL
The BSD crtbegin/crtend code now builds on all architectures, however
further work is needed to check if it works correctly.
MFC with: r339738
Sponsored by: DARPA, AFRL
You should not be using DES. You should not have been using DES for the
past 30 years.
The ed DES-CBC scheme lacked several desirable properties of a sealed
document system, even ignoring DES itself. In particular, it did not
provide the "integrity" cryptographic property (detection of tampering), and
it treated ASCII passwords as 64-bit keys (instead of using a KDF like
scrypt or PBKDF2).
Some general approaches ed(1) users might consider to replace the removed
DES mode:
1. Full disk encryption with something like AES-XTS. This is easy to
conceptualize, design, and implement, and it provides confidentiality for
data at rest. Like CBC, it lacks tampering protection. Examples include
GELI, LUKS, FileVault2.
2. Encrypted overlay ("stackable") filesystems (EncFS, PEFS?, CryptoFS,
others).
3. Native encryption at the filesystem layer. Ext4/F2FS, ZFS, APFS, and
NTFS all have some flavor of this.
4. Storing your files unencrypted. It's not like DES was doing you much
good.
If you have DES-CBC scrambled files produced by ed(1) prior to this change,
you may decrypt them with:
openssl des-cbc -d -iv 0 -K <key in hex> -in <inputfile> -out <plaintext>
Reviewed by: allanjude, bapt, emaste
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D17829
In the last decade(s) we have seen both short term or long term projects
committed to the tree which were considered or even marked "experimental".
While out-of-tree development has become easier than it used to be in
CVS times, there still is a need to have the code shipping with HEAD but
not enabled by default.
While people may think about VIMAGE as one of the recent larger, long term
projects, early protocol implementations (before they are standardised)
are others. (Free)BSD historically was one of the operating systems
which would have running code at early stages and help develop and
influence standardisation and the industry.
Give developers an opportunity to be more pro-active for early adoption
or running large scale code changes stumbling over each others but not
the user's feet. I have not added the option to NOTES in order to avoid
breaking supported option builds, which require constant compile testing.
Discussed with: people in the corridor
These are needed for .ctors/.dtors and .jcr handling. The former needs
all the function pointers to be called in the correct order from the
.init/.fini section. The latter just needs to call a gcj specific function
if it exists with a pointer to the start of the .jcr section.
This is currently disabled until __dso_handle support is added.
Reviewed by: emaste
MFC after: 1 month
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D17587
__DEFAULT_YES/NO. These options simply cannot work where we were using
__DEFAULT_NO. The proper thing to do in that case is to use the
BROKEN_OPTIONS knob instead.
well as on SPARC64 and can cause boot failures even when no encrypted
disks are present. Presumably, the reasons, while unknown, are the same
and most-likely are the result of some endian-unsafe code. Pending
finding the actual problem, extend the blacklist entry for these parts
of loader on SPARC to also cover all PowerPC platforms.
Approved by: re (kib)
Without this we get spurious output during boot as we try to run
nonexistant HyperV scripts on non-x86 models.
Reviewed by: kib
Approved by: re (gjb)
Differential Revision: https://reviews.freebsd.org/D17211
We want to build the 12.0 release artifacts with reproducible builds
mode enabled. Switch it on in HEAD now to enable testing with upcoming
ALPHA builds. We can revisit the default setting for HEAD after the
branch is created.
This change eliminates the build metadata (user, hostname, timestamp,
etc.) from the kernel and loader. If the src tree is a git, svn or p4
checkout with changes then the metadata is retained.
The WITHOUT_REPRODUCIBLE_BUILD src.conf(5) knob can be used to revert
to the previous behaviour.
Approved by: re (gjb)
Sponsored by: The FreeBSD Foundation
are fully debugged. With these options off, the unified "loader"
binary for sparc64 works to boot a kernel from ZFS.
Submitted by: kevans
Reviewed by: imp kevans
lld should now be a usable linker for armv7, and is already used as the
bootstrap linker (for linking the kernel and userland). Also enable as
the system linker now (/usr/bin/ld) for further testing and evaluation.
(This change will be reverted in case of unexpected fallout.)
Approved by: manu
Sponsored by: The FreeBSD Foundation
Create loader_{4th,lua,simp}{,.efi}. All of these are installed by
default. Create LOADER_DEFAULT_INTERP to specify the default
interpreter when no other is specified. LOADER_INTERP is the current
interpreter language building. Turn building of lua on by default to
match 4th. simploader is a simplified loader build w/o any interpreter
language (but with a simple loader). This is the historic behavir you
got with WITHOUT_FORTH. Make a hard link to the default loader. This
has to be a hard link rather than the more desirable soft link because
older zfsboot blocks don't support symlinks.
RelNotes: Yes
Differential Revision: https://reviews.freebsd.org/D16705
BPF (eBPF) is an independent instruction set architecture which is
introduced in Linux a few years ago. Originally, eBPF execute
environment was only inside Linux kernel. However, recent years there
are some user space implementation (https://github.com/iovisor/ubpf,
https://doc.dpdk.org/guides/prog_guide/bpf_lib.html) and kernel space
implementation for FreeBSD is going on
(https://github.com/YutaroHayakawa/generic-ebpf).
The BPF target support can be enabled using WITH_LLVM_TARGET_BPF, as it
is not built by default.
Submitted by: Yutaro Hayakawa <yhayakawa3720@gmail.com>
Reviewed by: dim, bdrewery
Differential Revision: https://reviews.freebsd.org/D16033
As of r336972 lld is capable linking the armv7 kernel and userland,
so enable it by default.
PR: 229050
Reviewed by: kevans
Tested by: kevans
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D16528
By default ld.lld should be the bootstrap linker (only) on i386 right
now. Once the i386 exp-run with LLD_IS_LD has a good result this will
also be enabled by default.
Reported by: andrew
Pointy hat to: emaste
Akin to r327783 for amd64. lld has been usable for amd64 for quite some
time, but a couple of issues remained that affected i386. These were
recently addressed upstream in lld and merged into FreeBSD or addressed
directly in FreeBSD (r326831, r326879, r326897, r326957, r333401,
r334626, r336664).
Similarly to the intial amd64 commit this change enables lld only as the
bootstrap linker (used to link the kernel and userland libraries and
executables), while GNU ld.bfd is still installed as /usr/bin/ld and
used for ports builds. That will be changed shortly, after an exp-run.
This is a recommit of r327823 after additional lld fixes.
PR: 225128 (exp-run)
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
This moves the bulk of the geli support from lib386/biosdisk.c into a new
geli/gelidev.c which implements a devsw-type device whose dv_strategy()
function handles geli decryption. Support for all arches comes from moving
the taste-and-attach code to the devopen() function in libsa.
After opening any DEVT_DISK device, devopen() calls the new function
geli_probe_and_attach(), which will "attach" the geli code to the open_file
struct by creating a geli_devdesc instance to replace the disk_devdesc
instance in the open_file. That routes all IO for the device through the
geli code.
A new public geli_add_key() function is added, to allow arch/vendor-specific
code to add keys obtained from custom hardware or other sources.
With these changes, geli support will be compiled into all variations of
loader(8) on all arches because the default is WITH_LOADER_GELI.
Relnotes: yes
Sponsored by: Microchip Technology Inc
Differential Revision: https://reviews.freebsd.org/D15743
Add src.conf knob to disable the installation of /var/db/services.db
Default to leaving services.db in place, but allow the removal of the
file and its creation with a src.conf knob.
This file ends up being 2MB in size. For small systems this is a waste
of space but its a tradeoff.
Reviewed by: bdrewery
Differential Revision: https://reviews.freebsd.org/D9655
This is needed for -m32 support which is used in the kernel cloudabi32 module.
Tweak the style to make it easier to understand.
MFC after: 2 weeks
X-MFC-with: r335706
Reported by: Mark Millard
Sponsored by: Dell EMC
LLVM_TARGET_* will auto be set based on LLVM_TARGET_ALL and MK_CLANG.
If LLVM_TARGET_ALL is disabled, during a cross-build, then SYSTEM_COMPILER
and SYSTEM_LINKER are auto disabled.
This option should be used by users rather than the per-arch LLVM_TARGET
options as it is simpler to maintain for them should the supported
target list change.
MFC after: 2 weeks
Reviewed by: sbruno, dim
Sponsored by: Dell EMC
Differential Revision: https://reviews.freebsd.org/D16020
This makes it possible, through src.conf(5) settings, to select which
LLVM targets you want to build during buildworld. The current list is:
* (WITH|WITHOUT)_LLVM_TARGET_AARCH64
* (WITH|WITHOUT)_LLVM_TARGET_ARM
* (WITH|WITHOUT)_LLVM_TARGET_MIPS
* (WITH|WITHOUT)_LLVM_TARGET_POWERPC
* (WITH|WITHOUT)_LLVM_TARGET_SPARC
* (WITH|WITHOUT)_LLVM_TARGET_X86
To not influence anything right now, all of these are on by default, in
situations where clang is enabled.
Selectively turning a few targets off manually should work. Turning on
only one target should work too, even if that target does not correspond
to the build architecture. (In that case, LLVM_NATIVE_ARCH will not be
defined, and you can only use the resulting clang executable for
cross-compiling.)
I performed a few measurements on one of the FreeBSD.org reference
machines, building clang from scratch, with all targets enabled, and
with only the x86 target enabled. The latter was ~12% faster in real
time (on a 32-core box), and ~14% faster in user time. For a full
buildworld the difference will probably be less pronounced, though.
Reviewed by: bdrewery
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D11077
This works similar to WITH_SYSTEM_COMPILER added in r300354. It only
supports lld via WITH_LLD_BOOTSTRAP.
When both SYSTEM_COMPILER and SYSTEM_LINKER logic passes then libclang
will not build in cross-tools. If either check fails though then
libclang is built.
The .info is reworked to notify when libclang will be built since if
either clang or lld needs to be rebuilt, but not the other, the
notification can lead to confusion on why "clang is building".
-fuse-ld= is not used with this method so some combinations of compiler
and linker are expected to fail.
A new 'make test-system-linker' target is added to see the logic results.
Makefile.inc1:
CROSS_BINUTILS_PREFIX support had to be moved higher up so that XLD
could be set and MK_LLD_BOOTSTRAP disabled before checking SYSTEM_LINKER
logic as done with SYSTEM_COMPILER. This also required moving where
bsd.linker.mk was read since XLD needs to be set before parsing it. This
creates a situation where src.opts.mk can not test LINKER_FEATURES or
add LLD_BOOTSTAP to BROKEN_OPTIONS.
Reviewed by: emaste (earlier version)
Sponsored by: Dell EMC
Differential Revision: https://reviews.freebsd.org/D15894
Implement MK_NVME now that the expression for where NVMe is
complicated. Default it to "yes" for x86 and powerpc64 and
no everywhere else. Use it in camcontrol to define WITH_NVME
for those platforms where we support nvme.
This should fix the newly introduced nvme files to camcontrol
which were building everywhere.
Pointy Hat To: imp
Sponsored by: Netflix
The migration to LLVM's lld linker has been in progress for quite some
time - about three years ago I opened an upstream LLVM meta-bug to track
issues using lld as FreeBSD's linker, and about 1.5 years ago requested
the first exp-run with lld as the system linker.
As of r327783 we enabled LLD_BOOTSTRAP by default on amd64, using lld as
the linker to link the kernel and world, but GNU ld was still installed
as /usr/bin/ld.
The vast majority of issues observed when building ports with lld as the
system linker have now been solved, so set LLD_IS_LD by default on amd64
and install lld as /usr/bin/ld. A small number of port failures remain
and these will be addressed in the near future.
Thanks to antoine@ for handling the exp-runs, krion@ for investigating
many port failures and adding LLD_UNSAFE or other fixes or workarounds,
and everyone who helped investigate, fix or tag ports.
PR: 214864 (exp-run)
Sponsored by: The FreeBSD Foundation
freebsd-update depends on phttpget from portsnap. We could move phttpget
out of portsnap and build it as long as WITHOUT_FREEBSD_UPDATE and
WITHOUT_PORTSNAP are not both set, but for now just make the dependency
explicit.
PR: 228220
Reported by: Dries Michiels
Sponsored by: The FreeBSD Foundation
It was an old TRE that had plenty of bugs and no performance gain over
regex(3). I disabled it by default in r323615, and there was some confusion
about what the knob does- likely due to poor naming on my part- to the tune
of "well, it sounds like it should speed things up" (mentioned by multiple
people).
To compound this, I have no intention of maintaining a second regex
implementation. If someone would like to step up and volunteer to maintain a
lean-and-mean implementation for grep, this is OK, but we have very few
volunteers to maintain even our primary regex implementation.
As with Clang, build our toolchain components by default when the host
compiler is capable of doing so, to make them available for testing and
experimentation.
Sponsored by: The FreeBSD Foundation
These have become unsorted from everything else. This is desync'd from
stable/11 due to some hand-merging that was done there, so the MFC of this
will look slightly different.
MFC after: 3 days
This is originally based on a patch from David Chisnall for soft-float
N64 but has since been updated to support O32, N32, and hard-float ABIs.
The soft-float O32, N32, and N64 support has been committed upstream.
The hard-float changes are still in review upstream.
Enable LLVM_LIBUNWIND on mips when building with a suitable (C+11-capable)
toolchain. This has been tested with external GCC for all ABIs and
O32 and N64 with clang.
Reviewed by: emaste
Obtained from: CheriBSD (original N64 patch)
Sponsored by: DARPA / AFRL
Differential Revision: https://reviews.freebsd.org/D14701
ConnectX-4/5 devices in mlx5core.
The dump is obtained by reading a predefined register map from the
non-destructive crspace, accessible by the vendor-specific PCIe
capability (VSC). The dump is stored in preallocated kernel memory and
managed by the mlx5tool(8), which communicates with the driver using a
character device node.
The utility allows to store the dump in format
<address> <value>
into a file, to reset the dump content, and to manually initiate the
dump.
A call to mlx5_fwdump() should be added at the places where a dump
must be fetched automatically. The most likely place is right before a
firmware reset request.
Submitted by: kib@
MFC after: 1 week
Sponsored by: Mellanox Technologies
LLD is a cross toolchain component. It shouldn't be built when
requesting a build without building a cross compiler.
(CROSS_COMPILER is somewhat unfortunately named; in any case, lld
should be treated as GNU binutils here.)
Submitted by: Dan McGregor <dan.mcgregor at usask.ca>
MFC after: 1 week
Makefile.${MACHINE_ARCH} and remove the now-empty files. Generate the
*32 directories on the necessary architectures (well, currently only
amd64) on the fly. Remove LOADER_EFI variable and co-locate it with
EFI.
Differential Review: https://reviews.freebsd.org/D14546
