converting user allocation address into overhead pointer
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D41150
The ovu_magic is not neccessary overlaps with low byte of the ov_next,
for the big endian machines.
There is no range checking in the allocator.
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D41150
With ifunc based SIMD dispatch, we cannot use the amd64 assembly
implementations of string functions. Modify rtld to instead use
the generic functions. To avoid an architecture-specific special
case, this change is applied to all architectures.
This change is a prerequisite to and formerly part of D40693.
Sponsored by: FreeBSD Foundation
Approved by: kib
See also: D40693
Differential Revision: https://reviews.freebsd.org/D41050
For instance, dso might be mapped as needed but not yet initialized from
the other subtree of needed objects, while current object' constructor
does dlopen() for the dso. Right now rtld does relocations and other
processing based on the arrival of new objects in the global list, which
is not happens there. Directly check for the initialization state of
the object, for which we would return the handle.
One practical use case of this support is e.g. dlopen("libthr.so",
RTLD_NOLOAD) by libraries that are threading-aware but happy to live
with libc pthread shims if the program is not multithreaded.
Reviewed by: tijl
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Add a stop indicator to rtld_start to satisfy unwinders:
The right unwinding stop indicator should be CFI-undefined PC.
https://dwarfstd.org/doc/Dwarf3.pdf - page 118:
If a Return Address register is defined in the virtual unwind table,
and its rule is undefined (for example, by DW_CFA_undefined), then
there is no return address and no call address, and the virtual
unwind of stack activations is complete.
That is allows gdb and libunwind successfully stop when unwinding stack
from global constructors and destructors.
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D40949
Initial stack pointer is preserved in calle-saved %esi,
use it bellow to pass initial stack pointer to _rtld().
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D40950
These are no longer referenced, with the one user of each now using the
double-underscore version with "32" as an argument instead.
Reviewed by: kib, brooks, jhb
Differential Revision: https://reviews.freebsd.org/D40935
We still have a tiny amount of libcompat-specific code in rtld_paths.h,
but it's been deduplicated as much as possible, and in future we may
wish to just push these variables down to the few consumers of them and
make them use the double-underscore variants with a libcompat argument
rather than give them names here.
See commit 8fad2cda93 ("bsd.compat.mk: Provide new CPP and sub-make
variables") for the context behind this change.
Reviewed by: kib, brooks, jhb
Differential Revision: https://reviews.freebsd.org/D40925
1. Add a stop indicator to rtld_start to satisfy unwinders on aarch64:
The right unwinding stop indicator should be CFI-undefined PC.
https://dwarfstd.org/doc/Dwarf3.pdf - page 118:
If a Return Address register is defined in the virtual unwind table,
and its rule is undefined (for example, by DW_CFA_undefined), then
there is no return address and no call address, and the virtual
unwind of stack activations is complete.
2. Add a proper annotations for CFA.
That is allows gdb and libunwind successfully stop when unwinding stack
from global constructors and destructors.
Reviewed by:
Differential Revision: https://reviews.freebsd.org/D40844
Use the LDP (load pair registers) instruction to load entry point
arguments from the stack.
Reviewed by:
Differential Revision: https://reviews.freebsd.org/D40843
The right unwinding stop indicator should be CFI-undefined PC.
https://dwarfstd.org/doc/Dwarf3.pdf - page 118:
If a Return Address register is defined in the virtual unwind table,
and its rule is undefined (for example, by DW_CFA_undefined), then
there is no return address and no call address, and the virtual
unwind of stack activations is complete.
That is allows gdb and libunwind successfully stop when unwinding stack
from global constructors and destructors.
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D40794
Some architectures (powerpc and RISC-V) always use 0 for the post TLS
size in which case tls_init_align isn't used by
calculate_tls_post_size. Use __unused to quiet the warning for these
platforms.
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D40673
If we are allocating static offset for an object with dynamic index,
return failure. In the opposite case, if dynamic index is requested for
statically allocated TLS area, directly use the offset instead of
setting the index.
Taken from NetBSD Joerg Sonnenberger change for src/libexec/ld.elf_so/tls.c
rev. 1.18.
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
The meaning of the flag is that static TLS allocation was done.
Taken from NetBSD Joerg Sonnenberger change for src/libexec/ld.elf_so/tls.c
rev. 1.18.
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
The SPDX folks have obsoleted the BSD-2-Clause-NetBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.
Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix
The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.
Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix
rtld's Makefile used to add -L${LIBDIR} to LDFLAGS when MK_TOOLCHAIN was
no. This was done as part of a change to fix building rtld with
MK_TOOLCHAIN == no (although I'm not sure this part was necessary).
In any case as of 5f2e84015d libc_pic.a is built independent of the
MK_TOOLCHAIN setting and the main part of the workaround has already
been removed. Remove the rest now.
This reverts commit c0f5aeb032.
Reviewed by: jrtc27
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39938
Do not terminate scanning group when 'd' is encountered.
The 'b' option must be last in the group, same as 'f'.
Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D39544
Quoting from https://maskray.me/blog/2023-04-12-elf-hash-function:
The System V Application Binary Interface (generic ABI) specifies the
ELF object file format. When producing an output executable or shared
object needing a dynamic symbol table (.dynsym), a linker generates a
.hash section with type SHT_HASH to hold a symbol hash table. A DT_HASH
tag is produced to hold the address of .hash.
The function is supposed to return a value no larger than 0x0fffffff.
Unfortunately, there is a bug. When unsigned long consists of more than
32 bits, the return value may be larger than UINT32_MAX. For instance,
elf_hash((const unsigned char *)"\xff\x0f\x0f\x0f\x0f\x0f\x12") returns
0x100000002, which is clearly unintended, as the function should behave
the same way regardless of whether long represents a 32-bit integer or
a 64-bit integer.
Reviewed by: kib, Fangrui Song
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39517
These libraries are linked to directly by applications rather than
opened at runtime via dlopen().
Discussed with: oshogbo
Reviewed by: markj, emaste
Differential Revision: https://reviews.freebsd.org/D39245
Use it instead of defining IN_RTLD by base sources that want P_OSREL_
defines in userspace, but are not rtld.
This allows to remove abuse of IN_RTLD from userspace.
Reviewed by: dchagin, markj, imp
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D38585
After the removal of ia64 and sparc64, all current architectures
support executable stacks at an architectural level.
This reverts commit 1290d38ac5.
Reviewed by: kib
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D37904
similar to the same token in glibc.
Requested and reviewed by: bapt
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D37329
for the default output. For '-a' (per-object needed printout) the
[preloaded] banner is kept.
Instead, use special format2 for printing the preloaded objects (and
vdso), which does not include DT_NEEDED, since there is no object
needing the printed one.
In this way, the output is more compatible with glibc.
Example:
LD_PRELOAD=/lib/libthr.so.3 LD_TRACE_LOADED_OBJECTS=1 /libexec/ld-elf.so.1 /bin/ls
libutil.so.9 => /lib/libutil.so.9 (0x801099000)
libncursesw.so.9 => /lib/libncursesw.so.9 (0x8010b0000)
libc.so.7 => /lib/libc.so.7 (0x801123000)
[vdso] (0x7ffffffff000)
/lib/libthr.so.3 (0x80106c000)
Note the absense of the part before and including '=>' for preloaded
libthr.so.3, and for vdso.
PR: 265750
Reviewed by: jhb
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D36616
Rtld locks from libthr defer signals delivery, which means that binding
is not possible while a signal handler is executed.
Binding might upgrade read-locked rtld_bind_lock to write-lock, if
symbol resolution requires loading filters. If a signal would be delivered
while rtld is in read-locked section, and signal handler needs binding
which upgrades the lock, for non-threaded image that uses default rtld
locks, we get the rtld data structures modified under the top-level
active rtld frame.
To correct the problem, mask signals for read-locking of default locks
in addition to the write-locking. It is very cheap now with
sigfastblock(2).
Note that the global state is used to track pre-locked state of either
sigfastblock(2) or signal mask (if sigfastblock(2) is administratively
disabled). It is fine for non-threaded images since there are no other
threads. But I believe that it is fine for threaded images using libc_r
as well, since masking signals disables preemption (I did not tested
it).
NetBSD PR: https://gnats.netbsd.org/56979
Reported by: tmunro
Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D36396
Commit 24d0c9c1f5 introduced the following regression:
% ldd /bin/ls
/bin/ls:
libutil.so.9 => /lib/libutil.so.9 (0x1021000)
libncursesw.so.9 => /lib/libncursesw.so.9 (0x1021000)
libc.so.7 => /lib/libc.so.7 (0x1021000)
Note that the base address is the same for all displayed libraries.
Fix it by passing correct object to trace_print_obj().
Fixes: 24d0c9c1f5
Reviewed by: jrtc27
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
MFC after: 3 days
Differential revision: https://reviews.freebsd.org/D36198
55abf23dd3 inverted the value passed to origin_subst_one when rolling
up the existing code into a loop. If the first token is found ($ORIGIN),
this results in a wild free of part of strtab. Processing the second
token works fine and will act how the first should have regardless of
whether found, allocating memory for the string without freeing.
Processing subsequent tokens however will then leak, regardless of
whether found, as they will also believe they need to allocate memory
and can't free the string.
Found by: CHERI
Reviewed by: kib, markj
Fixes: 55abf23dd3 ("rtld: make token substitution table-driven")
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D35792
To allow for a dynamic page size on arm64 have the runtime linker
query the kernel for the currentl page size.
Reviewed by: kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34765
Before, preloaded objects, if not listed as needed for any normally
linked objects, were silently ignored.
Preloaded objects are printed with the `[preloaded]` herald. The list
includes the objects not listed explicitly as recursive dependencies of
the main object, effectively dsos loaded by LD_PRELOAD mechanism.
vdso is listed as well, since it is not needed by anything.
Since there is no DT_NEEDED entry for LD_PRELOADed objects, they are
usually printed using LD_TRACE_LOADED_OBJECTS_FTM2 format due to the
failure of the heuristic based on the presence of the 'lib' prefix.
Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D34716
it seems that glibc supports them, and such spelling is mentioned in the
ld.bfd manual. Idea seems to auto-correct some quoting/makefile sytnax
errors on linker command line.
Reviewed by: emaste, markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D34247
- Include <machine/tls.h> in MD rtld_machdep.h headers.
- Remove local definitions of TLS_* constants from rtld_machdep.h
headers and libc using the values from <machine/tls.h> instead.
- Use _tcb_set() instead of inlined versions in MD
allocate_initial_tls() routines in rtld. The one exception is amd64
whose _tcb_set() invokes the amd64_set_fsbase ifunc. rtld cannot
use ifuncs, so amd64 inlines the logic to optionally write to fsbase
directly.
- Use _tcb_set() instead of _set_tp() in libc.
- Use '&_tcb_get()->tcb_dtv' instead of _get_tp() in both rtld and libc.
This permits removing _get_tp.c from rtld.
- Use TLS_TCB_SIZE and TLS_TCB_ALIGN with allocate_tls() in MD
allocate_initial_tls() routines in rtld.
Reviewed by: kib, jrtc27 (earlier version)
Differential Revision: https://reviews.freebsd.org/D33353
This matches the TLS_TCB_ALIGN definition in libc.
Reviewed by: kib, jrtc27
Sponsored by: The University of Cambridge, Google Inc.
Differential Revision: https://reviews.freebsd.org/D33349
Previously TLS_DTV_OFFSET was added to the offset passed to
tls_get_addr_common; however, this approach matches powerpc and RISC-V
and better matches the intention.
Reviewed by: kib, jrtc27
Sponsored by: The University of Cambridge, Google Inc.
Differential Revision: https://reviews.freebsd.org/D33347
This is the more standard name for the bias of dtv pointers used on
other platforms. This also fixes a few other places that were using
the wrong bias previously on MIPS such as dlpi_tls_data in struct
dl_phdr_info and the recently added __libc_tls_get_addr().
Reviewed by: kib, jrtc27
Sponsored by: The University of Cambridge, Google Inc.
Differential Revision: https://reviews.freebsd.org/D33346
Consistently use `return ()'.
Fix some spacing issues with types formatting, and around binary ops.
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
File descriptor we operate on might reference something that is not a
file, e.g. shmfd. In this case, we cannot check MNT_NOEXEC in
principle.
If fstatfs(2) caused some failure on normal filesystem, then typical
expectation is that read or mmap of this file would also fail. If not,
mmap(2) PROT_EXEC on MNT_NOEXEC filesystem returns EACCES.
Reported by: obiwac@gmail.com
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Default value for ld_dynamic_weak is true, non-default settings should
be activated for the false value.
Reported by: obiwac@gmail.com
Sponsored by: The FreeBSD Foundation
MFC after: 3 days
but still for tightly coupled things like ldd(1)
Rename paths.h to rtld_paths.h.
Add guard for rtld-specific externs declarations.
Add _COMPAT32_BASENAME_RTLD and _COMPAT32_PATH_RTLD.
Reviewed by: arichardson, jhb
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D32464
Reset both pagepool_start and pagepool_end after a mmap(2) failure,
to avoid using invalid pagepool either for allocation or munmap(2).
PR: 259076
Noted by: Denis Koreshkov <dynamic-wind@mail.ru>
Reviewed by: arichardson
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D32514
PIE and shared objects usually have many relative relocations. In
2017/2018, a compact relative relocation format RELR was proposed on
https://groups.google.com/g/generic-abi/c/bX460iggiKg/m/GxjM0L-PBAAJ
("Proposal for a new section type SHT_RELR") and is a pre-standard.
RELR usually takes 3% or smaller space than R_*_RELATIVE relocations.
The virtual memory size of a mostly statically linked PIE is typically
5~10% smaller.
ld.lld --pack-dyn-relocs=relr emits RELR relocations. DT_RELR has been
adopted by Android bionic, Linux kernel's arm64 port, Chrome OS (patched
glibc).
This patch adds DT_RELR support to FreeBSD rtld-elf.
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D32524
This has the effect of installing the same file twice at the same location
and confuse pkgbase as we add this file twice in the package config part.
MFC after: 1 week
Sponsored by: Beckhoff Automation GmbH & Co. KG
to ignore lack of execute permission on the binary. The check is a
bad security theatre anyway.
Reviewed by: arichardson, imp
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D32464
In the situation with limited address space, together with
fragmentation, it is possible for mmap() request in morecore() to fail
when asking for required size + NPOOLPAGES, but succeed without the
addend. Retry allocation there.
PR: 259076
Reported by: Denis Koreshkov <dynamic-wind@mail.ru>
Reviewed by: arichardson
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D32474
Scan through the set of environment variables during initialization and
store values in the corresponding ld_env_var_desc structure, in the
single pass at init time. This does not eliminate use of getenv(3) and
unsetenv(3) completely, but provides a foundation to do that as the next
step.
Also organize the scan in a way that makes it easier to support aliases
like LD_DEBUG vs. LD_64_DEBUG.
Suggested by: arichardson
Reviewed by: arichardson, markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D31545
Instead of specifying the main name part of the environment variable as the
string literal, create array of the var names and access them by symbolic
index. Convert main name parts into complete names by prefixing with
ABI-specific ld_env_vars.
This way the name is not repeated, and also it can carry additional
proporties explicitly. For instance, cleanup of the environment for
the setuid image does not require retyping all names.
Reviewed by: arichardson, markj
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D31545
The current lookup prefers a strong definition to a STB_WEAK definition
(similar to glibc pre-2.2 behavior) which does not conform to the ELF
specification.
The non-compliant behavior provoked https://reviews.llvm.org/D4418
which was intended to fix -shared-libasan but introduced
new problems (and caused some sanitizer tests (e.g.
test/asan/TestCases/interception_failure_test.cpp) to fail): sanitizer
interceptors are STB_GLOBAL instead of STB_WEAK, so defining a second
STB_GLOBAL interceptor can lead to a multiple definition linker error.
For example, in a -fsanitize={address,memory,...} build, libc functions
like malloc/free/strtol/... cannot be provided by user object files.
See
https://docs.freebsd.org/cgi/getmsg.cgi?fetch=16483939+0+archive/2014/freebsd-current/20140716.freebsd-current
for discussions.
This patch implements the ELF-compliant behavior when LD_DYNAMIC_WEAK is
set. STB_WEAK wrestling in symbol lookups in `Search the dynamic linker
itself` are untouched.
Reviewed by: kib
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D26352
Variant I architectures use off and Variant II ones use size + off.
Define TLS_VARIANT_I/TLS_VARIANT_II symbols similarly to how libc
handles it.
Reviewed by: kib
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D31539
Differential revision: https://reviews.freebsd.org/D31541
For a Variant II architecture, the TP offset of a TLS symbol is st_value -
tlsoffset + r_addend. tlsoffset is computed by either calculate_tls_offset
or calculate_first_tls_offset.
The return value of calculate_first_tls_offset is the smallest integer
satisfying res >= size and (-res) % p_align = p_vaddr % p_align
(= p_offset % p_align). (The formula is a bit contrived. The basic idea
is to subtract the minimum integer from size + align - 1 so that the result
ihas the expected remainder.)
Reviewed by: kib
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D31538
Differential revision: https://reviews.freebsd.org/D31541
lld rounds up p_memsz(PT_GNU_RELRO) to satisfy common-page-size. If the
page size is smaller than common-page-size, rounding up relro_size may
incorrectly make some RW pages read-only.
GNU ld, gold, and ld.lld ensures p_vaddr+p_memsz is a multiple of
common-page-size. While max-page-size >= system the page size,
common-page-size can be smaller than the system page size.
Submitted by: MaskRay
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D31498
This adds two new options WITH_ASAN/WITH_UBSAN that can be set to
enable instrumentation of all binaries with AddressSanitizer and/or
UndefinedBehaviourSanitizer. This current patch is almost sufficient
to get a complete buildworld with sanitizer instrumentation but in
order to actually build and boot a system it depends on a few more
follow-up commits.
Reviewed By: brooks, kib, markj
Differential Revision: https://reviews.freebsd.org/D31043
lld emits several GOT relocations referencing the null sumbol in libc.so
when compiled with -ftls-model=initial-exec. This symbol is specified
to be undefined.
We generally do not handle dynamic TLS relocations against weak,
undefined symbols, so avoid printing a warning here. This makes it
possible to compile libc.so using the initial-exec TLS model on arm64.
Reviewed by: jrtc27, kib
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31069
This prevents these tests from being compiled with ASAN since the asan
interceptors also define opendir() but matching the libc function.
Reviewed By: oshogbo, kib, markj
Differential Revision: https://reviews.freebsd.org/D31038
The mprotect() call was failing on CheriBSD when changing rtld's relro
page permissions due to missing CHERI capability permissions on the
mprotect() argument but did not report an error since the return value
was being ignored. It should never fail on any supported FreeBSD
architecture, but checking the return value seems like a good
sanity check to me.
Reviewed By: kib, imp
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D30820
There should be no error after startup.
PR: 255698
Reported by: Eugene M. Kim <astralblue@gmail.com>
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
rtld preserves its current error message around calls to user init/fini
lists, to not override original error with potential secondary errors
caused by user code recursing into rtld. After 4d9128da54,
the preservation of the string itself is not enough, the 'seen'
indicator must be preserved as well. Otherwise, since new code does not
clear string (it cannot), call to _rtld_error() from errmsg_restore()
revived whatever message was consumed last.
Change errmsg_save() to return structure recording both 'seen' indicator
and the message, if any.
PR: 255698
Reported by: Eugene M. Kim <astralblue@gmail.com>
Sponsored by: The FreeBSD Foundation
MFC after: 3 days
