Let's make the creds directories a bit more discoverable and make it
easier for users to use them. This also allows us to fix the
mode to 0700 for /etc instead of the usual 0755 which is what probably
would happen if users had to create this directory themselves.
Build option "link-portabled-shared" to build a statically linked
systemd-portabled by using
-Dlink-portabled-shared=false
on systems with full systemd stack except systemd-portabled, such
as CentOS/RHEL 9.
As part of the build, we would populate build/test/sys/ using
sys-script.py, and then udev-test.p[ly] would create a tmpfs instance
on build/test/tmpfs and copy the sys tree to build/test/tmpfs/sys.
Also, we had udev-test.p[ly] which called test-udev. test-udev was
marked as a manual test and installed, but neither udev-test.p[ly] or
sys-script.py were.
test-udev is renamed to udev-rule-runner, which reduces confusion and
frees up the test-udev name. udev-test.py is renamed to test-udev.py.
All three files are now installed.
test-udev.py is modified to internally call sys-script.py to set up the
sys tree. Copying and creating it from scratch should take the same
amount of time. We avoid having a magic directory, everything is now
done underneath a temporary directory.
test-udev.py is now a normal installed test, and run-unit-tests.py will
pick it up. When test-udev.py is invoked from meson, the path to
udev-rule-runner is passed via envvar; when it is invoked via
run-unit-tests.py or directly, it looks for udev-rule-runner in a relative
path.
The goal of this whole change is to let Debian drop the 'udev' test.
It called sys-script.py and udev-test.pl from the source directory and
had to recreate a bunch of the logic. Now test-udev.py will now be called
via 'upstream'.
We install a kernel with layout=uki and uki_generator=ukify, and test
that a UKI gets installed in the expected place. The two plugins cooperate,
so it's easiest to test them together.
We can always build the standalone version whenever we build the normal version
(the dependencies are the same). In most builds standalone binaries would be
disabled. But it is occasionally useful to have them for testing, so move the
conditional to install:, so the binaries can be build by giving the explicit
target name.
The default of 'build_by_default' for executable() is sadly true (since meson
0.38.0), so need to specify build_by_default: too.
Also add systemd-shutdown.standalone to public_programs for additional testing.
When executable() or custom_target() has install: that is conditional as is
false (i.e. not install:true), it won't be built by default. (build_by_default:
defaults to install:). But if that program is added to public_programs, it will
be build by default because it is pulled in by the test, effectively defeating
the disablement.
While at it, make 'ukify' follow the same pattern as 'kernel-install'.
They will be used later together.
/usr/lib/systemd/tests may contain more than the unit tests. For example on
SUSE we also install the integration tests there.
Putting the unit tests in a dedicated directory named 'unit-tests' makes the
layout cleaner.
Note that `run-unit-tests.py` has not been moved so we don't need to adjust
(Fedora) packaging and users also don't need to descend into the subdirectory.
This adds back sd-boot builds by using meson compile targets directly.
We can do this now, because userspace binaries use the special
dependency that allows us to easily separate flags, so that we don't
pass anything to EFI builds that shouldn't be passed.
Additionally, we pass a bunch of flags to hopefully disable/override any
distro provided flags that should not be used for EFI binaries.
Fixes: #12275
This drops all mentions of gnu-efi and its manual build machinery. A
future commit will bring bootloader builds back. A new bootloader meson
option is now used to control whether to build sd-boot and its userspace
tooling.
Although udev rules are already being checked by rule-syntax-check.py
script, also check them using udevadm verify which performs more
thorough checks.
With this change we'll install a symlink /sbin/mount.ddi →
systemd-dissect. If invoked that way we'll do the equivalent of
systemd-dissect --mount.
This makes DDIs mountable directly via the "mount" command, by
specifying the "-t ddi" pseudo file system type. Moreover you can now
mount DDIs directly via /etc/fstab, by specifying "ddi" in the file
system column (3rd column).
IN C23, thread_local is a reserved keyword and we shall therefore
do nothing to redefine it. glibc has it defined for older standard
version with the right conditions.
v2 by Yu Watanabe:
Move the definition to missing_threads.h like the way we define e.g.
missing syscalls or missing definitions, and include it by the users.
Co-authored-by: Yu Watanabe <watanabe.yu+github@gmail.com>
When mkosi is run from git-worktree(1), the .git is not a repository
directory but a textfile pointing to the real git dir
(e.g. /home/user/systemd/.git/worktrees/systemd-worktree). This git dir
is not bind mounted into build environment and it fails with:
> fatal: not a git repository: /home/user/systemd/.git/worktrees/systemd-worktree
> test/meson.build:190:16: ERROR: Command `/usr/bin/env -u GIT_WORK_TREE /usr/bin/git --git-dir=/root/src/.git ls-files ':/test/dmidecode-dumps/*.bin'` failed with status 128.
There is already a fallback to use shell globbing instead of ls-files,
use it with git worktrees as well.
Most of the support for valgrind was under HAVE_VALGRIND_VALGRIND_H, i.e. we
would enable if the valgrind headers were found. The operations then we be
conditionalized on RUNNING_UNDER_VALGRIND.
But in a few places we had code which was conditionalized on VALGRIND, i.e. the
config option. I noticed because I compiled with -Dvalgrind=true on a machine
that didn't have valgrind.h, and the build failed because
RUNNING_UNDER_VALGRIND was not defined. My first idea was to add a check that
the header is present if the option is set, but it seems better to just remove
the option. The code to support valgrind is trivial, and if we're
!RUNNING_UNDER_VALGRIND, it has negligible cost. And the case of running under
valgrind is always some special testing/debugging mode, so we should just do
those extra steps to make valgrind output cleaner. Removing the option makes
things simpler and we don't have to think if something should be covered by the
one or the other configuration bit.
I had a vague recollection that in some places we used -Dvalgrind=true not
for valgrind support, but to enable additional cleanup under other sanitizers.
But that code would fail to build without the valgrind headers anyway, so
I'm not sure if that was still used. If there are uses like that, we can
extend the condition for cleanup_pools().
Although this slightly more verbose it makes it much easier to reason
about. The code that produces the tests heavily benefits from this.
Test lists are also now sorted by test name.
A lot of tests can be defined by just their filename. Moving into their
own list keeps things simpler, especially with the next commit. It also
makes it easier to keep the lists sorted.
Allow defining the default keymap to be used by
vconsole-setup through a build option. A template
vconsole.conf also gets populated by tmpfiles if
it doesn't exist.
This will warn if fake flexible arrays are re-introduced. I'm not using
-Werror=… because we may still get warnings when compiling against old kernel
headers. We can crank this up to error later.
-fstrict-flex-arrays means that the compiler doesn't have to assume that any
trailing array is a flex array. I.e. unless the array is declared without a
specified size, only indices in the declared range are valid.
-Warray-bounds turns on the warnings about out-of-bounds array accesses.
-Warray-bounds=2 does some more warnings, with higher false positive rate. But
it doesn't seem to yield any false positives in our codebase, so enable it.
clang supports -Warray-bounds, but not -Warray-bounds=2.
gcc supports both.
gcc-13 supports -fstrict-flex-arrays.
See https://people.kernel.org/kees/bounded-flexible-arrays-in-c for a long
discussion of use in the kernel.
Config options are -Ddefault-timeout-sec= and -Ddefault-user-timeout-sec=.
Existing -Dupdate-helper-user-timeout= is renamed to -Dupdate-helper-user-timeout-sec=
for consistency. All three options take an integer value in seconds. The
renaming and type-change of the option is a small compat break, but it's just
at compile time and result in a clear error message. I also doubt that anyone was
actually using the option.
This commit separates the user manager timeouts, but keeps them unchanged at 90 s.
The timeout for the user manager is set to 4/3*user-timeout, which means that it
is still 120 s.
Fedora wants to experiment with lower timeouts, but doing this via a patch would
be annoying and more work than necessary. Let's make this easy to configure.
The unlink command removes an entry from the ESP including
referenced files that are not referenced in other entries. That is
useful eg to have multiple entries that use the same kernel with
different options.
The cleanup command removes all files that are not referenced by any
entry.
These options allow measuring the volume key used for unlocking the
volume to a TPM2 PCR. This is ideally used for the volume key of the
root file system and can then be used to bind other resources to the
root file system volume in a secure way.
See: #24503
We converted to not using #ifdef for most of our defines because the syntax is
nicer and we are protected against typos and can set -Werror=undef. Let's do
the same for SD_BOOT. The define is nicely hidden in build.h for normal builds,
and for EFI builds we were already setting SD_BOOT on the commandline.
Some gymnastics were needed to import ukify as a module. Before the file
was templated, this was trivial: insert the directory in sys.path, call import.
But it's a real pain to import the unsuffixed file after processing. Instead,
the untemplated file is imported, which works well enough for tests and is
very simple.
The tests can be called via pytest:
PATH=build/:$PATH pytest -v src/ukify/test/test_ukify.py
or directly:
PATH=build/:$PATH src/ukify/test/test_ukify.py
or via the meson test machinery output:
meson test -C build test-ukify -v
or without verbose output:
meson test -C build test-ukify
Zekret files are obfuscated using base64.
The option is added because we have a similar one for kernel-install. This
program requires python, and some people might want to skip it because of this.
The tool is installed in /usr/lib/systemd for now, since the interface might
change.
A template file is used, but there is no .in suffix.
The problem is that we'll later want to import the file as a module
for tests, but recent Python versions make it annoyingly hard to import
a module from a file without a .py suffix. imp.load_sources() works, but it
is deprecated and throws warnings.
importlib.machinery.SourceFileLoader().load_module() works, but is also
deprecated. And the documented replacements are a maze of twisted little
callbacks that result in an empty module.
So let's take the easy way out, and skip the suffix which makes it easy
to import the template as a module after adding the directory to sys.path.
I'd like to use this as a basis for an exitrd:
When compiled with -Dstandalone-binaries=true -Db_lto=true -Dbuildtype=release,
the new file is 800k. It's more than I'd like, but still quite a bit less
than libsystemd-shared.so, which is 3800k.
systemd-cryptenroll complains (but succeeds!) upon binding to a signed PCR
policy:
$ systemd-cryptenroll --unlock-key-file=/tmp/passphrase --tpm2-device=auto
--tpm2-public-key=... --tpm2-signature=..." /tmp/tmp.img
ERROR:esys:src/tss2-esys/esys_iutil.c:394:iesys_handle_to_tpm_handle() Error: Esys invalid ESAPI handle (40000001).
WARNING:esys:src/tss2-esys/esys_iutil.c:415:iesys_is_platform_handle() Convert handle from TPM2_RH to ESYS_TR, got: 0x40000001
ERROR:esys:src/tss2-esys/esys_iutil.c:394:iesys_handle_to_tpm_handle() Error: Esys invalid ESAPI handle (40000001).
WARNING:esys:src/tss2-esys/esys_iutil.c:415:iesys_is_platform_handle() Convert handle from TPM2_RH to ESYS_TR, got: 0x4000000
New TPM2 token enrolled as key slot 1.
The problem seems to be that Esys_LoadExternal() function from tpm2-tss
expects a 'ESYS_TR_RH*' constant specifying the requested hierarchy and not
a 'TPM2_RH_*' one (see Esys_LoadExternal() -> Esys_LoadExternal_Async() ->
iesys_handle_to_tpm_handle() call chain).
It all works because Esys_LoadExternal_Async() falls back to using the
supplied values when iesys_handle_to_tpm_handle() fails:
r = iesys_handle_to_tpm_handle(hierarchy, &tpm_hierarchy);
if (r != TSS2_RC_SUCCESS) {
...
tpm_hierarchy = hierarchy;
}
Note, TPM2_RH_OWNER was used on purpose to support older tpm2-tss versions
(pre https://github.com/tpm2-software/tpm2-tss/pull/1531), use meson magic
to preserve compatibility.
Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Offline encryption can be done without mounting the luks device. For
now we still use loop devices to split out the partition we want to
write to but in a later commit we'll replace this with a regular file.
For offline encryption, we need to keep 2x the luks header size space
free at the end of the partition, so this means our encrypted partitions
will be 16M larger than before.
Due to "historical reasons" both gcc and clang treat *all* trailing
arrays members as flexible arrays, this has an evil side effect
of inhibiting bounds checks on such members as __builtin_object_size
cannot say for sure that:
struct {
...
type foo[3];
}
has a trailing foo member of fixed size rather than unspecified.
Ideally we should use -fstrict-flex-arrays as is, but we have to
tolerate kernel uapi headers that use [0] and third party libraries
written in c89 that may use [1] like curl.
Follow-up to c47511da7e.
```
archlinux_systemd_ci: In file included from ../build/src/dissect/dissect.c:15:
archlinux_systemd_ci: ../build/src/basic/build.h:4:10: fatal error: version.h: No such file or directory
archlinux_systemd_ci: 4 | #include "version.h"
archlinux_systemd_ci: | ^~~~~~~~~~~
archlinux_systemd_ci: compilation terminated.
```
```
archlinux_systemd_ci: In file included from ../build/src/journal/cat.c:13:
archlinux_systemd_ci: ../build/src/basic/build.h:4:10: fatal error: 'version.h' file not found
archlinux_systemd_ci: #include "version.h"
archlinux_systemd_ci: ^~~~~~~~~~~
archlinux_systemd_ci: 1 error generated.
```
```
archlinux_systemd_ci: In file included from ../build/src/sysext/sysext.c:10:
archlinux_systemd_ci: ../build/src/basic/build.h:4:10: fatal error: version.h: No such file or directory
archlinux_systemd_ci: 4 | #include "version.h"
archlinux_systemd_ci: | ^~~~~~~~~~~
archlinux_systemd_ci: compilation terminated.
archlinux_systemd_ci: FAILED: systemd-inhibit.p/src_login_inhibit.c.o
```
```
archlinux_systemd_ci: In file included from ../build/src/login/inhibit.c:12:
archlinux_systemd_ci: ../build/src/basic/build.h:4:10: fatal error: version.h: No such file or directory
archlinux_systemd_ci: 4 | #include "version.h"
archlinux_systemd_ci: | ^~~~~~~~~~~
archlinux_systemd_ci: compilation terminated.
```
...
version.h can be generated after compilation starts, creating a race condition
between compilation of various .c files and creation of version.h. Let's add it
as a dependency to more build targets that require version.h or build.h.
So far we played whack'a'mole by adding versiondep whenever compilation failed.
In principle any target which includes compilation (i.e. any that has .c
sources directly), could require this. I don't understand why we didn't see
more failures… But it seems reasonable to just add the dependency more widely.
In the Xen case, it's the hypervisor which manages kexec. We thus
have to ask it whether a kernel is loaded, instead of relying on
/sys/kernel/kexec_loaded.
Using fsopen()/fsconfig(), we can check if hidepid/subset are supported to
avoid the noisy logs from the kernel if they aren't supported. This works
on centos/redhat 8 as well since they've backported fsopen()/fsconfig().
Repart is growing into an important tool on its own, and users might
want to install newer versions on systems that have older systemd. Let's
make this easier by providing a standalone binary.
Related to https://github.com/systemd/mkosi/issues/1228.
Semi-quoting https://github.com/systemd/systemd/issues/25057:
clang-16 has made the choice to turn on -Werror=implicit-function-declaration,implicit-int.
(See Gentoo's tracker bug https://bugs.gentoo.org/870412).
Added in commit 132c73b57a, systemd now does a
check to see if libatomic is needed with some compile/link tests with e.g.
__atomic_exchange_1, but the tests don't provide a prototype for
__atomic_exchange_1 so with clang-16 the test fails, breaking the build.
Let's simplify things by linking to libatomic unconditionally if it is found
and seems to work. If actually unneeded, it might be dropped via --as-needed.
This seems to work with gcc and clang.
declare_dependency() is used instead of cc.find_library(), because the latter
picks up a symlink in gcc private directory (e.g.
/usr/lib/gcc/x86_64-redhat-linux/12/libatomic.so), and we don't want that.
Fixes#25057.
The lists of directives for fuzzer tests are maintained manually in the
repo. There is a tools/check-directives.sh script that runs during test
phase and reports stale directive lists.
Let's rework the script into a generator so that these directive files
are created on-the-flight and needn't be updated whenever a unit file
directives change. The scripts is rewritten in Python to get rid of gawk
dependency and each generated file is a separate meson target so that
incremental builds refresh what is just necessary (and parallelize
(negligible)).
Note: test/fuzz/fuzz-unit-file/directives-all.slice is kept since there
is not automated way to generate it (it is not covered by the check
script neither).
--convert writes the journal files read by journalctl to the given
location. The location should be specified as a full journal file
path (e.g. /a/b/c/converted.journal). The directory specifies where
the converted journal files will be stored. The filename specifies
the naming convention the converted journal files will follow.
