Reloading is a heavy-weight operation, and currently it is not
possible to stop an orchestrator from spamming reload requests.
Add configuration options to allow rate-limiting.
This improves the performance of parsing the file and reduces memory pressure.
Running 'fuzz-etc-hosts timeout-strv' with valgrind,
Before:
total heap usage: 321,020 allocs, 321,020 frees, 15,820,387,193 bytes allocated
real 0m23.531s
user 0m21.458s
sys 0m1.961s
After:
total heap usage: 112,408 allocs, 112,408 frees, 7,297,480 bytes allocated
real 0m8.664s
user 0m8.545s
sys 0m0.065s
Hopefully fixes oss-fuzz#47708 (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=47708).
So, i think "erofs" is probably the better, more modern alternative to
"squashfs". Many of the benefits don't matter too much to us I guess,
but there's one thing that stands out: erofs has a UUID in the
superblock, squashfs has not. Having an UUID in the superblock matters
if the file systems are used in an overlayfs stack, as overlayfs uses
the UUIDs to robustly and persistently reference inodes on layers in
case of metadata copy-up.
Since we probably want to allow such uses in overlayfs as emplyoed by
sysext (and the future syscfg) we probably should ramp up our erofs game
early on. Hence let's natively support erofs, test it, and in fact
mention it in the docs before squashfs even.
Some distributions have started phasing out SHA1, which breaks
the systemd-measure test case in its current form. Let's make sure we
can use SHA1 for signing beforehand to mitigate this.
Spotted on RHEL 9, where SHA1 signatures are disallowed by [0]:
```
openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -out "/tmp/pcrsign-private.pem"
...
openssl rsa -pubout -in "/tmp/pcrsign-private.pem" -out "/tmp/pcrsign-public.pem"
writing RSA key
/usr/lib/systemd/systemd-measure sign --current --bank=sha1 --private-key="/tmp/pcrsign-private.pem" --public-key="/tmp/pcrsign-public.pem"
Failed to initialize signature context.
```
[0] https://gitlab.com/redhat/centos-stream/rpms/openssl/-/blob/c9s/0049-Selectively-disallow-SHA1-signatures.patch
Otherwise the `terminate()` method sends SIGKILL rather quickly (~0.3s),
which then leaves a dangling scope on the host system, breaking further
test executions.
This handles a Debian-specific quirk where /etc/default/locale is used
instead of /etc/locale.conf. There is currently special handling for
this in testsuite-73.sh, so the quirk should be handled here too for
consistency.
--include-partitions and --exclude-partitions now fully exclude
partitions from repart. Whenever a partition type is excluded, we
don't take any partitions of that type into account at all when
running systemd-repart.
--skip-partitions= is introduced to do what --exclude-partitions did
previously. Any skipped partitions are taken into acount when doing
size calculations, but are not yet populated.
Why do we need both concepts? Exclusion is needed so that we can
use shared repart definitions to generate bootable and non-bootable
images. When generating a non-bootable image, we use --exclude-partitions
to exclude the ESP partition. Skipping is needed so that we can
populate the root partition while skipping the ESP partition, get
the roothash of the root partition, use that to generate a UKI, and
finally populate the ESP partition with the UKI included.
- Use inst_recursive() and image_install() helpers where appropriate
- Update comments to explain why we need to install the test data manually in
$initdir
- Install manual/ in $initdir as TEST-35-LOGIN relies on
manual/test-session-properties
To make sure rootless mode keeps working, let's run all repart
integration tests that we can without root privileges. The only ones
we need to keep running with root privileges are the tests that operate
on a block/loop device and those that use --image=.
mkfs.vfat doesn't support specifying a root directory to bootstrap
the filesystem from (see https://github.com/dosfstools/dosfstools/issues/183).
Instead, we can use the mcopy tool from the mtools package to copy
files into the vfat filesystem after creating it without needing to
mount the vfat filesystem.
This is useful to force off fancy unicode glyph use (i.e. use "->"
instead of "→"), which is useful in tests where locales might be
missing, and thus control via $LC_CTYPE is not reliable.
Use this in TEST-58, to ensure the output checks we do aren't confused
by missing these glyphs being unicode or not.
fsck(8) is located in /usr/sib/ on Debian sid:
stdout:
*** Running /home/christian/Coding/workspaces/systemd/test/testdata/test-fstab-generator/test-01-dev-nfs.input
*** Running /home/christian/Coding/workspaces/systemd/test/testdata/test-fstab-generator/test-02-dhcp.input
*** Running /home/christian/Coding/workspaces/systemd/test/testdata/test-fstab-generator/test-03-dhcp6.input
*** Running /home/christian/Coding/workspaces/systemd/test/testdata/test-fstab-generator/test-04-nfs.input
*** Running /home/christian/Coding/workspaces/systemd/test/testdata/test-fstab-generator/test-05-nfs4.input
*** Running /home/christian/Coding/workspaces/systemd/test/testdata/test-fstab-generator/test-06-ipv4.input
*** Running /home/christian/Coding/workspaces/systemd/test/testdata/test-fstab-generator/test-07-ipv6.input
*** Running /home/christian/Coding/workspaces/systemd/test/testdata/test-fstab-generator/test-08-implicit-nfs.input
*** Running /home/christian/Coding/workspaces/systemd/test/testdata/test-fstab-generator/test-09-cifs.input
*** Running /home/christian/Coding/workspaces/systemd/test/testdata/test-fstab-generator/test-10-iscsi.input
*** Running /home/christian/Coding/workspaces/systemd/test/testdata/test-fstab-generator/test-11-live.input
*** Running /home/christian/Coding/workspaces/systemd/test/testdata/test-fstab-generator/test-12-dev-sdx.input
--- /dev/fd/63 2022-11-04 15:39:13.131532174 +0100
+++ /dev/fd/62 2022-11-04 15:39:13.131532174 +0100
@@ -6,3 +6,4 @@
initrd-usr-fs.target.requires
initrd-usr-fs.target.requires/sysroot.mount
sysroot.mount
+systemd-fsck-root.service
**** Unexpected output for /home/christian/Coding/workspaces/systemd/test/testdata/test-fstab-generator/test-12-dev-sdx.input
stderr:
Skipping root directory handling, as root on NFS was requested.
Skipping root directory handling, as root on NFS was requested.
Skipping root directory handling, as root on NFS was requested.
Skipping root directory handling, as root on NFS was requested.
Skipping root directory handling, as root on NFS was requested.
Skipping root directory handling, as root on NFS was requested.
Skipping root directory handling, as root on NFS was requested.
Skipping root directory handling, as root on NFS was requested.
Skipping root directory handling, as root on CIFS was requested.
Skipping root directory handling, as root on iSCSI was requested.
Skipping root directory handling, as root on live image was requested.
Found entry what=/dev/sdx1 where=/sysroot type=n/a opts=ro
Checking was requested for /dev/sdx1, but the fsck command does not exist.
In my understanding user group "3" (aka "sys") is kept for historical reasons
but not really useful these days. That's probably explained why this group
isn't defined on openSUSE.
Hence let's drop reference to this user group, this shouldn't lessen the
revelance of the test since SupplementaryGroups= is still tested with 2 other
groups.
Also, fix a race condition introduced by d16684fe13:
```
[ 16.904218] H testsuite-26.sh[394]: + systemd-run --unit failed.service /bin/false
[ 16.964783] H systemd[845]: failed.service: Executing: /bin/false
[ 16.965062] H systemd[1]: Started failed.service.
[ 16.965462] H testsuite-26.sh[844]: Running as unit: failed.service
[ 16.966390] H testsuite-26.sh[394]: + systemctl is-failed failed.service
[ 16.977970] H testsuite-26.sh[846]: active
[ 16.978403] H systemd[1]: failed.service: Main process exited, code=exited, status=1/FAILURE
[ 16.978478] H systemd[1]: failed.service: Failed with result 'exit-code'.
```
If another bound interface (dummy98) will be removed before that dummy99
is processed by udevd, then removing dummy98 in the next step makes the
target interface (test1) bring down.
Follow-up for 3e2f7c46da.
The script has a shebang and .sh extension, so make it executable
W: systemd-tests: script-not-executable [usr/lib/systemd/tests/testdata/units/assert.sh]
The outstanding kernel panic should be already fixed in recent enough
kernels by [0]. To make the test safe to run anywhere, let's implement
a simple kernel version check and run the test only if we're running
with at least kernel 6.x. The patch might be in some 5.x kernels as
well, but let's be on the safe side and use 6.x as a baseline here
(which is currently the case for Arch and Fedora Rawhide anyway).
[0] https://lore.kernel.org/netdev/7b3fd03e1a46047e5ffe2a389fe74501f0a93206.1656519221.git.sd@queasysnail.net/T/#u
- use `udevadm wait` instead of `udevadm info --wait-initialized`,
- use `timeout` command instead of the fixed time sleep,
- add basic tests for #25106,
- add brief comment about #25115.
This fix unfortunately introduced a much worse regression that
is affecting many users, so let's revert it for now and rework
it in the next release.
This reverts commit 5ded3917a1.
Fixes https://github.com/systemd/systemd/issues/24984
The lists of directives for fuzzer tests are maintained manually in the
repo. There is a tools/check-directives.sh script that runs during test
phase and reports stale directive lists.
Let's rework the script into a generator so that these directive files
are created on-the-flight and needn't be updated whenever a unit file
directives change. The scripts is rewritten in Python to get rid of gawk
dependency and each generated file is a separate meson target so that
incremental builds refresh what is just necessary (and parallelize
(negligible)).
Note: test/fuzz/fuzz-unit-file/directives-all.slice is kept since there
is not automated way to generate it (it is not covered by the check
script neither).
The precise bounds of the scheduling priority depend on the scheduling policy,
so depending on the order in which the two settings are specified the
validation might pass or fail.
When checking the setting only validate the outer range (valid values in general are 0 to 99),
and let the execution fail later if the priority does not match the
specified policy (1 to 99 for RR/FIFO, 0 for the rest).
Fixes https://github.com/systemd/systemd/issues/20320
Quoting https://github.com/systemd/systemd/pull/25050#discussion_r998721845:
This part seems to be quite racy, at least in the C8S job:
[ 1767.520856] H testsuite-15.sh[35]: *** test transient slice drop-ins
[ 1767.520856] H testsuite-15.sh[35]: + mkdir -p /etc/systemd/system/slice.d
[ 1767.522480] H testsuite-15.sh[35]: + mkdir -p /etc/systemd/system/a-.slice.d
[ 1767.524992] H testsuite-15.sh[35]: + mkdir -p /etc/systemd/system/a-b-.slice.d
[ 1767.526799] H testsuite-15.sh[35]: + mkdir -p /etc/systemd/system/a-b-c.slice.d
[ 1767.528302] H testsuite-15.sh[35]: + echo -e '[Unit]\nDocumentation=man:drop1'
[ 1767.528434] H testsuite-15.sh[35]: + echo -e '[Unit]\nDocumentation=man:drop2'
[ 1767.528519] H testsuite-15.sh[35]: + echo -e '[Unit]\nDocumentation=man:drop3'
[ 1767.528595] H testsuite-15.sh[35]: + echo -e '[Unit]\nDocumentation=man:drop4'
[ 1767.528676] H testsuite-15.sh[35]: + systemctl cat a-b-c.slice
[ 1767.541321] H systemctl[1042]: No files found for a-b-c.slice.
[ 1767.542854] H systemd[1]: testsuite-15.service: Main process exited, code=exited, status=1/FAILURE
[ 1767.542995] H systemd[1]: testsuite-15.service: Failed with result 'exit-code'.
[ 1767.543360] H systemd[1]: Failed to start testsuite-15.service.
[ 1767.543542] H systemd[1]: testsuite-15.service: Consumed 1.586s CPU time.
[ 1767.543938] H systemd[1]: Reached target testsuite.target.
[ 1767.545737] H systemd[1]: Starting end.service...
Otherwise we might hit a race where we read the test log just before
it's fully written to the disk:
```
======================================================================
FAIL: test_interleaved (__main__.ExecutionResumeTest.test_interleaved)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/root/systemd/test/test-exec-deserialization.py", line 170, in test_interleaved
self.check_output(expected_output)
File "/root/systemd/test/test-exec-deserialization.py", line 111, in check_output
self.assertEqual(output, expected_output)
AssertionError: 'foo\n' != 'foo\nbar\n'
foo
+ bar
```
With some debug:
```
test_interleaved (__main__.ExecutionResumeTest.test_interleaved) ...
Assertion failed; file contents just after the assertion:
b'foo\n'
File contents 5 seconds later:
b'foo\nbar\n'
FAIL
```
Seen quite often in CentOS CI on the fast baremetal machines.
(s) is just ugly with a vibe of DOS. In most cases just using the normal plural
form is more natural and gramatically correct.
There are some log_debug() statements left, and texts in foreign licenses or
headers. Those are not touched on purpose.
TEST-69 uses a Python wrapper around the systemd-nspawn call, which on
error calls the `spawn.terminate()` method. However, with no arguments
it will only use SIGHUP and SIGINT signals - this might leave a stuck
container around, causing fails if the test is run again. With `force=True`
SIGKILL is used as well (if necessary).
TEST-34 complains in `test_check_writable` when running with gcov, as
the build directory tree is not writable with DynamicUser=true. As I had
no luck with $GCOV_PREFIX and other runtime gcov configuration, let's
just ignore the gcov errors for this test.
We want to test four things:
- that the transient units are successfully started when drop-ins exist
- that the transient setings override the defaults
- the drop-ins override the transient settings (the same as for a normal unit)
- that things are the same before and after a reload
To make things more fun, we start and stop units in two different ways: via
systemctl and via a direct busctl invocation. This gives us a bit more coverage
of different code paths.
Slices are worth testing too, because they don't need a fragment path so they
behave slightly differently than service units. I'm making this a separate
patch from the actual tests that I wanted to add later because it's complex
enough on its own.
clear_services() is renamed to clear_units() and now takes a full
unit name including the suffix as an argument.
_clear_service() is renamed to clear_unit() and changed likewise.
create_service() didn't have the same underscore prefix, and I don't think
it's useful or needed for a local function, so it is removed.
No functional change.
