Rewrite the test in bash and make it part of our integration test suite,
so it's actually executed in all our upstream CI environments.
The original test is flaky in environments where daemon-reload might
occur during the test runtime (e.g. when running the test in parallel
with the systemd-networkd test suite). Also, it was run only in CentOS
CI in limited way (i.e. without sanitizers), since it tests the host's
systemd, instead of the just built one.
Resolves: #29943
udevadm lock did not propagate the return code from the child process
because all positive values were treated as success.
v2:
Now 'udevadm test-builtin' ignores all positive return values from the
builtin commands. Otherwise, as the hwdb builtin returns an positive value
when a matching entry found, 'udevadm test-builtin hwdb' will fail.
v3:
Initialize partition table before calling 'sfdisk --delete'.
Co-authored-by: Yu Watanabe <watanabe.yu+github@gmail.com>
Skip calling start and stop methods on unit objects, as doing that is
not only time consuming, but it also starts/stops units that interfere
with the machine state. The actual code paths should be covered (to some
degree) by the respective method counterparts on the manager object.
Since the triggered unit intentionally fails without consuming any data
from the socket, we'd try to trigger it again and again, and we might
try to check the unit state in one of the "in-between" states, failing
the test:
[ 165.271698] H testsuite-07.sh[1032]: + systemctl start badbin_assert.socket
[ 165.977637] H testsuite-07.sh[1032]: + socat - ABSTRACT-CONNECT:badbin_assert.socket
[ 165.983787] H systemd[1]: Cannot find unit for notify message of PID 1039, ignoring.
[ 166.817187] H testsuite-07.sh[1032]: + timeout 10 sh -c 'while systemctl is-active badbin_assert.service; do sleep .5; done'
[ 167.049218] H testsuite-07.sh[1065]: active
[ 167.146854] H systemd[1]: Listening on badbin_assert.socket.
[ 167.163473] H systemd[1]: badbin_assert.socket: Incoming traffic
[ 167.542626] H systemd[1]: Cannot find unit for notify message of PID 1065, ignoring.
[ 167.543437] H (badbin)[1062]: badbin_assert.service: Failed to execute /tmp/badbin: Exec format error
[ 167.548346] H systemd[1]: badbin_assert.service: Main process exited, code=exited, status=203/EXEC
[ 167.549482] H systemd[1]: badbin_assert.service: Failed with result 'exit-code'.
[ 167.561537] H systemd[1]: badbin_assert.socket: Incoming traffic
[ 167.933390] H systemd[1]: Started badbin_assert.service.
[ 167.950489] H (badbin)[1070]: badbin_assert.service: Failed to execute /tmp/badbin: Exec format error
[ 167.956318] H systemd[1]: badbin_assert.service: Main process exited, code=exited, status=203/EXEC
[ 167.957173] H systemd[1]: badbin_assert.service: Failed with result 'exit-code'.
[ 167.974609] H systemd[1]: badbin_assert.socket: Incoming traffic
[ 168.042838] H testsuite-07.sh[1072]: failed
[ 168.094431] H testsuite-07.sh[1075]: ++ systemctl show -P ExecMainStatus badbin_assert.service
[ 168.704022] H systemd[1]: Started badbin_assert.service.
[ 168.778680] H (badbin)[1074]: badbin_assert.service: Failed to execute /tmp/badbin: Exec format error
[ 168.826881] H systemd[1]: badbin_assert.service: Main process exited, code=exited, status=203/EXEC
[ 168.833825] H systemd[1]: badbin_assert.service: Failed with result 'exit-code'.
[ 168.923931] H testsuite-07.sh[1032]: + [[ 0 == 203 ]]
[ 168.951492] H systemd[1]: Cannot find unit for notify message of PID 1075, ignoring.
[ 168.999862] H testsuite-07.sh[615]: + echo 'Subtest /usr/lib/systemd/tests/testdata/units/testsuite-07.issue-30412.sh failed'
[ 168.999862] H testsuite-07.sh[615]: Subtest /usr/lib/systemd/tests/testdata/units/testsuite-07.issue-30412.sh failed
Follow-up for 1eeaa93de3 and 28a2d27650.
This effectively reverts fa6f37c043 just for TEST-04, as we nuke the
journal repeatedly in this test which makes it particularly hard to
debug. Let's hope the issue behind fa6f37c043 won't bite us back in this
case.
Follow-up for: fa6f37c043
Reverts: 8f7c876bdc
This is no longer necessary, as the test for which this was introduced
in the first place has this handled explicitly (testsuite-04.journal.sh).
Follow-up to 9457dd8bae.
With OpenSSL 3.2.0+ this is necessary, otherwise the verification
of such CA certificate fails badly:
$ openssl s_client -CAfile /run/systemd/remote-pki/ca.crt -connect localhost:19532
...
Connecting to ::1
CONNECTED(00000003)
Can't use SSL_get_servername
depth=1 C=CZ, L=Brno, O=Foo, OU=Bar, CN=Test CA
verify error:num=79:invalid CA certificate
verify return:1
depth=1 C=CZ, L=Brno, O=Foo, OU=Bar, CN=Test CA
verify error:num=26:unsuitable certificate purpose
verify return:1
...
---
SSL handshake has read 1566 bytes and written 409 bytes
Verification error: unsuitable certificate purpose
---
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
Server public key is 2048 bit
This TLS version forbids renegotiation.
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 26 (unsuitable certificate purpose)
These tests are not interested in most addresses specified in the .network file.
As 10-many-address.conf drop-in config for the .network file contains so
many addresses, hopefully this improves test performance.
ExecStart= and friends for .service and .socket can be specified
multiple times.
This also checks all commands for .mount and .swap, not only for the
current control command.
If the target mount point is an automount, checking it for writeability
without triggering it first is iffy and yields different results based
on kernel version:
~# systemd-run --wait --pipe -p ProtectSystem=yes bash -xec 'uname -r; mount -l | grep boot; test ! -w /boot'
Running as unit: run-u36.service; invocation ID: f948ff4f3c8e4bcfba364ead94bd0ad9
+ uname -r
4.18.0-529.el8.x86_64
+ mount -l
+ grep boot
systemd-1 on /boot type autofs (rw,relatime,fd=43,pgrp=1,timeout=120,minproto=5,maxproto=5,direct,pipe_ino=356096)
+ test '!' -w /boot
Finished with result: exit-code
Main processes terminated with: code=exited/status=1
~# systemd-run --wait --pipe -p ProtectSystem=yes bash -xec 'uname -r; mount -l | grep boot; test ! -w /boot'
Running as unit: run-u274.service; invocation ID: ccc53ed63c3249348cf714f97a3a7026
+ uname -r
6.6.7-arch1-1
+ mount -l
+ grep boot
systemd-1 on /boot type autofs (rw,relatime,fd=95,pgrp=1,timeout=120,minproto=5,maxproto=5,direct,pipe_ino=730583)
+ test '!' -w /boot
Finished with result: success
Main processes terminated with: code=exited/status=0
One solution would be to use /boot/ instead of just /boot, which triggers
the automount during the check, but in that case the mount would happen
_after_ we apply the ProtectSystem= stuff, so the mount point would
be unexpectedly writable:
~# systemd-run --wait --pipe -p ProtectSystem=yes bash -xec 'uname -r; mount -l | grep boot; test ! -w /boot/ || mount -l | grep boot'
Running as unit: run-u282.service; invocation ID: 2154f6b4cbd34ddeb3e246cb7c991918
+ uname -r
6.6.7-arch1-1
+ mount -l
+ grep boot
systemd-1 on /boot type autofs (rw,relatime,fd=95,pgrp=1,timeout=120,minproto=5,maxproto=5,direct,pipe_ino=730583)
+ test '!' -w /boot/
+ mount -l
+ grep boot
systemd-1 on /boot type autofs (rw,relatime,fd=95,pgrp=1,timeout=120,minproto=5,maxproto=5,direct,pipe_ino=730583)
/dev/vda2 on /boot type vfat (rw,nosuid,nodev,noexec,relatime,nosymfollow,fmask=0077,dmask=0077,codepage=437,iocharset=ascii,shortname=mixed,utf8,errors=remount-ro)
Let's just trigger the (possible) automounts explicitly before we do any
checks to avoid all this stuff.
Also, when at it, check that ProtectSystem=yes|full correctly protects
the ESP mount as well.
Follow-up for 97bbb9cfbd.
Otherwise we might occasionally hit the start rate limit, as we restart
the service a bunch of times:
[ 3702.280886] testsuite-75.sh[1135]: + tee /tmp/tmp.wUL8bkJwrt
[ 3702.283684] testsuite-75.sh[1135]: {}
[ 3702.284254] testsuite-75.sh[46]: + restart_resolved
[ 3702.284302] testsuite-75.sh[46]: + systemctl stop systemd-resolved.service
[ 3702.310678] testsuite-75.sh[1140]: + systemctl is-failed systemd-resolved.service
[ 3702.316766] testsuite-75.sh[1141]: inactive
[ 3702.316998] testsuite-75.sh[46]: + systemctl start systemd-resolved.service
[ 3702.322315] systemd[1]: systemd-resolved.service: Start request repeated too quickly.
[ 3702.322343] systemd[1]: systemd-resolved.service: Failed with result 'start-limit-hit'.
[ 3702.322609] systemd[1]: Failed to start systemd-resolved.service - Network Name Resolution.
[ 3702.323619] systemctl[1142]: Job for systemd-resolved.service failed.
[ 3702.323839] systemctl[1142]: See "systemctl status systemd-resolved.service" and "journalctl -xeu systemd-resolved.service" for details.
[ 3702.325035] systemd[1]: testsuite-75.service: Failed with result 'exit-code'.
[ 3702.325391] systemd[1]: Failed to start testsuite-75.service - Tests for systemd-resolved.
Follow-up for b1384db11b and 6ef512c0bb.
This patch enables IFNAME_VALID_ALTERNATIVE for checks guarding the
parsing of RestrictNetworkInterfaces=.
The underlying implementation for this option already supports
altnames.
Most of the integration tests have been made to not write to /usr
but some genuinely need to do so.
Because mkosi creates images with a read-only /usr
it is not normally writeable.
By mounting an overlayfs with /usr as the lower dir
and upper and working dirs in /var tests may write to /usr.
This test is for if devices transition from plugged -> dead -> plugged
on boot, but it is normal to see loop devices transition plugged -> dead
on first boot when systemd-repart adds the root partition.
The systemd-analyze integration test also tests chroots.
It builds its chroot by bind-mounting /,
but since /usr might be a separate mountpoint
it should create the chroot with --rbind.
It's pulled in by one of the udev rules (63-md-raid-arrays.rules) and it
fails every time, because there's no valid email address in
/etc/mdadm.conf:
[ 5.778153] testsuite-64.sh[403]: mdadm: array /dev/md/mdmirror started.
[ 5.819137] kernel: md/raid1:md127: not clean -- starting background reconstruction
[ 5.819141] kernel: md/raid1:md127: active with 2 out of 2 mirrors
[ 5.819159] kernel: md127: detected capacity change from 0 to 129024
[ 5.821950] kernel: md: resync of RAID array md127
...
[ 5.887192] mdadm[424]: mdadm: No mail address or alert command - not monitoring.
[ 5.890772] systemd[1]: Starting mdmonitor.service...
[ 5.891718] systemd[1]: Started mdmonitor.service.
[ 5.892570] systemd[1]: mdmonitor.service: Main process exited, code=exited, status=1/FAILURE
[ 5.892618] systemd[1]: mdmonitor.service: Failed with result 'exit-code'.
And as we (re)assemble the MD devices multiple times, this gets quite
noisy, especially since we later start hitting the service start rate
limit.
Fedora has the mdmonitor.service patched, so it won't start without
/etc/mdadm.conf being present, but Arch uses the upstream unit which
doesn't have such guard.
Let's just mask the service completely, which replaces all that noise
with one warning:
[ 6.553583] testsuite-64.sh[294]: + udevadm wait --settle ...
[ 6.580700] systemd[1]: sys-devices-virtual-block-md127.device: Failed to enqueue SYSTEMD_WANTS job, ignoring: Unit mdmonitor.service is masked.
Just use ExecParam directly, as these are all internal to sd-exec now
anyway. Avoids double close when execution fails after FDs are set up
for inheritance and were already re-arranged.
Fixes https://github.com/systemd/systemd/issues/30412
On Fedora systemd recently moved all of its configuration files to
/usr/lib/ [0], so make sure we account for this case as well.
[ 42.450325] testsuite-17.sh[800]: + mkdir -p /run/udev/rules.d
[ 42.466504] testsuite-17.sh[800]: + cp -f /etc/udev/udev.conf /etc/udev/udev.conf.bckp
[ 42.503348] testsuite-17.sh[802]: cp: cannot stat '/etc/udev/udev.conf': No such file or directory
[0] 29eb35530b?branch=rawhide
Since we restart systemd-udevd here a couple of times, we might hit the
rate limit in later tests:
[ 26.028355] testsuite-17.sh[2074]: + udevadm control -e
[ 26.028355] testsuite-17.sh[2074]: + udevadm control -l emerg
[ 26.126160] systemd[1]: systemd-udevd.service: Start request repeated too quickly.
[ 26.126213] systemd[1]: systemd-udevd.service: Failed with result 'start-limit-hit'.
[ 26.140310] systemd[1]: Failed to start systemd-udevd.service.
[ 26.140897] systemd[1]: systemd-udevd-control.socket: Failed with result 'service-start-limit-hit'.
[ 26.141286] systemd[1]: systemd-udevd-kernel.socket: Failed with result 'service-start-limit-hit'.
[ 26.142225] testsuite-17.sh[2074]: + udevadm control -l alert
[ 26.149206] udevadm[2088]: Failed to send request to set log level: No such file or directory
Follow-up to: 6ef512c0bb
So far we created the target directory, and the source for bind mounts,
but not workdir/upperdir for overlays, so it has to be done separately
and strictly before the unit is started, which is annoying. Check the
options when creating directories, and if upper/work directories are
specified, create them.
For unit instances install_info_discover() returns path to the template,
which then generates confusing errors when passed to
do_unit_file_enable():
~# build/systemctl --root=/tmp/systemctl-test.N9ysbz reenable templ1@two.service
Unit name: templ1@two.service; p: /etc/systemd/system/templ1@.service
Removed "/tmp/systemctl-test.N9ysbz/etc/systemd/system/services.target.wants/templ1@two.service".
Failed to reenable templ1@.service, destination unit services.target is a non-template unit.
This can also be seen with a different reproducer using getty@.service
and a simple bind mount to / - there's no error this time, but it tries
to create a symlink for the default instance (from DefaultInstance=tty1),
which is also incorrect:
~# SYSTEMD_LOG_LEVEL=debug systemctl --root /mnt/bindroot/ reenable getty@test.service
Symlink /mnt/bindroot/etc/systemd/system/getty.target.wants/getty@tty1.service → /usr/lib/systemd/system/getty@.service already exists
Follow-up to: 29a7c59abbResolves: #24740
If a unit is running in an image and wants to survive a soft-reboot,
then it can't be deactivated by the storage of the image going away.
Relax the dependency to a Wants=. Access to the image is not needed
when the unit is running anyway, so downgrade to Wants=.
The read-only bit is flipped after setting up all the mounts, so that
bind mounts can be added. Remove the early config, and add a unit
test.
Fixes https://github.com/systemd/systemd/issues/30372
When [Link] MTU= is specified in a .network file, we have no idea about
that what kind of interface will be configured with the .network file.
The maximum and minimum MTU size depend on the kind of interface.
So, we should not filter MTU eagerly in the parser.
Closes#30140.
When --after-cursor=/--cursor-file= is used together with a journal
filter, we still skipped over the first matching entry even if it wasn't
the entry the cursor points at, thus missing one "valid" entry
completely. Let's fix this by checking if the entry cursor after seeking
matches the user provided cursor, and skip to the next entry only when
the cursors match.
Resolves: #30288
Request with Range header like 'entries=<cursor>:' (with a colon at the end,
invalid syntax per the doc), is now rejected with error 400 Bad Request.
fix#4883
We can't use the systemd-journal-upload user here, since it's created
dynamically by DynamicUser=yes. However, we can use the group specified
in SupplementaryGroups=, so do exactly that.
The original reason for deny-listing it was that it's flaky there. I'm
not sure if that's still the case, but the Ubuntu CI jobs for i*86 are
gone, so this file shouldn't be needed anymore anyway.
Let's consider the following case:
- the direction is down,
- no cached entry,
- the array has 5 entry objects,
- the function test_object() reutns TEST_LEFT for the 1st object,
- the 2nd, 3rd, and 4th objects are broken, so generic_array_bisect_step()
returns TEST_RIGHT for the object.
Then, previously, generic_array_bisect_step() updated the values like the following:
0th: (m = 5, left = 0, right = 4, i = 4) -> (m = 4, left = 0, right = 3, RIGHT)
1st: (m = 4, left = 0, right = 3, i = 1) -> (m = 4, left = 2, right = 3, LEFT)
2nd: (m = 4, left = 2, right = 3, i = 2) -> (m = 2, left = 2, right = 1, RIGHT) <- ouch!!
So, assert(left < right) in generic_array_bisect() was triggered.
See issue #30210.
In such situation, there is no matching entry in the array. By returning
TEST_GOTO_PREVIOUS, generic_array_bisect() handles the result so.
Fixes a bug introduced by ab8f553d1e.
Fixes#30210.
The errors are valid, since the file system is indeed not writable, but
we don't care about the missing coverage data in this case.
Follow-up to 4a43c2b3a1.
The header and keyfile are necessary only for opening the device, not
for closing, so it is not necessary to deactivate the generated
cryptsetup unit when the header or keyfile backing store are removed.
This is especially useful in the case of softreboot, when the new
mount root is setup under /run/nextroot/ but we don't want to close
the cryptsetup devices for encrypted /var/ or so, and we simply
mount it directly on /run/nextroot/var/ before the soft-reboot.
This is the equivalent of RequiresMountsFor=, but adds Wants= instead
of Requires=. It will be useful for example for the autogenerated
systemd-cryptsetup units.
Fixes https://github.com/systemd/systemd/issues/11646
The previous patch 466266c does not make sense indeed, that is to say, if the SYSTEMD_READY is not recorded in the database, the GOTO="systemd_end" will not be applied.
The IMPORT{db} is actually a matching token, it returns false when there is no SYSTEMD_READY recorded in the database.
The previous patch 466266c tended to inherit the state of SYSTEMD_READY from the database and skip to the end of current rule file. But when the database does not contain SYSTEMD_READY, e.g., the dm-* is not set db_persistent during initrd and the database will be cleared after switching root, the following rules will still be applied not as expected.
This drops logs of failed unit tests in TEST-02-UNITTESTS from the journal,
as the expected logs from test-varlink-idl makes the post script fail.
Saving logs in journal is simply noisy, and we will output anyway after
qemu or nspawn finished by check_result_{qemu,nspawn}_unittests().
It's been there since the test was introduced and I'm not really sure
what was the original intention behind it, but it makes systemd sad:
[ 4.909056] systemd[1]: /usr/lib/systemd/tests/testdata/units/testsuite-44.service:13: Unknown key name 'LogTarget' in section 'Service', ignoring.
Turns out that redirecting a lot of output to the console can have some
funny effects, like random kernel soft lockups. I spotted this in
various CIs, but it remained almost entirely hidden thanks to
`softlockup_panic=1`, until 1a36d2672f which introduced a couple of
tests that log quite a lot in a short amount of time. This, in
combination with newer kernel version, which, for some reason, seem to
be more susceptible to such soft lockups, made the Arch Linux jobs soft
lockup quite a lot, see [0].
While debugging this I also noticed that runs which don't redirect
stdout/stderr to the console are noticeably faster, e.g.:
# TEST-71 nspawn + QEMU (KVM), StandardOutput=journal+console
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:24.64
# TEST-71 nspawn + QEMU (KVM), StandardOutput=journal
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:17.95
# TEST-71 nspawn + QEMU, StandardOutput=journal+console
Elapsed (wall clock) time (h:mm:ss or m:ss): 2:04.70
# TEST-71 nspawn + QEMU, StandardOutput=journal
Elapsed (wall clock) time (h:mm:ss or m:ss): 1:44.48
# TEST-04 QEMU, StandardOutput=journal+console
Elapsed (wall clock) time (h:mm:ss or m:ss): 4:22.70
# TEST-04 QEMU, StandardOutput=console
Elapsed (wall clock) time (h:mm:ss or m:ss): 5:04.67
Given all this, let's effectively revert ba7abf79a5, and dump the
testsuite-related journal messages only after the test finishes, so they
don't go through the slow console.
Resolves: systemd/systemd-centos-ci#660
[0] https://github.com/systemd/systemd-centos-ci/issues/660
Let's save all journals from the test machine instead of calling export
on each journal file separately, which makes the code less complicated
(and probably faster).
The logic is taken from dump ratelimit: if the config changes, we discard the
counters. This allows the user apply new limits and "start from scratch" in
that case.
This actually makes StartLimitIntervalSec=infinity (or with a large interval)
work as expected, because the counter is maintained even if daemon-reload
operations are interleaved.
We have not tested if the settings actually filter DHCP servers.
Let's add a test case for the settings.
Note, the .network file used here has been unused since
0730e3767d. So, we can freely reuse it
without changing other test cases.
Closes#30107.
I'm pretty sure this is not the only case, but it's the one I recently
noticed. Even though we call ddebug() from a function, that function is
called before ddebug() is defined, resulting in the same issue as if we
called just ddebug() in its place, i.e.:
..//test-functions: line 276: ddebug: command not found
