Currently we spawn services by forking a child process, doing a bunch
of work, and then exec'ing the service executable.
There are some advantages to this approach:
- quick: we immediately have access to all the enourmous amount of
state simply by virtue of sharing the memory with the parent
- easy to refactor and add features
- part of the same binary, will never be out of sync
There are however significant drawbacks:
- doing work after fork and before exec is against glibc's supported
case for several APIs we call
- copy-on-write trap: anytime any memory is touched in either parent
or child, a copy of that page will be triggered
- memory footprint of the child process will be memory footprint of
PID1, but using the cgroup memory limits of the unit
The last issue is especially problematic on resource constrained
systems where hard memory caps are enforced and swap is not allowed.
As soon as PID1 is under load, with no page out due to no swap, and a
service with a low MemoryMax= tries to start, hilarity ensues.
Add a new systemd-executor binary, that is able to receive all the
required state via memfd, deserialize it, prepare the appropriate
data structures and call exec_child.
Use posix_spawn which uses CLONE_VM + CLONE_VFORK, to ensure there is
no copy-on-write (same address space will be used, and parent process
will be frozen, until exec).
The sd-executor binary is pinned by FD on startup, so that we can
guarantee there will be no incompatibilities during upgrades.
Let's mention that we just need the latest stable release of mkosi,
not the latest git commit. We also split the instructions for building
on the host and the instructions for building with mkosi into two blocks,
as it's not required to build on the host anymore to build with mkosi.
On normal systems, triggering a timeout should be a bug in code or
configuration error, so I do not think we should extend the default
timeout. Also, we should not introduce a 'first class' configuration
option about that. But, making it configurable may be useful for cases
such that "an extremely highly utilized system (lots of OOM kills,
very high CPU utilization, etc)".
Closes#25441.
The tool initially just measured the boot phase, but was subsequently
extended to measure file system and machine IDs, too. At AllSystemsGo
there were request to add more, and make the tool generically
accessible.
Hence, let's rename the binary (but not the pcrphase services), to make
clear the tool is not just measureing the boot phase, but a lot of other
things too.
The tool is located in /usr/lib/ and still relatively new, hence let's
just rename the binary and be done with it, while keeping the unit names
stable.
While we are at it, also move the tool out of src/boot/ and into its own
src/pcrextend/ dir, since it's not really doing boot related stuff
anymore.
ispell made some suggestions which I applied.
Addresses: https://github.com/systemd/systemd/pull/29209#pullrequestreview-1632623460
Also adds a brief paragraph about initrd transitions. (Plymouth really
should start using the fdstore for pinning DRM objects, and stop trying
to survive the initrd→host transition)
There were a couple spelling/grammatical errors in the docs that made
it hard to read and understand parts of this doc. I cleaned up those
errors and reflowed the line breaks to keep to the 80 char limit.
The article "a" goes before consonant sounds and "an" goes before vowel
sounds. This commit changes an to a for UKI, UDP, UTF-8, URL, UUID, U-Label, UI
and USB, since they start with the sound /ˌjuː/.
Since mkosi is now smart enough to drop the caches when the list of
packages changes, let's enable Incremental= mode by default to ensure
a good experience for anyone new to hacking on systemd with mkosi.
ImportCredential= takes a credential name and searches for a matching
credential in all the credential stores we know about it. It supports
globs which are expanded so that all matching credentials are loaded.
The kernel, systemd, and many other things print their version during boot.
sd-boot and sd-stub are also important, so let's print the version if EFI_DEBUG.
(If !EFI_DEBUG, continue to be quiet.)
When updating the docs, I saw that that the text in HACKING.md was out of date.
Instead of trying to update the instructions there, make it shorter and refer
the reader to tools/debug-sd-boot.sh for details.
Before 7cd43e34c5, it was possible to use
SYSTEMD_PROC_CMDLINE=systemd.condition-first-boot to override autodetection.
But now this doesn't work anymore, and it's useful to be able to do that for
testing.
We provide the same stability for all the headers that are public.
Also, mark id128 as portable to other systems. There is really nothing in the
code that would make it hard. It would probably work out-of-the-box.
Let's start moving towards a more involved partitioning setup to
test our stuff more when using mkosi.
The root partition is generated on boot with systemd-repart.
CentOS supports neither erofs nor btrfs so we use squashfs and xfs
instead.
We also enable SecureBoot= locally for additional coverage. This
and the use of verity means users need to run `mkosi genkey` once
to generate the keys necessary to do secure boot and verity.
This implements a minimal subset of #24961, but in a lot more
restrictive way: we only allow one level of subcgroup (as that's enough
to address the no-processes in inner cgroups rule), and does not change
anything about threaded cgroup logic or similar, or make any of this new
behaviour mandatory.
All this does is this: all non-control processes we invoke for a unit
we'll invoke in a subgroup by the specified name.
We'll later port all our current services that use cgroup delegation
over to this, i.e. user@.service, systemd-nspawn@.service and
systemd-udevd.service.
To make it consistent with other env vars, e.g. $SYSTEMD_ESP_PATH or
$SYSTEMD_XBOOTLDR_PATH.
This is useful when the root is specified by a file descriptor, instead
of a path.
Fixes RHBZ#2183546 (https://bugzilla.redhat.com/show_bug.cgi?id=2183546).
Previously, journal file is always compressed with the default algorithm
set at compile time. So, if a newer algorithm is used, journal files
cannot be read by older version of journalctl that does not support the
algorithm.
Co-authored-by: Colin Walters <walters@verbum.org>
- Drop Netdev= as it was removed in mkosi
- Always install python-psutil in the final image (required for networkd tests)
- Always Install python-pytest in the final image (required for ukify tests)
- Use the narrow glob for all centos python packages
- Drop the networkd mkosi config files (the default image can be used instead)
- Use ".conf" as the mkosi config file suffix everywhere
- Copy src/ to /root/src in the final image and set gdb substitute path in
.gdbinit to make gdb work properly
This is useful to identify log messages with metadata from the images
they run on. Look for ID/VERSION_ID/IMAGE_ID/IMAGE_VERSION/BUILD_ID,
with a SYSEXT_ prefix if we are looking at an extension, and append via
LogExtraFields= as respectively PORTABLE_NAME_AND_VERSION= in case of a
single image. In case of extensions, append as PORTABLE_ROOT_NAME_AND_VERSION=
for the base and one PORTABLE_EXTENSION_AND_VERSION= for each extension.
Example with a base and two extensions, with the unit coming from the
first extension:
[Service]
RootImage=/home/bluca/git/systemd/base.raw
Environment=PORTABLE=app0.raw
BindReadOnlyPaths=/etc/os-release:/run/host/os-release
LogExtraFields=PORTABLE=app0.raw
Environment=PORTABLE_ROOT=base.raw
LogExtraFields=PORTABLE_ROOT=base.raw
LogExtraFields=PORTABLE_ROOT_NAME_AND_VERSION=debian_10
ExtensionImages=/home/bluca/git/systemd/app0.raw
LogExtraFields=PORTABLE_EXTENSION=app0.raw
LogExtraFields=PORTABLE_EXTENSION_NAME_AND_VERSION=app_0
ExtensionImages=/home/bluca/git/systemd/app1.raw
LogExtraFields=PORTABLE_EXTENSION=app1.raw
LogExtraFields=PORTABLE_EXTENSION_NAME_AND_VERSION=app_1
When a portable service uses extensions, we use the 'main' image name
(the one where the unit was found in) as PORTABLE=. It is useful to
also list all the images actually used at runtime, as they might
contain libraries and so on.
Use PORTABLE_ROOT= for the image/directory that is used as RootImage=
or RootDirectory=, and PORTABLE_EXTENSION= for the image/directory that
is used as ExtensionImages= or ExtensionDirectories=.
Note that these new fields are only added if extensions are used,
there's no change for single-DDI portables.
Example with a base and two extensions, with the unit coming from the
first extension:
[Service]
RootImage=/home/bluca/git/systemd/base.raw
Environment=PORTABLE=app0.raw
BindReadOnlyPaths=/etc/os-release:/run/host/os-release
LogExtraFields=PORTABLE=app0.raw
LogExtraFields=PORTABLE_ROOT=base.raw
ExtensionImages=/home/bluca/git/systemd/app0.raw
LogExtraFields=PORTABLE_EXTENSION=app0.raw
ExtensionImages=/home/bluca/git/systemd/app1.raw
LogExtraFields=PORTABLE_EXTENSION=app1.raw
This way we can quickly find the most recent entry, without searching or
traversing entry array chains.
This is relevant later, as it it allows us to quickly determine the most
recent timestamps of each journal file, in a roughly atomic way.
(The one case that is left unchanged is '< <(subcommand)'.)
This way, the style with no gap was already dominant. This way, the reader
immediately knows that ' < ' is a comparison operator and ' << ' is a shift.
In a few cases, replace custom EOF replacement by just EOF. There is no point
in using someting like "_EOL" unless "EOF" appears in the text.
The documentation on moving an existing homedir into a systemd-homed managed
one suggests using rsync(1) with a bunch of flags to preserve as much metadata
as possible: permissions, xattrs, timestamps, etc. The previously suggested
flags were:
rsync -aHAXv --remove-source-files …
… which does include mtimes, but not ctimes and atimes, because -a does not
include those:
--archive, -a archive mode is -rlptgoD (no -A,-X,-U,-N,-H)
This change adds the -N and -U flags to preserve even more file timestamps,
turning the command into:
rsync -aHANUXv --remove-source-files …
The new flags are:
--crtimes, -N preserve create times (newness)
--atimes, -U preserve access (use) times
