Let's start moving towards a more involved partitioning setup to
test our stuff more when using mkosi.
The root partition is generated on boot with systemd-repart.
CentOS supports neither erofs nor btrfs so we use squashfs and xfs
instead.
We also enable SecureBoot= locally for additional coverage. This
and the use of verity means users need to run `mkosi genkey` once
to generate the keys necessary to do secure boot and verity.
Let's make sure we log more of what mkosi's doing so we can debug
issues better. Note this also makes mkosi set SYSTEMD_LOG_LEVEL=debug
when running programs so we'll get all the systemd debug logging as
well.
Instead of building the initrds for the mkosi images with dracut,
let's switch to using mkosi presets to build the initrd with mkosi
as well.
This commit splits up our single image build into three separate
mkosi presets:
1. The "base" preset. This image contains systemd and all its runtime
dependencies. The sole purpose of this image is to serve as a base image
for the initrd and the final image. It's also responsible for building
systemd from source with the build script. The results are installed into
the base image. Note that we install the systemd and udev packages into this
image as well to prevent package managers from overriding the systemd we built
from source with the distro packaged systemd if it's pulled in as a dependency
by another package from the initrd or final profiles.
2. The "initrd" preset. This image provides the initrd. It's trivial and does
nothing more than packaging the base image up as a zstd compressed initramfs and
adds /init and /etc/initrd-release symlinks to the image.
3. The "final" preset. This image builds on top of the base image and adds
a kernel and extra packages that are useful for testing and debugging.
We also split out the optional kernel build into a separate set of config files
that are only included if a kernel to build is actually provided.
Note that this commit doesn't really change anything about how mkosi is used.
The commands remain the same, except that mkosi will now build all the presets
in order. "mkosi summary" will show the summary of all the presets. "mkosi qemu,
boot, shell" will always boot the final preset. With "-f", all presets will be
built and the final one is booted. "-i" makes a cache of each preset.
The only thing to keep in mind is that specifying config via the mkosi CLI will
apply to each of the presets. e.g. any extra packages added with "-p" will be
installed in both the initrd and the final image. To apply local configuration
to a single preset, create a file 00-local.conf in
mkosi.presets/<profile>/mkosi.conf.d and put all the preset specific configuration
in there.
Let's use the new support for matching against any distribution in
a list of distributions to start sharing most things between the
ubuntu/debian configs and centos/fedora configs.
follow-up to https://github.com/systemd/systemd/pull/27071
in order to create Github Releases, the job needs permissions to write
contents
also:
- pinned the `softprops/action-gh-release` action to a specific commit
- made it only active on the `systemd` organization repos (so not on
forks)
This also migrates the configuration to the new format that was
just merged in mkosi. Specifically, we make use of the new [Match]
sections to only include specific config snippets per distro.
We have some missing coverage in the CI, all builds enable these features,
but there are often changes and they cover a lot of code. Do one build
without them to ensure we don't break builds.
make a github release for every tag that starts with `v*`,
and a pre-release if the tag contains "-rc".
on the 'systemd/systemd' repo, the "Release" will be draft, so that the
release manager can fill up the notes manually.
on 'systemd/systemd-stable' the release will be created immediately.
info about the action used:
https://github.com/softprops/action-gh-release
- Drop Netdev= as it was removed in mkosi
- Always install python-psutil in the final image (required for networkd tests)
- Always Install python-pytest in the final image (required for ukify tests)
- Use the narrow glob for all centos python packages
- Drop the networkd mkosi config files (the default image can be used instead)
- Use ".conf" as the mkosi config file suffix everywhere
- Copy src/ to /root/src in the final image and set gdb substitute path in
.gdbinit to make gdb work properly
- ACLs are not set on generated directories anymore by default, so
we enable them explictly now so that when running unprivileged mkosi,
the user running mkosi can remove all generated files and directories.
- We don't explicitly set QemuHeadless= anymore as the option was removed
and made the default.
- We set the loglevel= kernel cmdline argument explicitly now as mkosi
doesn't set it by default anymore.
This drops all mentions of gnu-efi and its manual build machinery. A
future commit will bring bootloader builds back. A new bootloader meson
option is now used to control whether to build sd-boot and its userspace
tooling.
"Failed to dissect image: connection timed out" messages have been
appearing sporadically in mkosi CI. Let's enable debug logging to
help figure out why.
Let's make sure we're testing unprivileged builds properly. Usage
of SourceFileTransfer= and SourceFileTransferFinal= are removed as
they were dropped by mkosi. SourceFileTransfer=mount is now the
default in mkosi so behavior for the build script is unchanged. We
stop copying sources in the final image until mkosi adds support
for virtiofs.
`github.event.issue.pull_request` is an object, not a boolean.
This is the root cause of why the step that is supposed to remove labels
is always skipped. Having this condition in place is not necessary since
the workflow is run on the `pull_request_target` event.
(The one case that is left unchanged is '< <(subcommand)'.)
This way, the style with no gap was already dominant. This way, the reader
immediately knows that ' < ' is a comparison operator and ' << ' is a shift.
In a few cases, replace custom EOF replacement by just EOF. There is no point
in using someting like "_EOL" unless "EOF" appears in the text.
Due to the limitation of `GITHUB_TOKEN` when running workflows from forks,
it's required to split the `development_freeze` workflow in two.
* First workflow will run on the `pull_request` trigger and save the PR
number in the artifact. This workflow is running with read-only permissions
on `GITHUB_TOKEN`.
* Second workflow will get triggered on `workflow_run`. It will be run
directly in the `systemd/systemd` context and can get permission to be
able to create comments on PR.
GITHUB_TOKEN limitations:
* https://docs.github.com/en/actions/security-guides/automatic-token-authentication#permissions-for-the-github_token
GitHub Security Labs Article - How to correctly and safely overcome GITHUB_TOKEN limitations:
* https://securitylab.github.com/research/github-actions-preventing-pwn-requests/
This gives us some nice test coverage for secure boot enrolling and the
stub secure boot workound. The authenticated EFI variables are already
created by mkosi, all we need to do is request secure boot to be used.
It's still useful to test the EFI handover logic in systemd-boot.
We use a mkosi.prepare script to install a newer python and update
the system to use it.
We also add the necessary deps for ukify to the mkosi configs.
CentOS Stream 8 is dropped from CI because its python version is too
old (3.6) to be able to run ukify.
Let's make sure we're testing with the latest changes in mkosi. This
includes both the switch to systemd-repart and ukify, making sure we
get extra testing coverage for those components.
This also drops options from the centos config that have been removed
in the newer mkosi.
For some reason idmapping runs into some issues so we disable it for
now.
Meson started warning when 'setup' is not used:
WARNING: Running the setup command as `meson [options]` instead of `meson setup [options]` is ambiguous and deprecated.
Also add more quoting in output to make the message clearer.
This erase_obj() machinery looks like voodoo and creates an awful lot of
noise as soon as we get back to building with -O0. We can do this in a
more simple way by introducing a struct that holds the information we
need on cleanup. When building with optimization enabled, all this gets
inlined and the eraser vanishes.
- Mention "/please-review" in the contributing guide
- Remove "needs-rebase" on push
- Don't add "please-review" if a green label is set
- Don't add please-review label to draft PRs
- Add please-review when a PR moves out of draft
When a pull request is opened/updated, add "please-review" and
remove a few other labels.
When a comment is made with /please-review on a PR. Add the
"please-review" label to the PR.
Rather than passing seeds up to userspace via EFI variables, pass seeds
directly to the kernel's EFI stub loader, via LINUX_EFI_RANDOM_SEED_TABLE_GUID.
EFI variables can potentially leak and suffer from forward secrecy
issues, and processing these with userspace means that they are
initialized much too late in boot to be useful. In contrast,
LINUX_EFI_RANDOM_SEED_TABLE_GUID uses EFI configuration tables, and so
is hidden from userspace entirely, and is parsed extremely early on by
the kernel, so that every single call to get_random_bytes() by the
kernel is seeded.
In order to do this properly, we use a bit more robust hashing scheme,
and make sure that each input is properly memzeroed out after use. The
scheme is:
key = HASH(LABEL || sizeof(input1) || input1 || ... || sizeof(inputN) || inputN)
new_disk_seed = HASH(key || 0)
seed_for_linux = HASH(key || 1)
The various inputs are:
- LINUX_EFI_RANDOM_SEED_TABLE_GUID from prior bootloaders
- 256 bits of seed from EFI's RNG
- The (immutable) system token, from its EFI variable
- The prior on-disk seed
- The UEFI monotonic counter
- A timestamp
This also adjusts the secure boot semantics, so that the operation is
only aborted if it's not possible to get random bytes from EFI's RNG or
a prior boot stage. With the proper hashing scheme, this should make
boot seeds safe even on secure boot.
There is currently a bug in Linux's EFI stub in which if the EFI stub
manages to generate random bytes on its own using EFI's RNG, it will
ignore what the bootloader passes. That's annoying, but it means that
either way, via systemd-boot or via EFI stub's mechanism, the RNG *does*
get initialized in a good safe way. And this bug is now fixed in the
efi.git tree, and will hopefully be backported to older kernels.
As the kernel recommends, the resultant seeds are 256 bits and are
allocated using pool memory of type EfiACPIReclaimMemory, so that it
gets freed at the right moment in boot.
https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#example-including-and-excluding-paths
> If you define a path with the ! character, you must also define at least one
> path without the ! character. If you only want to exclude paths, use
> paths-ignore instead.
>
> The order that you define patterns matters:
> A matching negative pattern (prefixed with !) after a positive match will
> exclude the path.
> A matching positive pattern after a negative match will include the path
> again.
Even if some of the exluded paths *could* impact the build, generally it's a
waste of time to do mkosi builds on them. Let's skip to releave the builders a
bit.
GitHub Action `devel-freezer` helps with development freeze notifications
during the RC phase. It will create comments using predefined messages on
newly created and updated PRs when the RC tag has been released.
Also, it will update comments once a new major version has been released.
Documentation available at: https://github.com/redhat-plumbers-in-action/devel-freezer
In the Xen case, it's the hypervisor which manages kexec. We thus
have to ask it whether a kernel is loaded, instead of relying on
/sys/kernel/kexec_loaded.
Also, unify the string quotation a bit and drop one unnecessary
expression syntax (as everything in `if` statements is automatically
evaluated as an expression).
* chore: enable scorecard action
* chore: add badge to the README file
* chore: enable on config file update
* chore: update scorecard to 2.0.4
* chore: run scorecard on PR at main branch
* chore: add condition to publish_result key
* chore: skip upload to code scanning if PR
* chore: only runs scorecard in the main repo
Resolves: #25042
Since LGTM is no longer enabled for the systemd repo (as it's going to
be discontinued by the EOY), let's run CodeQL on every PR instead to
replace it.
It's still failing in Ubuntu:
● isc-dhcp-server.service loaded failed failed ISC DHCP IPv4 server
● isc-dhcp-server6.service loaded failed failed ISC DHCP IPv6 server
Just disable them via the kernel command line masking.
Explicitly enable all features/components in the mkosi build to
ensure they all get built and we get an error if they can't be built.
We also rework the packages sections of all mkosi configs to reduce
duplication and cover all the dependencies necessary to build/use all
systemd features.
Note that for the final image, since systemd is installed by default
in base images, we rely on that to install the base library dependencies
and we only list extra optional dependencies and tools that aren't already
installed by default into the base image.
We also drop the centos stream 8 mkosi build as dependencies on that
distro are too out-of-date to be able to build all systemd features.
Since centos stream 9 has been out for a while, let's focus on that
and leave it to downstream to keep systemd building on centos stream 8.
Finally, there's a few additions to the mkosi scripts to make sure
services don't start by default on boot.
The warning isn't that serious and mostly there to inform the user that
lld/mold cannot build efi binaries. It is also better to build test with
fatal meson warnings.
