Given that gold is pretty much unmaintained and does not support
`-static-pie` for bootloader components it should be safe to drop.
Also switch to clang-17 while we're at it.
If the systemd version on the host is too old and there's no local
build directory, use the default tools tree which will build an
image containing all the tooling required to build systemd and use
that to build the other presets.
Although, this is CI, we can still do better. It also ensures that any
env var changes make it into the script, as things like PATH would not
survive a `sudo -E`.
Configuration now takes priority over CLI options so we have to
configure the defaults for settings that we want to allow overriding
from the CLI. We also explicitly set some other settings so that they
can't be overridden from the CLI anymore. For example the base and
initrd image should never be made bootable so we set Bootable=no
explicitly for both.
The mkosi Arch CI doesn't work as the keyring package is out
of date and cannot be built due to various build toolchain
issues. Disable the job as it always fails and confuses
submitters.
Now that we use meson feature options for our dependencies, we can just
rely on '--auto-features=disabled' to do the same. One benefit of this
is that specific features can still be force-enabled by overriding it
with the appropriate '-Dfeature=enabled' flag.
The two remaining uses for skip-deps can simply rely on their default
logic that sets the value to 'no' when the dependency is disabled.
By using meson features we can replace the handcrafted dependency
auto-detection by just passing the value from get_option directly to the
required arg for dependency, find_library etc.
'auto' features make the dependency optional, 'enabled' requires it
while 'disabled' features will skip detection entirely.
Any skipped or not found dependency will just be a no-op when passed to
build steps and therefore we can also skip the creation of empty vars.
The use of skip_deps for these is dropped here as meson provides a way
to disable all optional features in one go by passing
'-Dauto_features=disabled'.
This update introduces the explicit Dependencies= setting, instead
of relying on implicit dependencies via alphanumerical ordering.
We also take the opportunity to rename the "final" preset to the
"system" preset, which seems like a better name.
Exclude all `.in` files because they may contain unsupported syntax, and
they have to be preprocessed first. For example:
```sh
Error: SHELLCHECK_WARNING:
./src/rpm/systemd-update-helper.in:130:37: warning[SC1083]: This { is literal. Check expression (missing ;/\n?) or quote it.
```
Related to: https://github.com/systemd/systemd/pull/28521
To avoid apt complaining:
+ apt-get -y install clang-15 lldb-15 lld-15 clangd-15
Reading package lists...
Building dependency tree...
Reading state information...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
The following packages have unmet dependencies:
python3-lldb-14 : Conflicts: python3-lldb-x.y
python3-lldb-15 : Conflicts: python3-lldb-x.y
E: Error, pkgProblemResolver::Resolve generated breaks, this may be caused by held packages.
When the mkosi CI fails to boot, it just sits there waiting at the emergency
console until the job times out. Add a drop-in for emergency.service in the
CI configuration so that instead it exists immediately.
It's been a while since we introduced Differential ShellCheck and it
proved to be quite useful (and in some ways even better than the shellcheck
run by super-linter). So, to have only one linter scream at us for not
knowing how to write bash properly, let's drop the super-linter's one in
favor of Differential ShellCheck.
Follow-up for https://github.com/systemd/systemd/pull/24328#pullrequestreview-1074127504
mkosi now supports CentOS SIGs natively so we drop our own definition
of that and use the mkosi builtin one. We also enable hyperscale for
both CentOS 8 and CentOS 9 for consistency and add epel-next as well
which is a requirement for Hyperscale.
We now run repart before starting systemd-nspawn to make sure that
the root partition is also generated when we boot the image in a
container instead of a VM.
To make sure we start from scratch for both the container boot and
the VM boot, we also enable Ephemeral to make sure all changes to
the image are ephemeral.
vsock should work properly after the latest release of mkosi. But
to make sure it works, let's exit with 123 in case of success and
check for that in Github Actions.
Upload results from CIFuzz using SARIF.
This will allow CIFuzz to report issues in the security tab.
This is a better UI than having to look through logs.
TODO(google/oss-fuzz#10452): Add proper descriptions of UBSAN bugs.
We only really care about lowering the device timeout so we get to
a shell faster when the root device doesn't appear so let's only
lower that timeout instead of lowering all default timeouts.
Let's start moving towards a more involved partitioning setup to
test our stuff more when using mkosi.
The root partition is generated on boot with systemd-repart.
CentOS supports neither erofs nor btrfs so we use squashfs and xfs
instead.
We also enable SecureBoot= locally for additional coverage. This
and the use of verity means users need to run `mkosi genkey` once
to generate the keys necessary to do secure boot and verity.
Let's make sure we log more of what mkosi's doing so we can debug
issues better. Note this also makes mkosi set SYSTEMD_LOG_LEVEL=debug
when running programs so we'll get all the systemd debug logging as
well.
Instead of building the initrds for the mkosi images with dracut,
let's switch to using mkosi presets to build the initrd with mkosi
as well.
This commit splits up our single image build into three separate
mkosi presets:
1. The "base" preset. This image contains systemd and all its runtime
dependencies. The sole purpose of this image is to serve as a base image
for the initrd and the final image. It's also responsible for building
systemd from source with the build script. The results are installed into
the base image. Note that we install the systemd and udev packages into this
image as well to prevent package managers from overriding the systemd we built
from source with the distro packaged systemd if it's pulled in as a dependency
by another package from the initrd or final profiles.
2. The "initrd" preset. This image provides the initrd. It's trivial and does
nothing more than packaging the base image up as a zstd compressed initramfs and
adds /init and /etc/initrd-release symlinks to the image.
3. The "final" preset. This image builds on top of the base image and adds
a kernel and extra packages that are useful for testing and debugging.
We also split out the optional kernel build into a separate set of config files
that are only included if a kernel to build is actually provided.
Note that this commit doesn't really change anything about how mkosi is used.
The commands remain the same, except that mkosi will now build all the presets
in order. "mkosi summary" will show the summary of all the presets. "mkosi qemu,
boot, shell" will always boot the final preset. With "-f", all presets will be
built and the final one is booted. "-i" makes a cache of each preset.
The only thing to keep in mind is that specifying config via the mkosi CLI will
apply to each of the presets. e.g. any extra packages added with "-p" will be
installed in both the initrd and the final image. To apply local configuration
to a single preset, create a file 00-local.conf in
mkosi.presets/<profile>/mkosi.conf.d and put all the preset specific configuration
in there.
Let's use the new support for matching against any distribution in
a list of distributions to start sharing most things between the
ubuntu/debian configs and centos/fedora configs.
follow-up to https://github.com/systemd/systemd/pull/27071
in order to create Github Releases, the job needs permissions to write
contents
also:
- pinned the `softprops/action-gh-release` action to a specific commit
- made it only active on the `systemd` organization repos (so not on
forks)
This also migrates the configuration to the new format that was
just merged in mkosi. Specifically, we make use of the new [Match]
sections to only include specific config snippets per distro.
We have some missing coverage in the CI, all builds enable these features,
but there are often changes and they cover a lot of code. Do one build
without them to ensure we don't break builds.
make a github release for every tag that starts with `v*`,
and a pre-release if the tag contains "-rc".
on the 'systemd/systemd' repo, the "Release" will be draft, so that the
release manager can fill up the notes manually.
on 'systemd/systemd-stable' the release will be created immediately.
info about the action used:
https://github.com/softprops/action-gh-release
- Drop Netdev= as it was removed in mkosi
- Always install python-psutil in the final image (required for networkd tests)
- Always Install python-pytest in the final image (required for ukify tests)
- Use the narrow glob for all centos python packages
- Drop the networkd mkosi config files (the default image can be used instead)
- Use ".conf" as the mkosi config file suffix everywhere
- Copy src/ to /root/src in the final image and set gdb substitute path in
.gdbinit to make gdb work properly
- ACLs are not set on generated directories anymore by default, so
we enable them explictly now so that when running unprivileged mkosi,
the user running mkosi can remove all generated files and directories.
- We don't explicitly set QemuHeadless= anymore as the option was removed
and made the default.
- We set the loglevel= kernel cmdline argument explicitly now as mkosi
doesn't set it by default anymore.
This drops all mentions of gnu-efi and its manual build machinery. A
future commit will bring bootloader builds back. A new bootloader meson
option is now used to control whether to build sd-boot and its userspace
tooling.
"Failed to dissect image: connection timed out" messages have been
appearing sporadically in mkosi CI. Let's enable debug logging to
help figure out why.
Let's make sure we're testing unprivileged builds properly. Usage
of SourceFileTransfer= and SourceFileTransferFinal= are removed as
they were dropped by mkosi. SourceFileTransfer=mount is now the
default in mkosi so behavior for the build script is unchanged. We
stop copying sources in the final image until mkosi adds support
for virtiofs.
`github.event.issue.pull_request` is an object, not a boolean.
This is the root cause of why the step that is supposed to remove labels
is always skipped. Having this condition in place is not necessary since
the workflow is run on the `pull_request_target` event.
(The one case that is left unchanged is '< <(subcommand)'.)
This way, the style with no gap was already dominant. This way, the reader
immediately knows that ' < ' is a comparison operator and ' << ' is a shift.
In a few cases, replace custom EOF replacement by just EOF. There is no point
in using someting like "_EOL" unless "EOF" appears in the text.
Due to the limitation of `GITHUB_TOKEN` when running workflows from forks,
it's required to split the `development_freeze` workflow in two.
* First workflow will run on the `pull_request` trigger and save the PR
number in the artifact. This workflow is running with read-only permissions
on `GITHUB_TOKEN`.
* Second workflow will get triggered on `workflow_run`. It will be run
directly in the `systemd/systemd` context and can get permission to be
able to create comments on PR.
GITHUB_TOKEN limitations:
* https://docs.github.com/en/actions/security-guides/automatic-token-authentication#permissions-for-the-github_token
GitHub Security Labs Article - How to correctly and safely overcome GITHUB_TOKEN limitations:
* https://securitylab.github.com/research/github-actions-preventing-pwn-requests/
This gives us some nice test coverage for secure boot enrolling and the
stub secure boot workound. The authenticated EFI variables are already
created by mkosi, all we need to do is request secure boot to be used.
It's still useful to test the EFI handover logic in systemd-boot.
We use a mkosi.prepare script to install a newer python and update
the system to use it.
We also add the necessary deps for ukify to the mkosi configs.
CentOS Stream 8 is dropped from CI because its python version is too
old (3.6) to be able to run ukify.
Let's make sure we're testing with the latest changes in mkosi. This
includes both the switch to systemd-repart and ukify, making sure we
get extra testing coverage for those components.
This also drops options from the centos config that have been removed
in the newer mkosi.
For some reason idmapping runs into some issues so we disable it for
now.
Meson started warning when 'setup' is not used:
WARNING: Running the setup command as `meson [options]` instead of `meson setup [options]` is ambiguous and deprecated.
Also add more quoting in output to make the message clearer.
This erase_obj() machinery looks like voodoo and creates an awful lot of
noise as soon as we get back to building with -O0. We can do this in a
more simple way by introducing a struct that holds the information we
need on cleanup. When building with optimization enabled, all this gets
inlined and the eraser vanishes.
- Mention "/please-review" in the contributing guide
- Remove "needs-rebase" on push
- Don't add "please-review" if a green label is set
- Don't add please-review label to draft PRs
- Add please-review when a PR moves out of draft
When a pull request is opened/updated, add "please-review" and
remove a few other labels.
When a comment is made with /please-review on a PR. Add the
"please-review" label to the PR.
Rather than passing seeds up to userspace via EFI variables, pass seeds
directly to the kernel's EFI stub loader, via LINUX_EFI_RANDOM_SEED_TABLE_GUID.
EFI variables can potentially leak and suffer from forward secrecy
issues, and processing these with userspace means that they are
initialized much too late in boot to be useful. In contrast,
LINUX_EFI_RANDOM_SEED_TABLE_GUID uses EFI configuration tables, and so
is hidden from userspace entirely, and is parsed extremely early on by
the kernel, so that every single call to get_random_bytes() by the
kernel is seeded.
In order to do this properly, we use a bit more robust hashing scheme,
and make sure that each input is properly memzeroed out after use. The
scheme is:
key = HASH(LABEL || sizeof(input1) || input1 || ... || sizeof(inputN) || inputN)
new_disk_seed = HASH(key || 0)
seed_for_linux = HASH(key || 1)
The various inputs are:
- LINUX_EFI_RANDOM_SEED_TABLE_GUID from prior bootloaders
- 256 bits of seed from EFI's RNG
- The (immutable) system token, from its EFI variable
- The prior on-disk seed
- The UEFI monotonic counter
- A timestamp
This also adjusts the secure boot semantics, so that the operation is
only aborted if it's not possible to get random bytes from EFI's RNG or
a prior boot stage. With the proper hashing scheme, this should make
boot seeds safe even on secure boot.
There is currently a bug in Linux's EFI stub in which if the EFI stub
manages to generate random bytes on its own using EFI's RNG, it will
ignore what the bootloader passes. That's annoying, but it means that
either way, via systemd-boot or via EFI stub's mechanism, the RNG *does*
get initialized in a good safe way. And this bug is now fixed in the
efi.git tree, and will hopefully be backported to older kernels.
As the kernel recommends, the resultant seeds are 256 bits and are
allocated using pool memory of type EfiACPIReclaimMemory, so that it
gets freed at the right moment in boot.
https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#example-including-and-excluding-paths
> If you define a path with the ! character, you must also define at least one
> path without the ! character. If you only want to exclude paths, use
> paths-ignore instead.
>
> The order that you define patterns matters:
> A matching negative pattern (prefixed with !) after a positive match will
> exclude the path.
> A matching positive pattern after a negative match will include the path
> again.
Even if some of the exluded paths *could* impact the build, generally it's a
waste of time to do mkosi builds on them. Let's skip to releave the builders a
bit.
GitHub Action `devel-freezer` helps with development freeze notifications
during the RC phase. It will create comments using predefined messages on
newly created and updated PRs when the RC tag has been released.
Also, it will update comments once a new major version has been released.
Documentation available at: https://github.com/redhat-plumbers-in-action/devel-freezer
In the Xen case, it's the hypervisor which manages kexec. We thus
have to ask it whether a kernel is loaded, instead of relying on
/sys/kernel/kexec_loaded.
Also, unify the string quotation a bit and drop one unnecessary
expression syntax (as everything in `if` statements is automatically
evaluated as an expression).
* chore: enable scorecard action
* chore: add badge to the README file
* chore: enable on config file update
* chore: update scorecard to 2.0.4
* chore: run scorecard on PR at main branch
* chore: add condition to publish_result key
* chore: skip upload to code scanning if PR
* chore: only runs scorecard in the main repo
Resolves: #25042
Since LGTM is no longer enabled for the systemd repo (as it's going to
be discontinued by the EOY), let's run CodeQL on every PR instead to
replace it.
