docs: add Manuals and Documentation for Users and Administrators

This commit is contained in:
hulkoba 2024-01-29 12:55:10 +01:00
parent 163e2c8346
commit 6b2a277624
No known key found for this signature in database
GPG key ID: ACB6C4A3A4F2BE2F
9 changed files with 813 additions and 1 deletions

53
docs/API_FILE_SYSTEMS.md Normal file
View file

@ -0,0 +1,53 @@
---
title: API File Systems
category: Manuals and Documentation for Users and Administrators
layout: default
SPDX-License-Identifier: LGPL-2.1-or-later
---
# API File Systems
_So you are seeing all kinds of weird file systems in the output of mount(8) that are not listed in `/etc/fstab`, and you wonder what those are, how you can get rid of them, or at least change their mount options._
The Linux kernel provides a number of different ways for userspace to communicate with it. For many facilities there are system calls, others are hidden behind Netlink interfaces, and even others are exposed via virtual file systems such as `/proc` or `/sys`. These file systems are programming interfaces, they are not actually backed by real, persistent storage. They simply use the file system interface of the kernel as interface to various unrelated mechanisms. Similarly, there are file systems that userspace uses for its own API purposes, to store shared memory segments, shared temporary files or sockets. In this article we want to discuss all these kind of _API file systems_. More specifically, here's a list of these file systems typical Linux systems currently have:
* `/sys` for exposing kernel devices, drivers and other kernel information to userspace
* `/proc` for exposing kernel settings, processes and other kernel information to userspace
* `/dev` for exposing kernel device nodes to userspace
* `/run` as location for userspace sockets and files
* `/tmp` as location for volatile, temporary userspace file system objects (X)
* `/sys/fs/cgroup` (and file systems below that) for exposing the kernel control group hierarchy
* `/sys/kernel/security`, `/sys/kernel/debug` (X), `/sys/kernel/config` (X) for exposing special purpose kernel objects to userspace
* `/sys/fs/selinux` for exposing SELinux security data to userspace
* `/dev/shm` as location for userspace shared memory objects
* `/dev/pts` for exposing kernel pseudo TTY device nodes to userspace
* `/proc/sys/fs/binfmt_misc` for registering additional binary formats in the kernel (X)
* `/dev/mqueue` for exposing mqueue IPC objects to userspace (X)
* `/dev/hugepages` as a userspace API for allocating "huge" memory pages (X)
* `/sys/fs/fuse/connections` for exposing kernel FUSE connections to userspace (X)
* `/sys/firmware/efi/efivars` for exposing firmware variables to userspace
All these _API file systems_ are mounted during very early boot-up of systemd and are generally not listed in `/etc/fstab`. Depending on the used kernel configuration some of these API file systems might not be available and others might exist instead. As these interfaces are important for kernel-to-userspace and userspace-to-userspace communication they are mounted automatically and without configuration or interference by the user. Disabling or changing their parameters might hence result in applications breaking as they can no longer access the interfaces they need.
Even though the default settings of these file systems should normally be suitable for most setups, in some cases it might make sense to change the mount options, or possibly even disable some of these file systems.
Even though normally none of these API file systems are listed in `/etc/fstab` they may be added there. If so, any options specified therein will be applied to that specific API file system. Hence: to alter the mount options or other parameters of these file systems, simply add them to `/etc/fstab` with the appropriate settings and you are done. Using this technique it is possible to change the source, type of a file system in addition to simply changing mount options. That is useful to turn `/tmp` to a true file system backed by a physical disk.
It is possible to disable the automatic mounting of some (but not all) of these file systems, if that is required. These are marked with (X) in the list above. You may disable them simply by masking them:
```sh
systemctl mask dev-hugepages.mount
```
This has the effect that the huge memory page API FS is not mounted by default, starting with the next boot. See [Three Levels of Off](http://0pointer.de/blog/projects/three-levels-of-off.html) for more information on masking.
The systemd service [systemd-remount-fs.service](http://www.freedesktop.org/software/systemd/man/systemd-remount-fs.service.html) is responsible for applying mount parameters from `/etc/fstab` to the actual mounts.
## Why are you telling me all this? I just want to get rid of the tmpfs backed /tmp!
You have three options:
1. Disable any mounting on `/tmp` so that it resides on the same physical file system as the root directory. For that, execute `systemctl mask tmp.mount`
2. Mount a different, physical file system to `/tmp`. For that, simply create an entry for it in `/etc/fstab` as you would do for any other file system.
3. Keep `/tmp` but increase/decrease the size of it. For that, also just create an entry for it in `/etc/fstab` as you would do for any other `tmpfs` file system, and use the right `size=` option.

View file

@ -0,0 +1,123 @@
---
title: Socket Activation with Popular Daemons
category: Manuals and Documentation for Users and Administrators
layout: default
SPDX-License-Identifier: LGPL-2.1-or-later
---
## nginx
nginx includes an undocumented, internal socket-passing mechanism based on the `NGINX` environmental variable. It uses this to perform reloads without having to close and reopen its sockets, but it's also useful for socket activation.
**/etc/nginx/my-nginx.conf**
```
http {
server {
listen [::]:80 ipv6only=on;
listen 80;
}
}
```
**/etc/systemd/system/my-nginx.service**
```
[Service]
User=nginx
Group=nginx
Environment=NGINX=3:4;
ExecStart=/usr/sbin/nginx -c/etc/nginx/my-nginx.conf
PrivateNetwork=true
```
**/etc/systemd/system/my-nginx.socket**
```
[Socket]
ListenStream=80
ListenStream=0.0.0.0:80
BindIPv6Only=ipv6-only
After=network.target
Requires=network.target
[Install]
WantedBy=sockets.target
```
## PHP-FPM
Like nginx, PHP-FPM includes a socket-passing mechanism an environmental variable. In PHP-FPM's case, it's `FPM_SOCKETS`.
This configuration is possible with any web server that supports FastCGI (like Apache, Lighttpd, or nginx). The web server does not need to know anything special about the socket; use a normal PHP-FPM configuration.
Paths are based on a Fedora 19 system.
### First, the configuration files
**/etc/php-fpm.d/my-php-fpm-pool.conf**
```
[global]
pid = /run/my-php-fpm-pool.pid ; Not really used by anything with daemonize = no, but needs to be writable.
error_log = syslog ; Will aggregate to the service's systemd journal.
daemonize = no ; systemd handles the forking.
[www]
listen = /var/run/my-php-fpm-pool.socket ; Must match systemd socket unit.
user = nginx ; Ignored but required.
group = nginx ; Ignored but required.
pm = static
pm.max_children = 10
slowlog = syslog
```
**/etc/systemd/system/my-php-fpm-pool.service**
```
[Service]
User=nginx
Group=nginx
Environment="FPM_SOCKETS=/var/run/my-php-fpm-pool.socket=3"
ExecStart=/usr/sbin/php-fpm --fpm-config=/etc/php-fpm.d/my-php-fpm-pool.conf
KillMode=process
```
**/etc/systemd/system/my-php-fpm-pool.socket**
```
[Socket]
ListenStream=/var/run/my-php-fpm-pool.socket
[Install]
WantedBy=sockets.target
```
### Second, the setup commands
```sh
sudo systemctl --system daemon-reload
sudo systemctl start my-php-fpm-pool.socket
sudo systemctl enable my-php-fpm-pool.socket
```
After accessing the web server, the service should be running.
```sh
sudo systemctl status my-php-fpm-pool.service
```
It's possible to shut down the service and re-activate it using the web browser, too. It's necessary to stop and start the socket to reset some shutdown PHP-FPM does that otherwise breaks reactivation.
```sh
sudo systemctl stop my-php-fpm-pool.socket my-php-fpm-pool.service
sudo systemctl start my-php-fpm-pool.socket
```

212
docs/DEBUGGING.md Normal file
View file

@ -0,0 +1,212 @@
---
title: Diagnosing Boot Problems
category: Manuals and Documentation for Users and Administrators
layout: default
SPDX-License-Identifier: LGPL-2.1-or-later
---
# Diagnosing Boot Problems
If your machine gets stuck during boot, first check if the hang happens before or after control passes to systemd.
Try to boot without `rhgb` and `quiet` on the kernel command line. If you see some messages like these:
* Welcome to Fedora _VERSION_ (_codename_)!"
* Starting _name_...
* \[ OK \] Started _name_.
then systemd is running. (See an actual [screenshot](f17boot.png).)
Debugging always gets easier if you can get a shell. If you do not get a login prompt, try switching to a different virtual terminal using CTRL+ALT+F\_\_. Problems with a display server startup may manifest themselves as a missing login on tty1, but other VTs working.
If the boot stops without presenting you with a login on any virtual console, let it retry for _up to 5 minutes_ before declaring it definitely stuck. There is a chance that a service that has trouble starting will be killed after this timeout and the boot will continue normally. Another possibility is that a device for an important mountpoint will fail to appear and you will be presented with _emergency mode_.
## If You Get No Shell
If you get neither a normal login nor the emergency mode shell, you will need to do additional steps to get debugging information out of the machine.
* Try CTRL+ALT+DEL to reboot.
* If it does not reboot, mention it in your bugreport. Meanwhile force the reboot with [SysRq](http://fedoraproject.org/wiki/QA/Sysrq) or hard reset.
* When booting the next time, you will have to add some kernel command line arguments depending on which of the debugging strategies you choose from the following options.
### Debug Logging to a Serial Console
If you have a hardware serial console available or if you are debugging in a virtual machine (e.g. using virt-manager you can switch your view to a serial console in the menu View -> Text Consoles or connect from the terminal using `virsh console MACHINE`), you can ask systemd to log lots of useful debugging information to it by booting with:
```sh
systemd.log_level=debug systemd.log_target=console console=ttyS0,38400 console=tty1
```
The above is useful if pid 1 is failing, but if a later but critical boot service is broken (such as networking), you can configure journald to forward to the console by using:
```sh
systemd.journald.forward_to_console=1 console=ttyS0,38400 console=tty1
```
console= can be specified multiple times, systemd will output to all of them.
### Booting into Rescue or Emergency Targets
To boot directly into rescue target add `systemd.unit=rescue.target` or just `1` to the kernel command line. This target is useful if the problem occurs somewhere after the basic system is brought up, during the starting of "normal" services. If this is the case, you should be able to disable the bad service from here. If the rescue target will not boot either, the more minimal emergency target might.
To boot directly into emergency shell add `systemd.unit=emergency.target` or `emergency` to the kernel command line. Note that in the emergency shell you will have to remount the root filesystem read-write by yourself before editing any files:
```sh
mount -o remount,rw /
```
Common issues that can be resolved in the emergency shell are bad lines in **/etc/fstab**. After fixing **/etc/fstab**, run `systemctl daemon-reload` to let systemd refresh its view of it.
If not even the emergency target works, you can boot directly into a shell with `init=/bin/sh`. This may be necessary in case systemd itself or some libraries it depends on are damaged by filesystem corruption. You may need to reinstall working versions of the affected packages.
If `init=/bin/sh` does not work, you must boot from another medium.
### Early Debug Shell
You can enable shell access to be available very early in the startup process to fall back on and diagnose systemd related boot up issues with various systemctl commands. Enable it using:
```sh
systemctl enable debug-shell.service
```
or by specifying
```sh
systemd.debug-shell=1
```
on the kernel command line.
**Tip**: If you find yourself in a situation where you cannot use systemctl to communicate with a running systemd (e.g. when setting this up from a different booted system), you can avoid communication with the manager by specifying `--root=`:
```sh
systemctl --root=/ enable debug-shell.service
```
Once enabled, the next time you boot you will be able to switch to tty9 using CTRL+ALT+F9 and have a root shell there available from an early point in the booting process. You can use the shell for checking the status of services, reading logs, looking for stuck jobs with `systemctl list-jobs`, etc.
**Warning:** Use this shell only for debugging! Do not forget to disable systemd-debug-shell.service after you've finished debugging your boot problems. Leaving the root shell always available would be a security risk.
It is also possible to alias `kbrequest.target` to `debug-shell.service` to start the debug shell on demand. This has the same security implications, but avoids running the shell always.
### verify prerequisites
A (at least partly) populated `/dev` is required. Depending on your setup (e.g. on embedded systems), check that the Linux kernel config options `CONFIG_DEVTMPFS` and `CONFIG_DEVTMPFS_MOUNT` are set. Also support for cgroups and fanotify is recommended for a flawless operation, so check that the Linux kernel config options `CONFIG_CGROUPS` and `CONFIG_FANOTIFY` are set. The message "Failed to get D-Bus connection: No connection to service manager." during various `systemctl` operations is an indicator that these are missing.
## If You Can Get a Shell
When you have systemd running to the extent that it can provide you with a shell, please use it to extract useful information for debugging. Boot with these parameters on the kernel command line:
```sh
systemd.log_level=debug systemd.log_target=kmsg log_buf_len=1M printk.devkmsg=on
```
in order to increase the verbosity of systemd, to let systemd write its logs to the kernel log buffer, to increase the size of the kernel log buffer, and to prevent the kernel from discarding messages. After reaching the shell, look at the log:
```sh
journalctl -b
```
When reporting a bug, pipe that to a file and attach it to the bug report.
To check for possibly stuck jobs use:
```sh
systemctl list-jobs
```
The jobs that are listed as "running" are the ones that must complete before the "waiting" ones will be allowed to start executing.
# Diagnosing Shutdown Problems
Just like with boot problems, when you encounter a hang during shutting down, make sure you wait _at least 5 minutes_ to distinguish a permanent hang from a broken service that's just timing out. Then it's worth testing whether the system reacts to CTRL+ALT+DEL in any way.
If shutdown (whether it be to reboot or power-off) of your system gets stuck, first test if the kernel itself is able to reboot or power-off the machine forcedly using one of these commands:
```sh
reboot -f
poweroff -f
```
If either one of the commands does not work, it's more likely to be a kernel, not systemd bug.
## Shutdown Completes Eventually
If normal reboot or poweroff work, but take a suspiciously long time, then
* boot with the debug options:
```sh
systemd.log_level=debug systemd.log_target=kmsg log_buf_len=1M printk.devkmsg=on enforcing=0
```
* save the following script as **/usr/lib/systemd/system-shutdown/debug.sh** and make it executable:
```sh
#!/bin/sh
mount -o remount,rw /
dmesg > /shutdown-log.txt
mount -o remount,ro /
```
* reboot
Look for timeouts logged in the resulting file **shutdown-log.txt** and/or attach it to a bugreport.
## Shutdown Never Finishes
If normal reboot or poweroff never finish even after waiting a few minutes, the above method to create the shutdown log will not help and the log must be obtained using other methods. Two options that are useful for debugging boot problems can be used also for shutdown problems:
* use a serial console
* use a debug shell - not only is it available from early boot, it also stays active until late shutdown.
# Status and Logs of Services
When the start of a service fails, systemctl will give you a generic error message:
```sh
# systemctl start foo.service
Job failed. See system journal and 'systemctl status' for details.
```
The service may have printed its own error message, but you do not see it, because services run by systemd are not related to your login session and their outputs are not connected to your terminal. That does not mean the output is lost though. By default the stdout, stderr of services are directed to the systemd _journal_ and the logs that services produce via `syslog(3)` go there too. systemd also stores the exit code of failed services. Let's check:
```sh
# systemctl status foo.service
foo.service - mmm service
Loaded: loaded (/etc/systemd/system/foo.service; static)
Active: failed (Result: exit-code) since Fri, 11 May 2012 20:26:23 +0200; 4s ago
Process: 1329 ExecStart=/usr/local/bin/foo (code=exited, status=1/FAILURE)
CGroup: name=systemd:/system/foo.service
May 11 20:26:23 scratch foo[1329]: Failed to parse config
```
In this example the service ran as a process with PID 1329 and exited with error code 1. If you run systemctl status as root or as a user from the `adm` group, you will get a few lines from the journal that the service wrote. In the example the service produced just one error message.
To list the journal, use the `journalctl` command.
If you have a syslog service (such as rsyslog) running, the journal will also forward the messages to it, so you'll find them in **/var/log/messages** (depending on rsyslog's configuration).
# Reporting systemd Bugs
Be prepared to include some information (logs) about your system as well. These should be complete (no snippets please), not in an archive, uncompressed.
Please report bugs to your distribution's bug tracker first. If you are sure that you are encountering an upstream bug, then first check [for existing bug reports](https://github.com/systemd/systemd/issues/), and if your issue is not listed [file a new bug](https://github.com/systemd/systemd/issues/new).
## Information to Attach to a Bug Report
Whenever possible, the following should be mentioned and attached to your bug report:
* The exact kernel command-line used. Typically from the bootloader configuration file (e.g. **/boot/grub2/grub.cfg**) or from **/proc/cmdline**
* The journal (the output of `journalctl -b > journal.txt`)
* ideally after booting with `systemd.log_level=debug systemd.log_target=kmsg log_buf_len=1M printk.devkmsg=on`
* The output of a systemd dump: `systemd-analyze dump > systemd-dump.txt`
* The output of `/usr/lib/systemd/systemd --test --system --log-level=debug > systemd-test.txt 2>&1`

115
docs/FAQ.md Normal file
View file

@ -0,0 +1,115 @@
---
title: Frequently Asked Questions
category: Manuals and Documentation for Users and Administrators
layout: default
SPDX-License-Identifier: LGPL-2.1-or-later
---
# Frequently Asked Questions
Also check out the [Tips & Tricks](../TIPS_AND_TRICKS)!
**Q: How do I change the current runlevel?**
A: In systemd runlevels are exposed via "target units". You can change them like this:
```sh
# systemctl isolate runlevel5.target
```
Note however, that the concept of runlevels is a bit out of date, and it is usually nicer to use modern names for this. e.g.:
```sh
# systemctl isolate graphical.target
```
This will only change the current runlevel, and has no effect on the next boot.
**Q: How do I change the default runlevel to boot into?**
A: The symlink /etc/systemd/system/default.target controls where we boot into by default. Link it to the target unit of your choice. For example, like this:
```sh
# ln -sf /usr/lib/systemd/system/multi-user.target /etc/systemd/system/default.target
```
or
```sh
# ln -sf /usr/lib/systemd/system/graphical.target /etc/systemd/system/default.target
```
**Q: How do I figure out the current runlevel?**
A: Note that there might be more than one target active at the same time. So the question regarding _the_ runlevel might not always make sense. Here's how you would figure out all targets that are currently active:
```sh
$ systemctl list-units --type=target
```
If you are just interested in a single number, you can use the venerable _runlevel_ command, but again, its output might be misleading.
**Q: I want to change a service file, but rpm keeps overwriting it in /usr/lib/systemd/system all the time, how should I handle this?**
A: The recommended way is to copy the service file from /usr/lib/systemd/system to /etc/systemd/system and edit it there. The latter directory takes precedence over the former, and rpm will never overwrite it. If you want to use the distributed service file again you can simply delete (or rename) the service file in /etc/systemd/system again.
**Q: My service foo.service as distributed by my operating system vendor is only started when (a connection comes in or some hardware is plugged in). I want to have it started always on boot, too. What should I do?**
A: Simply place a symlink from that service file in the multi-user.target.wants/ directory (which is where you should symlink everything you want to run in the old runlevel 3, i.e. the normal boot-up without graphical UI. It is pulled in by graphical.target too, so will be started for graphical boot-ups, too):
```sh
# ln -sf /usr/lib/systemd/system/foobar.service /etc/systemd/system/multi-user.target.wants/foobar.service
# systemctl daemon-reload
```
**Q: I want to enable another getty, how would I do that?**
A: Simply instantiate a new getty service for the port of your choice (internally, this places another symlink for instantiating another serial getty in the getty.target.wants/ directory).
```sh
# systemctl enable serial-getty@ttyS2.service
# systemctl start serial-getty@ttyS2.service
```
Note that gettys on the virtual console are started on demand. You can control how many you get via the NAutoVTs= setting in [logind.conf(7)](http://www.freedesktop.org/software/systemd/man/logind.html). Also see [this blog story](http://0pointer.de/blog/projects/serial-console.html).
**Q: How to I figure out which service a process belongs to?**
A: You may either use ps for that:
```sh
$ alias psc='ps xawf -eo pid,user,cgroup,args'
$ psc
...
```
Or you can even check /proc/$PID/cgroup directly. Also see [this blog story](http://0pointer.de/blog/projects/systemd-for-admins-2.html).
**Q: Why don't you use inotify to reload the unit files automatically on change?**
A: Unfortunately that would be a racy operation. For an explanation why and how we tried to improve the situation, see [the bugzilla report about this](https://bugzilla.redhat.com/show_bug.cgi?id=615527).
**Q: I have a native systemd service file and a SysV init script installed which share the same basename, e.g. /usr/lib/systemd/system/foobar.service vs. /etc/init.d/foobar -- which one wins?**
A: If both files are available the native unit file always takes precedence and the SysV init script is ignored, regardless whether either is enabled or disabled. Note that a SysV service that is enabled but overridden by a native service does not have the effect that the native service would be enabled, too. Enabling of native and SysV services is completely independent. Or in other words: you cannot enable a native service by enabling a SysV service by the same name, and if a SysV service is enabled but the respective native service is not, this will not have the effect that the SysV script is executed.
**Q: How can I use journalctl to display full (= not truncated) messages even if less is not used?**
A: Use:
```sh
# journalctl --full
```
**Q: Whenever my service tries to acquire RT scheduling for one of its threads this is refused with EPERM even though my service is running with full privileges. This works fine on my non-systemd system!**
A: By default, systemd places all systemd daemons in their own cgroup in the "cpu" hierarchy. Unfortunately, due to a kernel limitation, this has the effect of disallowing RT entirely for the service. See [My Service Can't Get Realtime!](../MY_SERVICE_CANT_GET_REATLIME) for a longer discussion and what to do about this.
**Q: My service is ordered after `network.target` but at boot it is still called before the network is up. What's going on?**
A: That's a long story, and that's why we have a wiki page of its own about this: [Running Services After the Network is up](../NETWORK_ONLINE)
**Q: My systemd system always comes up with `/tmp` as a tiny `tmpfs`. How do I get rid of this?**
A: That's also a long story, please have a look on [API File Systems](../API_FILE_SYSTEMS)

33
docs/INCOMPATIBILITIES.md Normal file
View file

@ -0,0 +1,33 @@
---
title: Compatibility with SysV
category: Manuals and Documentation for Users and Administrators
layout: default
SPDX-License-Identifier: LGPL-2.1-or-later
---
# Compatibility with SysV
systemd provides a fair degree of compatibility with the behavior exposed by the SysV init system as implemented by many distributions. Compatibility is provided both for the user experience and the SysV scripting APIs. However, there are some areas where compatibility is limited due to technical reasons or design decisions of systemd and the distributions. All of the following applies to SysV init scripts handled by systemd, however a number of them matter only on specific distributions. Many of the incompatibilities are specific to distribution-specific extensions of LSB/SysV init.
* If your distribution removes SysV init scripts in favor of systemd unit files typing "/etc/init.d/foobar start" to start a service will not work, since the script will not be available. Use the more correct "/sbin/service foobar start" instead, and your command will be forwarded to systemd. Note that invoking the init script directly has always been suboptimal since too much of the caller's execution context (environment block, umask, resource limits, audit trails, ...) ended up being inherited by the service, and invocation via "/sbin/service" used to clean this up at least partially. Invocation via /sbin/service works on both SysV and systemd systems. Also, LSB only standardizes invocation via "/sbin/service" anyway. (Note that some distributions ship both systemd unit files and SysV scripts for the services. For these invoking the init scripts will work as expected and the request be forwarded to systemd in any case.)
* LSB header dependency information matters. The SysV implementations on many distributions did not use the dependency information encoded in LSB init script headers, or used them only in very limited ways. Due to that they are often incorrect or incomplete. systemd however fully interprets these headers and follows them closely at runtime (and not at installation time like some implementations).
* Timeouts apply to all init script operations in systemd. While on SysV systems a hanging init script could freeze the system on systemd all init script operations are subject to a timeout of 5min.
* Services are executed in completely clean execution contexts, no context of the invoking user session is inherited. Not even $HOME or similar are set. Init scripts depending on these will not work correctly.
* Services cannot read from stdin, as this will be connected to /dev/null. That means interactive init scripts are not supported (i.e. Debian's X-Interactive in the LSB header is not supported either.) Thankfully most distributions do not support interaction in init scripts anyway. If you need interaction to ask disk or SSL passphrases please consider using the minimal password querying framework systemd supports. ([details](../PASSWORD_AGENTS), [manual page](http://0pointer.de/public/systemd-man/systemd-ask-password.html))
* Additional verbs for init scripts are not supported. If your init script traditionally supported additional verbs for your init script simply move them to an auxiliary script.
* Additional parameters to the standard verbs (i.e. to "start", "stop" and "status") are not supported. This was an extension of SysV that never was standardized officially, and is not supported in systemd.
* Overriding the "restart" verb is not supported. This verb is always implemented by systemd itself, and consists of a "stop" followed by a "start".
* systemd only stops running services. On traditional SysV a K link installed for shutdown was executed when going down regardless whether the service was started before or not. systemd is more strict here and does not stop service that weren't started in the first place.
* Note that neither S nor K links for runlevels 0 and 6 have any effect. Running services will be terminated anyway when shutting down, and no new SysV services are started at shut down.
* If systemd doesn't know which PID is the main PID of a service, it will not be able to track its runtime, and hence a service exiting on its own will not make systemd consider it stopped. Use the Red Hat "pidfile:" syntax in the SysV script header comment block to let systemd know which PID file (and hence PID) belongs to your service. Note that systemd cannot know if a SysV service is one of the kind where the runtime is defined by a specific process or whether it is one where there is none, hence the requirement of explicit configuration of a PID file in order to make systemd track the process lifetime. (Note that the Red Hat "pidfile:" stanza may only appear once in init scripts.)
* Runlevels are supported in a limited fashion only. SysV runlevels are mapped to systemd target units, however not all systemd target units map back to SysV runlevels. This is due to the fact that systemd targets are a lot more flexible and expressive than SysV runlevels. That means that checks for the current runlevel (with /sbin/runlevel or so) may well return "N" (i.e. unknown runlevel) during normal operation. Scripts that rely on explicit runlevel checks are incompatible with many setups. Avoid runlevel checks like these.
* Tools like /sbin/chkconfig might return misleading information when used to list enablement status of services. First of all, the tool will only see SysV services, not native units. Secondly, it will only show runlevel-related information (which does not fully map to systemd targets). Finally, the information shown might be overridden by a native unit file.
* By default runlevels 2,3,4 are all aliases for "multi-user.target". If a service is enabled in one of these runlevels, they'll be enabled in all of these. This is only a default however, and users can easily override the mappings, and split them up into individual runlevels if they want. However, we recommend moving on from runlevels and using the much more expressive target units of systemd.
* Early boot runlevels as they are used by some distributions are no longer supported. i.e. "fake", distribution-specific runlevels such as "S" or "b" cannot be used with systemd.
* On SysV systems changes to init scripts or any other files that define the boot process (such as /etc/fstab) usually had an immediate effect on everything started later. This is different on systemd-based systems where init script information and other boot-time configuration files are only reread when "systemctl daemon-reload" is issued. (Note that some commands, notably "systemctl enable"/"systemctl disable" do this implicitly however.) This is by design, and a safety feature, since it ensures that half-completed changes are not read at the wrong time.
* Multiple entries for the same mount path in /etc/fstab are not supported. In systemd there's only a single unit definition for each mount path read at any time. Also the listing order of mounts in /etc/fstab has no effect, mounts are executed in parallel and dependencies between them generated automatically depending on path prefixes and source paths.
* systemd's handling of the existing "nofail" mount option in /etc/fstab is stricter than it used to be on some sysvinit distributions: mount points that fail and are not listed as "nofail" will cause the boot to be stopped, for security reasons, as we we should not permit unprivileged code to run without everything listed — and not expressly exempted through "nofail" — being around. Hence, please mark all mounts where booting shall proceed regardless whether they succeeded or not with "nofail"
* Some SysV systems support an "rc.local" script that is supposed to be called "last" during boot. In systemd, the script is supported, but the semantics are less strict, as there is simply no concept of "last service", as the boot process is event- and request-based, parallelized and compositive. In general, it's a good idea to write proper unit files with properly defined dependncies, and avoid making use of rc.local.
* systemd assumes that the UID boundary between system and regular users is a choice the distribution makes, and not the administrator. Hence it expects this setting as compile-time option to be picked by the distribution. It will _not_ check /etc/login.defs during runtime.
Note that there are some areas where systemd currently provides a certain amount of compatibility where we expect this compatibility to be removed eventually.

View file

@ -0,0 +1,29 @@
---
title: My Service Can't Get Realtime!
category: Manuals and Documentation for Users and Administrators
layout: default
SPDX-License-Identifier: LGPL-2.1-or-later
---
# My Service Can't Get Realtime!
_So, you have a service that requires real-time scheduling. When you run this service on your systemd system it is unable to acquire real-time scheduling, even though it is full root and has all possible privileges. And now you are wondering what is going on and what you can do about it?_
## What is Going on?
By default systemd places all system services into their own control groups in the "cpu" hierarchy. This has the benefit that the CPU usage of services with many worker threads or processes (think: Apache with all its gazillion CGIs and stuff) gets roughly the same amount of CPU as a service with very few worker threads (think: MySQL). Instead of evening out CPU _per process_ this will cause CPU to be evened out _per service_.
Now, the "cpu" cgroup controller of the Linux kernel has one major shortcoming: if a cgroup is created it needs an explicit, absolute RT time budget assigned, or otherwise RT is not available to any process in the group, and an attempt to acquire it will fail with EPERM. systemd will not assign any RT time budgets to the "cpu" cgroups it creates, simply because there is no feasible way to do that, since the budget needs to be specified in absolute time units and comes from a fixed pool. Or in other words: we'd love to assign a budget, but there are no sane values we could use. Thus, in its default configuration RT scheduling is simply not available for any system services.
## Working Around the Issue
Of course, that's quite a limitation, so here's how you work around this:
* One option is to simply globally turn off that systemd creates a "cpu" cgroup for each of the system services. For that, edit `/etc/systemd/system.conf` and set `DefaultControllers=` to the empty string, then reboot. (An alternative is to disable the "cpu" controller in your kernel, entirely. systemd will not attempt to make use of controllers that aren't available in the kernel.)
* Another option is to turn this off for the specific service only. For that, edit your service file, and add `ControlGroup=cpu:/` to its `[Service]` section. This overrides the default logic for this one service only, and places all its processes back in the root cgroup of the "cpu" hierarchy, which has the full RT budget assigned.
* A third option is to simply assign your service a realtime budget. For that use `ControlGroupAttribute=cpu.rt_runtime_us 500000` in its `[Service]` or suchlike. See [the kernel documentation](http://www.kernel.org/doc/Documentation/scheduler/sched-design-CFS.txt) for details. The latter two options are not available for System V services. A possible solution is to write a small wrapper service file that simply calls the SysV script's start verb in `ExecStart=` and the stop verb in `ExecStop=`. (It also needs to set `RemainAfterExit=1` and `Type=forking`!)
Note that this all only applies to services. By default, user applications run in the root cgroup of the "cpu" hierarchy, which avoids these problems for normal user applications.
In the long run we hope that the kernel is fixed to not require an RT budget to be assigned for any cgroup created before a process can acquire RT (i.e. a process' RT budget should be derived from the nearest ancestor cgroup which has a budget assigned, rather than unconditionally its own uninitialized budget.) Ideally, we'd also like to create a per-user cgroup by default, so that users with many processes get roughly the same amount of CPU as users with very few.

View file

@ -0,0 +1,41 @@
---
title: Booting Without /usr is Broken
category: Manuals and Documentation for Users and Administrators
layout: default
SPDX-License-Identifier: LGPL-2.1-or-later
---
# Booting Without /usr is Broken
You probably discovered this page because your shiny new systemd system referred you here during boot time, when it warned you that booting without /usr pre-mounted wasn't supported anymore. And now you wonder what this all is about. Here's an attempt of an explanation:
One thing in advance: systemd itself is actually mostly fine with /usr on a separate file system that is not pre-mounted at boot time. However, the common basic set of OS components of modern Linux machines is not, and has not been in quite some time. And it is unlikely that this is going to be fixed any time soon, or even ever.
Most of the failures you will experience with /usr split off and not pre-mounted in the initramfs are graceful failures: they won't become directly visible, however certain features become unavailable due to these failures. Quite a number of programs these days hook themselves into the early boot process at various stages. A popular way to do this is for example via udev rules. The binaries called from these rules are sometimes located on /usr/bin, or link against libraries in /usr/lib, or use data files from /usr/share. If these rules fail udev will proceed with the next one, however later on applications will then not properly detect these udev devices or features of these devices. Here's a short, very in-comprehensive list of software we are aware of that currently are not able to provide the full set of functionality when /usr is split off and not pre-mounted at boot: udev-pci-db/udev-usb-db and all rules depending on this (using the PCI/USB database in /usr/share), PulseAudio, NetworkManager, ModemManager, udisks, libatasmart, usb\_modeswitch, gnome-color-manager, usbmuxd, ALSA, D-Bus, CUPS, Plymouth, LVM, hplip, multipath, Argyll, VMWare, the locale logic of most programs and a lot of other stuff.
You don't believe us? Well, here's a command line that reveals a few obvious cases of udev rules that will silently fail to work if /usr is split off and not pre-mounted: `egrep 'usb-db|pci-db|FROM_DATABASE|/usr' /*/udev/rules.d/*` -- and you find a lot more if you actually look for it. On my fresh Fedora 15 install that's 23 obvious cases.
## The Status Quo
Due to this, many upstream developers have decided to consider the problem of a separate /usr that is not mounted during early boot an outdated question, and started to close bugs regarding these issues as WONTFIX. We certainly cannot blame them, as the benefit of supporting this is questionable and brings a lot of additional work with it.
And let's clarify a few things:
1. **It isn't systemd's fault.** systemd mostly works fine with /usr on a separate file system that is not pre-mounted at boot.
2. **systemd is merely the messenger.** Don't shoot the messenger.
3. **There's no news in all of this.** The message you saw is just a statement of fact, describing the status quo. Things have been this way since a while.
4. **The message is merely a warning.** You can choose to ignore it.
5. **Don't blame us**, don't abuse us, it's not our fault. We have been working on the Linux userspace since quite some time, and simply have enough of the constant bug reports regarding these issues, since they are actually very hard to track down because the failures are mostly graceful. Hence we placed this warning into the early boot process of every systemd Linux system with a split off and not pre-mounted /usr, so that people understand what is going on.
## Going Forward
/usr on its own filesystem is useful in some custom setups. But instead of expecting the traditional Unix way to (sometimes mindlessly) distributing tools between /usr and /, and require more and more tools to move to /, we now just expect /usr to be pre-mounted from inside the initramfs, to be available before 'init' starts. The duty of the minimal boot system that consisted of /bin, /sbin and /lib on traditional Unix, has been taken over by the initramfs of modern Linux. An initramfs that supports mounting /usr on top of / before it starts 'init', makes all existing setups work properly.
There is no way to reliably bring up a modern system with an empty /usr. There are two alternatives to fix it: move /usr back to the rootfs or use an initramfs which can hide the split-off from the system.
On the Fedora distribution we have succeeded to clean up the situation and the confusion the current split between / and /usr has created. We have moved all tools that over time have been moved to / back to /usr (where they belong), and the root file system only contains compatibility symlinks for /bin and /sbin into /usr. All binaries of the system are exclusively located within the /usr hierarchy.
In this new definition of /usr, the directory can be mounted read-only by default, while the rootfs may be either read-write or read-only (for stateless systems) and contains only the empty mount point directories, compat-symlinks to /usr and the host-specific data like /etc, /root, /srv. In comparison to today's setups, the rootfs will be very small. The host-specific data will be properly separated from the installed operating system. The new /usr could also easily be shared read-only across several systems. Such a setup would be more efficient, can provide additional security, is more flexible to use, provides saner options for custom setups, and is much simpler to setup and maintain.
For more information on this please continue to [The Case for the /usr Merge](../THE_CASE_FOR_THE_USR_MERGE).

186
docs/TIPS_AND_TRICKS.md Normal file
View file

@ -0,0 +1,186 @@
---
title: Tips And Tricks
category: Manuals and Documentation for Users and Administrators
layout: default
SPDX-License-Identifier: LGPL-2.1-or-later
---
# Tips & Tricks
Also check out the [Frequently Asked Questions](http://www.freedesktop.org/wiki/Software/systemd/FrequentlyAskedQuestions)!
## Listing running services
```sh
$ systemctl
UNIT LOAD ACTIVE SUB JOB DESCRIPTION
accounts-daemon.service loaded active running Accounts Service
atd.service loaded active running Job spooling tools
avahi-daemon.service loaded active running Avahi mDNS/DNS-SD Stack
bluetooth.service loaded active running Bluetooth Manager
colord-sane.service loaded active running Daemon for monitoring attached scanners and registering them with colord
colord.service loaded active running Manage, Install and Generate Color Profiles
crond.service loaded active running Command Scheduler
cups.service loaded active running CUPS Printing Service
dbus.service loaded active running D-Bus System Message Bus
...
```
## Showing runtime status
```sh
$ systemctl status udisks2.service
udisks2.service - Storage Daemon
Loaded: loaded (/usr/lib/systemd/system/udisks2.service; static)
Active: active (running) since Wed, 27 Jun 2012 20:49:25 +0200; 1 day and 1h ago
Main PID: 615 (udisksd)
CGroup: name=systemd:/system/udisks2.service
└ 615 /usr/lib/udisks2/udisksd --no-debug
Jun 27 20:49:25 epsilon udisksd[615]: udisks daemon version 1.94.0 starting
Jun 27 20:49:25 epsilon udisksd[615]: Acquired the name org.freedesktop.UDisks2 on the system message bus
```
## cgroup tree
```sh
$ systemd-cgls
└ system
├ 1 /usr/lib/systemd/systemd --system --deserialize 18
├ ntpd.service
│ └ 8471 /usr/sbin/ntpd -u ntp:ntp -g
├ upower.service
│ └ 798 /usr/libexec/upowerd
├ wpa_supplicant.service
│ └ 751 /usr/sbin/wpa_supplicant -u -f /var/log/wpa_supplicant.log -c /etc/wpa_supplicant/wpa_supplicant.conf -u -f /var/log/wpa_supplicant.log -P /var/run/wpa_supplicant.pid
├ nfs-idmap.service
│ └ 731 /usr/sbin/rpc.idmapd
├ nfs-rquotad.service
│ └ 753 /usr/sbin/rpc.rquotad
├ nfs-mountd.service
│ └ 732 /usr/sbin/rpc.mountd
├ nfs-lock.service
│ └ 704 /sbin/rpc.statd
├ rpcbind.service
│ └ 680 /sbin/rpcbind -w
├ postfix.service
│ ├ 859 /usr/libexec/postfix/master
│ ├ 877 qmgr -l -t fifo -u
│ └ 32271 pickup -l -t fifo -u
├ colord-sane.service
│ └ 647 /usr/libexec/colord-sane
├ udisks2.service
│ └ 615 /usr/lib/udisks2/udisksd --no-debug
├ colord.service
│ └ 607 /usr/libexec/colord
├ prefdm.service
│ ├ 567 /usr/sbin/gdm-binary -nodaemon
│ ├ 602 /usr/libexec/gdm-simple-slave --display-id /org/gnome/DisplayManager/Display1
│ ├ 612 /usr/bin/Xorg :0 -br -verbose -auth /var/run/gdm/auth-for-gdm-O00GPA/database -seat seat0 -nolisten tcp
│ └ 905 gdm-session-worker [pam/gdm-password]
├ systemd-ask-password-wall.service
│ └ 645 /usr/bin/systemd-tty-ask-password-agent --wall
├ atd.service
│ └ 544 /usr/sbin/atd -f
├ ksmtuned.service
│ ├ 548 /bin/bash /usr/sbin/ksmtuned
│ └ 1092 sleep 60
├ dbus.service
│ ├ 586 /bin/dbus-daemon --system --address=systemd: --nofork --systemd-activation
│ ├ 601 /usr/libexec/polkit-1/polkitd --no-debug
│ └ 657 /usr/sbin/modem-manager
├ cups.service
│ └ 508 /usr/sbin/cupsd -f
├ avahi-daemon.service
│ ├ 506 avahi-daemon: running [epsilon.local]
│ └ 516 avahi-daemon: chroot helper
├ system-setup-keyboard.service
│ └ 504 /usr/bin/system-setup-keyboard
├ accounts-daemon.service
│ └ 502 /usr/libexec/accounts-daemon
├ systemd-logind.service
│ └ 498 /usr/lib/systemd/systemd-logind
├ crond.service
│ └ 486 /usr/sbin/crond -n
├ NetworkManager.service
│ ├ 484 /usr/sbin/NetworkManager --no-daemon
│ └ 8437 /sbin/dhclient -d -4 -sf /usr/libexec/nm-dhcp-client.action -pf /var/run/dhclient-wlan0.pid -lf /var/lib/dhclient/dhclient-903b6f6aa7a1-46c8-82a9-7f637dfbb3e4-wlan0.lease -cf /var/run/nm-d...
├ libvirtd.service
│ ├ 480 /usr/sbin/libvirtd
│ └ 571 /sbin/dnsmasq --strict-order --bind-interfaces --pid-file=/var/run/libvirt/network/default.pid --conf-file= --except-interface lo --listenaddress 192.168.122.1 --dhcp-range 192.168.122.2,1...
├ bluetooth.service
│ └ 479 /usr/sbin/bluetoothd -n
├ systemd-udev.service
│ └ 287 /usr/lib/systemd/systemd-udevd
└ systemd-journald.service
└ 280 /usr/lib/systemd/systemd-journald
```
### ps with cgroups
```sh
$ alias psc='ps xawf -eo pid,user,cgroup,args'
$ psc
PID USER CGROUP COMMAND
...
1 root name=systemd:/systemd-1 /bin/systemd systemd.log_target=kmsg systemd.log_level=debug selinux=0
415 root name=systemd:/systemd-1/sysinit.service /sbin/udevd -d
928 root name=systemd:/systemd-1/atd.service /usr/sbin/atd -f
930 root name=systemd:/systemd-1/ntpd.service /usr/sbin/ntpd -n
932 root name=systemd:/systemd-1/crond.service /usr/sbin/crond -n
935 root name=systemd:/systemd-1/auditd.service /sbin/auditd -n
943 root name=systemd:/systemd-1/auditd.service \_ /sbin/audispd
964 root name=systemd:/systemd-1/auditd.service \_ /usr/sbin/sedispatch
937 root name=systemd:/systemd-1/acpid.service /usr/sbin/acpid -f
941 rpc name=systemd:/systemd-1/rpcbind.service /sbin/rpcbind -f
944 root name=systemd:/systemd-1/rsyslog.service /sbin/rsyslogd -n -c 4
947 root name=systemd:/systemd-1/systemd-logger.service /lib/systemd/systemd-logger
950 root name=systemd:/systemd-1/cups.service /usr/sbin/cupsd -f
955 dbus name=systemd:/systemd-1/messagebus.service /bin/dbus-daemon --system --address=systemd: --nofork --systemd-activation
969 root name=systemd:/systemd-1/getty@.service/tty6 /sbin/mingetty tty6
970 root name=systemd:/systemd-1/getty@.service/tty5 /sbin/mingetty tty5
971 root name=systemd:/systemd-1/getty@.service/tty1 /sbin/mingetty tty1
973 root name=systemd:/systemd-1/getty@.service/tty4 /sbin/mingetty tty4
974 root name=systemd:/user/lennart/2 login -- lennart
1824 lennart name=systemd:/user/lennart/2 \_ -bash
975 root name=systemd:/systemd-1/getty@.service/tty3 /sbin/mingetty tty3
988 root name=systemd:/systemd-1/polkitd.service /usr/libexec/polkit-1/polkitd
994 rtkit name=systemd:/systemd-1/rtkit-daemon.service /usr/libexec/rtkit-daemon
...
```
## Changing the Default Boot Target
```sh
$ ln -sf /usr/lib/systemd/system/multi-user.target /etc/systemd/system/default.target
```
This line makes the multi user target (i.e. full system, but no graphical UI) the default target to boot into. This is kinda equivalent to setting runlevel 3 as the default runlevel on Fedora/sysvinit systems.
```sh
$ ln -sf /usr/lib/systemd/system/graphical.target /etc/systemd/system/default.target
```
This line makes the graphical target (i.e. full system, including graphical UI) the default target to boot into. Kinda equivalent to runlevel 5 on fedora/sysvinit systems. This is how things are shipped by default.
## What other units does a unit depend on?
For example, if you want to figure out which services a target like multi-user.target pulls in, use something like this:
```sh
$ systemctl show -p "Wants" multi-user.target
Wants=rc-local.service avahi-daemon.service rpcbind.service NetworkManager.service acpid.service dbus.service atd.service crond.service auditd.service ntpd.service udisks.service bluetooth.service cups.service wpa_supplicant.service getty.target modem-manager.service portreserve.service abrtd.service yum-updatesd.service upowerd.service test-first.service pcscd.service rsyslog.service haldaemon.service remote-fs.target plymouth-quit.service systemd-update-utmp-runlevel.service sendmail.service lvm2-monitor.service cpuspeed.service udev-post.service mdmonitor.service iscsid.service livesys.service livesys-late.service irqbalance.service iscsi.service netfs.service
```
Instead of "Wants" you might also try "WantedBy", "Requires", "RequiredBy", "Conflicts", "ConflictedBy", "Before", "After" for the respective types of dependencies and their inverse.
## What would get started if I booted into a specific target?
If you want systemd to calculate the "initial" transaction it would execute on boot, try something like this:
```sh
$ systemd --test --system --unit=foobar.target
```
for a boot target foobar.target. Note that this is mostly a debugging tool that actually does a lot more than just calculate the initial transaction, so don't build scripts based on this.

View file

@ -393,5 +393,25 @@
"category": "Related Packages",
"title": "C++ bindings for sd-bus",
"url": "https://github.com/Kistler-Group/sdbus-cpp/"
}
},
{
"category": "Documentation for Developers - external links",
"title": "On /etc/os-release",
"url": "http://0pointer.de/blog/projects/os-release.html"
},
{
"category": "Documentation for Developers - external links",
"title": "Control Groups vs. Control Groups",
"url": "http://0pointer.de/blog/projects/cgroups-vs-cgroups.html"
},
{
"category": "Documentation for Developers - external links",
"title": "The 30 Biggest Myths about systemd",
"url": "http://0pointer.de/blog/projects/the-biggest-myths.html"
},
{
"category": "Documentation for Developers - external links",
"title": "Introduction to systemd in French",
"url": "http://lea-linux.org/documentations/Systemd"
},
]