systemd/man/systemd-vmspawn.xml

<?xml version='1.0'?>
<!DOCTYPE refentry PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
  "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
<!ENTITY % entities SYSTEM "custom-entities.ent" >
%entities;
]>
<!-- SPDX-License-Identifier: LGPL-2.1-or-later -->

<refentry id="systemd-vmspawn" conditional="ENABLE_VMSPAWN"
    xmlns:xi="http://www.w3.org/2001/XInclude">

  <refentryinfo>
    <title>systemd-vmspawn</title>
    <productname>systemd</productname>
  </refentryinfo>

  <refmeta>
    <refentrytitle>systemd-vmspawn</refentrytitle>
    <manvolnum>1</manvolnum>
  </refmeta>

  <refnamediv>
    <refname>systemd-vmspawn</refname>
    <refpurpose>Spawn an OS in a virtual machine</refpurpose>
  </refnamediv>

  <refsynopsisdiv>
    <cmdsynopsis>
      <command>systemd-vmspawn</command>
      <arg choice="opt" rep="repeat">OPTIONS</arg>
      <arg choice="opt" rep="repeat">ARGS</arg>
    </cmdsynopsis>
  </refsynopsisdiv>

  <refsect1>
    <title>Description</title>
    <para><command>systemd-vmspawn</command> may be used to start a virtual machine from an OS image. In many ways it is similar to <citerefentry>
    <refentrytitle>systemd-nspawn</refentrytitle><manvolnum>1</manvolnum></citerefentry>, but
    launches a full virtual machine instead of using namespaces.</para>

    <para>File descriptors for <filename>/dev/kvm</filename> and <filename>/dev/vhost-vsock</filename> can be
    passed to <command>systemd-vmspawn</command> via systemd's native socket passing interface (see
    <citerefentry><refentrytitle>sd_listen_fds</refentrytitle><manvolnum>3</manvolnum></citerefentry> for
    details about the precise protocol used and the order in which the file descriptors are passed), these
    file descriptors must be passed with the names <literal>kvm</literal> and <literal>vhost-vsock</literal>
    respectively.</para>

    <para>Note: on Ubuntu/Debian derivatives systemd-vmspawn requires the user to be in the
    <literal>kvm</literal> group to use the VSOCK options.</para>
  </refsect1>

  <refsect1>
    <title>Options</title>

    <para>The excess arguments are passed as extra kernel command line arguments using SMBIOS.</para>

    <para>The following options are understood:</para>

    <variablelist>

      <varlistentry>
        <term><option>-q</option></term>
        <term><option>--quiet</option></term>

        <listitem><para>Turns off any status output by the tool
        itself. When this switch is used, the only output from vmspawn
        will be the console output of the Virtual Machine OS itself.</para>

        <xi:include href="version-info.xml" xpointer="v256"/></listitem>
      </varlistentry>

    </variablelist>

    <refsect2>
      <title>Image Options</title>

      <variablelist>
        <varlistentry>
          <term><option>-D</option></term>
          <term><option>--directory=</option></term>

          <listitem><para>Directory to use as file system root for the virtual machine.</para>

          <para>One of either <option>--directory=</option> or <option>--image=</option> must be specified.
          If neither are specified <option>--directory=.</option> is assumed.</para>

          <para>Note: If mounting a non-root owned directory you may require <option>--private-users=</option>
          to map into the user's subuid namespace. An example of how to use <constant>/etc/subuid</constant>
          for this is given later.</para>

          <xi:include href="version-info.xml" xpointer="v256"/>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term><option>-i</option></term>
          <term><option>--image=</option></term>

          <listitem><para>Root file system disk image (or device node) for the virtual machine.</para>

          <xi:include href="version-info.xml" xpointer="v255"/>
          </listitem>
        </varlistentry>
      </variablelist>
    </refsect2>

    <refsect2>
      <title>Host Configuration</title>

      <variablelist>
        <varlistentry>
          <term><option>--cpus=<replaceable>CPUS</replaceable></option></term>

          <listitem><para>The number of CPUs to start the virtual machine with.
          Defaults to 1.</para>

          <xi:include href="version-info.xml" xpointer="v255"/>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term><option>--ram=<replaceable>BYTES</replaceable></option></term>

          <listitem><para>The amount of memory to start the virtual machine with.
          Defaults to 2G.</para>

          <xi:include href="version-info.xml" xpointer="v255"/>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term><option>--kvm=<replaceable>BOOL</replaceable></option></term>

          <listitem><para>If <option>--kvm=</option> is not specified KVM support will be
          detected automatically. If true, KVM is always used, and if false, KVM is never used.</para>

          <xi:include href="version-info.xml" xpointer="v255"/></listitem>
        </varlistentry>

        <varlistentry>
          <term><option>--vsock=<replaceable>BOOL</replaceable></option></term>

          <listitem><para>If <option>--vsock=</option> is not specified VSOCK networking support will be
          detected automatically. If true, VSOCK networking is always used, and if false, VSOCK networking is never used.</para>

          <xi:include href="version-info.xml" xpointer="v255"/></listitem>
        </varlistentry>

        <varlistentry>
          <term><option>--vsock-cid=<replaceable>CID</replaceable></option></term>

          <listitem>
            <para>Sets the specific CID to use for the guest.
            Valid CIDs are in the range <constant>3</constant> to <constant>4294967294</constant> (<constant>0xFFFF_FFFE</constant>).
            CIDs outside of this range are reserved. By default vmspawn will attempt to derive a CID for the guest derived from the machine name,
            falling back to a random CID if this CID is taken.</para>

            <xi:include href="version-info.xml" xpointer="v255"/>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term><option>--tpm=<replaceable>BOOL</replaceable></option></term>

          <listitem>
            <para>If <option>--tpm=</option> is not specified vmspawn will detect the presence of <citerefentry project='debian'>
            <refentrytitle>swtpm</refentrytitle><manvolnum>8</manvolnum></citerefentry> and use it if available.
            If yes is specified <citerefentry project='debian'><refentrytitle>swtpm</refentrytitle><manvolnum>8</manvolnum></citerefentry>
            is always used, and if no is set <citerefentry project='debian'><refentrytitle>swtpm</refentrytitle>
            <manvolnum>8</manvolnum></citerefentry> is never used.</para>

            <para>Note: the virtual TPM used may change in future.</para>

            <xi:include href="version-info.xml" xpointer="v256"/>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term><option>--linux=<replaceable>PATH</replaceable></option></term>

          <listitem>
            <para>Set the linux kernel image to use for direct kernel boot.
            If a directory type image is used and <option>--linux=</option> was omitted, vmspawn will search for boot loader entries
            according to the
            <ulink url="https://uapi-group.org/specifications/specs/boot_loader_specification">Boot Loader Specification</ulink> assuming
            XBOOTLDR to be located at /boot and ESP to be /efi respectively.
            If no kernel was installed into the image then the image will fail to boot.</para>

            <xi:include href="version-info.xml" xpointer="v256"/>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term><option>--initrd=<replaceable>PATH</replaceable></option></term>

          <listitem>
            <para>Set the initrd to use for direct kernel boot.
            If the <option>--linux=</option> supplied is a
            <ulink url="https://uapi-group.org/specifications/specs/boot_loader_specification">Boot Loader Specification</ulink>
            Type #2 entry, then this argument is not required.
            If no initrd was installed into the image then the image will fail to boot.</para>

            <para><option>--initrd=</option> can be specified multiple times and vmspawn will merge them together.</para>

            <xi:include href="version-info.xml" xpointer="v256"/>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term><option>-n</option></term>
          <term><option>--network-tap</option></term>

          <listitem>
            <para>Create a TAP device to network with the virtual machine.</para>

            <para>Note: root privileges are required to use TAP networking.
            Additionally,
            <citerefentry><refentrytitle>systemd-networkd</refentrytitle><manvolnum>8</manvolnum></citerefentry>
            must be running and correctly set up on the host to provision the host interface. The relevant
            <literal>.network</literal> file can be found at
            <filename>/usr/lib/systemd/network/80-vm-vt.network</filename>.
            </para>

            <xi:include href="version-info.xml" xpointer="v255"/>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term><option>--network-user-mode</option></term>

          <listitem><para>Use user mode networking.</para>

          <xi:include href="version-info.xml" xpointer="v255"/></listitem>
        </varlistentry>

        <varlistentry>
          <term><option>--firmware=<replaceable>PATH</replaceable></option></term>

          <listitem><para>Takes an absolute path, or a relative path beginning with
          <filename>./</filename>. Specifies a JSON firmware definition file, which allows selecting the
          firmware to boot in the VM. If not specified a suitable firmware is automatically discovered. If the
          special string <literal>list</literal> is specified lists all discovered firmwares.</para>

          <xi:include href="version-info.xml" xpointer="v256"/></listitem>
        </varlistentry>

        <varlistentry>
          <term><option>--discard-disk=<replaceable>BOOL</replaceable></option></term>

          <listitem><para>Controls whether qemu processes discard requests from the VM.
          This prevents long running VMs from using more disk space than required.
          This is enabled by default.</para>

          <xi:include href="version-info.xml" xpointer="v256"/></listitem>
        </varlistentry>

        <varlistentry>
          <term><option>--secure-boot=<replaceable>BOOL</replaceable></option></term>

          <listitem><para>Configure whether to search for firmware which supports Secure Boot.</para>

          <para>If the option is not specified the first firmware which is detected will be used.
          If the option is set to yes then the first firmware with Secure Boot support will be selected.
          If no is specified then the first firmware without Secure Boot will be selected.</para>

          <xi:include href="version-info.xml" xpointer="v255"/></listitem>
        </varlistentry>
      </variablelist>
    </refsect2>

    <refsect2>
      <title>System Identity Options</title>

      <variablelist>
        <varlistentry>
          <term><option>-M</option></term>
          <term><option>--machine=</option></term>

          <listitem><para>Sets the machine name for this virtual machine. This
          name may be used to identify this virtual machine during its runtime
          (for example in tools like
          <citerefentry><refentrytitle>machinectl</refentrytitle><manvolnum>1</manvolnum></citerefentry>
          and similar).</para>

          <xi:include href="version-info.xml" xpointer="v255"/>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term><option>--uuid=</option></term>

          <listitem><para>Set the specified UUID for the virtual machine. The
          init system will initialize
          <filename>/etc/machine-id</filename> from this if this file is
          not set yet. Note that this option takes effect only if
          <filename>/etc/machine-id</filename> in the virtual machine is
          unpopulated.</para>

          <xi:include href="version-info.xml" xpointer="v256"/></listitem>
        </varlistentry>
      </variablelist>
    </refsect2>

    <refsect2>
      <title>Property Options</title>

      <variablelist>
        <varlistentry>
          <term><option>--register=</option></term>

          <listitem><para>Controls whether the virtual machine is registered with
          <citerefentry><refentrytitle>systemd-machined</refentrytitle><manvolnum>8</manvolnum></citerefentry>.  Takes a
          boolean argument, which defaults to <literal>yes</literal> when running as root, and <literal>no</literal> when
          running as a regular user.  This ensures that the virtual machine is accessible via
          <citerefentry><refentrytitle>machinectl</refentrytitle><manvolnum>1</manvolnum></citerefentry>.</para>

          <para>Note: root privileges are required to use this option as registering with
          <citerefentry><refentrytitle>systemd-machined</refentrytitle><manvolnum>8</manvolnum></citerefentry>
          requires privileged D-Bus method calls.</para>

          <xi:include href="version-info.xml" xpointer="v256"/></listitem>
        </varlistentry>
      </variablelist>
    </refsect2>

    <refsect2>
      <title>User Namespacing Options</title>

      <variablelist>
        <varlistentry>
          <term><option>--private-users=<replaceable>UID_SHIFT[:UID_RANGE]</replaceable></option></term>

          <listitem><para>Controls user namespacing under <option>--directory=</option>.
          If enabled,
          <citerefentry><refentrytitle>virtiofsd</refentrytitle><manvolnum>1</manvolnum></citerefentry>
          is instructed to map user and group ids (UIDs and GIDs).
          This involves mapping the private UIDs/GIDs used in the virtual machine (starting with the virtual machine's
          root user 0 and up) to a range of UIDs/GIDs on the host that are not used for other purposes (usually in the
          range beyond the host's UID/GID 65536).</para>

          <para>If one or two colon-separated numbers are specified, user namespacing is turned on. <replaceable>UID_SHIFT</replaceable>
          specifies the first host UID/GID to map, <replaceable>UID_RANGE</replaceable> is optional and specifies number of host
          UIDs/GIDs to assign to the virtual machine. If <replaceable>UID_RANGE</replaceable> is omitted, 65536 UIDs/GIDs are assigned.</para>

          <para>When user namespaces are used, the GID range assigned to each virtual machine is always chosen identical to the
          UID range.</para>

          <xi:include href="version-info.xml" xpointer="v256"/></listitem>
        </varlistentry>
      </variablelist>
    </refsect2>

    <refsect2>
      <title>Mount Options</title>

      <variablelist>

        <varlistentry>
          <term><option>--bind=<replaceable>PATH</replaceable></option></term>
          <term><option>--bind-ro=<replaceable>PATH</replaceable></option></term>

          <listitem><para>Mount a directory from the host into the virtual machine. Takes one of: a path
          argument — in which case the specified path will be mounted from the host to the same path in the virtual machine, or
          a colon-separated pair of paths — in which case the first specified path is the source in the host, and the
          second path is the destination in the virtual machine. If the source path is not absolute, it is resolved
          relative to the current working directory. The <option>--bind-ro=</option> option creates read-only bind mounts.
          Backslash escapes are interpreted, so <literal>\:</literal> may be used to embed colons in either path.
          This option may be specified multiple times for creating multiple independent bind mount points.</para>

          <xi:include href="version-info.xml" xpointer="v256"/></listitem>
        </varlistentry>

        <varlistentry>
          <term><option>--extra-drive=<replaceable>PATH</replaceable></option></term>

          <listitem><para>Takes a disk image or block device on the host and supplies it to the virtual machine as another drive.</para>

          <xi:include href="version-info.xml" xpointer="v256"/></listitem>
        </varlistentry>

      </variablelist>
    </refsect2>

    <refsect2>
      <title>Integration Options</title>

      <variablelist>
        <varlistentry>
          <term><option>--forward-journal=<replaceable>FILE|DIR</replaceable></option></term>

          <listitem><para>Forward the virtual machine's journal to the host.
          <citerefentry><refentrytitle>systemd-journal-remote</refentrytitle><manvolnum>8</manvolnum></citerefentry>
          is currently used to receive the guest VM's forwarded journal entries. This option determines where
          this journal is saved on the host and has the same semantics as
          <option>-o</option>/<option>--output</option> described in
          <citerefentry><refentrytitle>systemd-journal-remote</refentrytitle><manvolnum>8</manvolnum></citerefentry>.</para>

          <xi:include href="version-info.xml" xpointer="v256"/>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term><option>--pass-ssh-key=<replaceable>BOOL</replaceable></option></term>

          <listitem><para>By default an SSH key is generated to allow <command>systemd-vmspawn</command> to open
          a D-Bus connection to the VM's systemd bus. Setting this to "no" will disable SSH key generation.</para>

          <para>The generated keys are ephemeral. That is they are valid only for the current invocation of <command>systemd-vmspawn</command>,
          and are typically not persisted.</para>

          <xi:include href="version-info.xml" xpointer="v256"/>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term><option>--ssh-key-type=<replaceable>TYPE</replaceable></option></term>

          <listitem><para>Configures the type of SSH key to generate, see
          <citerefentry project="man-pages"><refentrytitle>ssh-keygen</refentrytitle><manvolnum>1</manvolnum></citerefentry>
          for more information.</para>

          <para>By default <literal>ed25519</literal> keys are generated, however <literal>rsa</literal> keys
          may also be useful if the VM has a particularly old version of <command>sshd</command>.</para>

          <xi:include href="version-info.xml" xpointer="v256"/>
          </listitem>
        </varlistentry>
      </variablelist>
    </refsect2>

    <refsect2>
      <title>Input/Output Options</title>

      <variablelist>
        <varlistentry>
          <term><option>--console=<replaceable>MODE</replaceable></option></term>

          <listitem><para>Configures how to set up the console of the VM. Takes one of
          <literal>interactive</literal>, <literal>read-only</literal>, <literal>native</literal>,
          <literal>gui</literal>. Defaults to <literal>interactive</literal>. <literal>interactive</literal>
          provides an interactive terminal interface to the VM. <literal>read-only</literal> is similar, but
          is strictly read-only, i.e. does not accept any input from the user. <literal>native</literal> also
          provides a TTY-based interface, but uses qemu native implementation (which means the qemu monitor
          is available). <literal>gui</literal> shows the qemu graphical UI.</para>

          <xi:include href="version-info.xml" xpointer="v256"/></listitem>
        </varlistentry>

        <varlistentry>
          <term><option>--background=<replaceable>COLOR</replaceable></option></term>

          <listitem><para>Change the terminal background color to the specified ANSI color as long as the VM
          runs. The color specified should be an ANSI X3.64 SGR background color, i.e. strings such as
          <literal>40</literal>, <literal>41</literal>, …, <literal>47</literal>, <literal>48;2;…</literal>,
          <literal>48;5;…</literal>. See <ulink
          url="https://en.wikipedia.org/wiki/ANSI_escape_code#SGR_(Select_Graphic_Rendition)_parameters">ANSI
          Escape Code (Wikipedia)</ulink> for details. Assign an empty string to disable any coloring. This
          only has an effect in <option>--console=interactive</option> and
          <option>--console=read-only</option> modes.</para>

          <xi:include href="version-info.xml" xpointer="v256"/>
          </listitem>
        </varlistentry>
      </variablelist>
    </refsect2>

    <refsect2>
      <title>Credentials</title>

      <variablelist>
        <varlistentry>
          <term><option>--load-credential=<replaceable>ID</replaceable>:<replaceable>PATH</replaceable></option></term>
          <term><option>--set-credential=<replaceable>ID</replaceable>:<replaceable>VALUE</replaceable></option></term>

          <listitem><para>Pass a credential to the virtual machine. These two options correspond to the
          <varname>LoadCredential=</varname> and <varname>SetCredential=</varname> settings in unit files. See
          <citerefentry><refentrytitle>systemd.exec</refentrytitle><manvolnum>5</manvolnum></citerefentry> for
          details about these concepts, as well as the syntax of the option's arguments.</para>

          <para>In order to embed binary data into the credential data for <option>--set-credential=</option>,
          use C-style escaping (i.e. <literal>\n</literal> to embed a newline, or <literal>\x00</literal> to
          embed a <constant>NUL</constant> byte). Note that the invoking shell might already apply unescaping
          once, hence this might require double escaping!</para>

          <xi:include href="version-info.xml" xpointer="v255"/></listitem>
        </varlistentry>
      </variablelist>

    </refsect2><refsect2>
      <title>Other</title>

      <variablelist>
        <xi:include href="standard-options.xml" xpointer="no-pager" />
        <xi:include href="standard-options.xml" xpointer="help" />
        <xi:include href="standard-options.xml" xpointer="version" />
      </variablelist>
    </refsect2>
  </refsect1>

  <xi:include href="common-variables.xml" />

  <refsect1>
    <title>Examples</title>

    <example>
      <title>Run an Arch Linux VM image generated by mkosi</title>

      <programlisting>
$ mkosi -d arch -p systemd -p linux --autologin -o image.raw -f build
$ systemd-vmspawn --image=image.raw
      </programlisting>
    </example>

    <example>
      <title>Import and run a Fedora 39 Cloud image using machinectl</title>

      <programlisting>
$ curl -L \
       -O https://download.fedoraproject.org/pub/fedora/linux/releases/&fedora_latest_version;/Cloud/x86_64/images/Fedora-Cloud-Base-&fedora_latest_version;-&fedora_cloud_release;.x86_64.raw.xz \
       -O https://download.fedoraproject.org/pub/fedora/linux/releases/&fedora_latest_version;/Cloud/x86_64/images/Fedora-Cloud-&fedora_latest_version;-&fedora_cloud_release;-x86_64-CHECKSUM \
       -O https://fedoraproject.org/fedora.gpg
$ gpgv --keyring ./fedora.gpg Fedora-Cloud-&fedora_latest_version;-&fedora_cloud_release;-x86_64-CHECKSUM
$ sha256sum -c Fedora-Cloud-&fedora_latest_version;-&fedora_cloud_release;-x86_64-CHECKSUM
# machinectl import-raw Fedora-Cloud-Base-&fedora_latest_version;-&fedora_cloud_release;.x86_64.raw.xz fedora-&fedora_latest_version;-cloud
# systemd-vmspawn -M fedora-&fedora_latest_version;-cloud
      </programlisting>
    </example>

    <example>
      <title>Build and run systemd's system image and forward the VM's journal to a local file</title>

      <programlisting>
$ mkosi build
$ systemd-vmspawn \
    -D mkosi.output/system \
    --private-users $(grep $(whoami) /etc/subuid | cut -d: -f2) \
    --linux mkosi.output/system.efi \
    --forward-journal=vm.journal \
    enforcing=0
      </programlisting>

      <para>Note: this example also uses a kernel command line argument to ensure SELinux isn't started in enforcing mode.</para>
    </example>

    <example>
      <title>SSH into a running VM using <command>systemd-ssh-proxy</command></title>

      <programlisting>
$ mkosi build
$ my_vsock_cid=3735928559
$ systemd-vmspawn \
    -D mkosi.output/system \
    --private-users $(grep $(whoami) /etc/subuid | cut -d: -f2) \
    --linux mkosi.output/system.efi \
    --vsock-cid $my_vsock_cid \
    enforcing=0
$ ssh root@vsock/$my_vsock_cid -i /run/user/$UID/systemd/vmspawn/machine-*-system-ed25519
      </programlisting>
    </example>
  </refsect1>

  <refsect1>
    <title>Exit status</title>

    <para>If an error occurred the value errno is propagated to the return code.
    If EXIT_STATUS is supplied by the running image that is returned.
    Otherwise EXIT_SUCCESS is returned.</para>
  </refsect1>

  <refsect1>
    <title>See Also</title>
    <para><simplelist type="inline">
      <member><citerefentry><refentrytitle>systemd</refentrytitle><manvolnum>1</manvolnum></citerefentry></member>
      <member><citerefentry project='debian'><refentrytitle>mkosi</refentrytitle><manvolnum>1</manvolnum></citerefentry></member>
      <member><citerefentry><refentrytitle>machinectl</refentrytitle><manvolnum>1</manvolnum></citerefentry></member>
      <member><citerefentry><refentrytitle>importctl</refentrytitle><manvolnum>1</manvolnum></citerefentry></member>
      <member><ulink url="https://uapi-group.org/specifications/specs/boot_loader_specification">Boot Loader Specification</ulink></member>
    </simplelist></para>
  </refsect1>
</refentry>