jmarya/myverse
jmarya/myverse
Watch
1
0
Fork
You've already forked myverse
 0
Code Issues 5 Pull requests Projects Releases Packages Wiki Activity Actions

Declarative disk layouts (disko templates) #6

New issue
Open
opened 2026-09-02 18:24:21 +00:00 by jmarya · 0 comments
jmarya commented 2026-09-02 18:24:21 +00:00
Owner
Copy link

Problem

myverse has no disk story at all. Grepping the tree: zero fileSystems, zero swapDevices, and no bootloader enablement anywhere except profiles/features/secure_boot.nix force-disabling systemd-boot. Every host has to hand-write partitioning, mounts, swap and boot config. hosts/live only escapes this because the installer-cd module supplies it.

Two consequences:

  1. A machine cannot be reinstalled from its own declarations. The most important thing about a host — how its storage is laid out — lives only in the residue of whatever someone typed once.
  2. #4 is blocked. The installer's disk step has nothing to offer the user. "Pick a device and a named layout" requires named layouts to exist.

Design: a template is a module, not a partition table

The tempting version of this is "ship some disko snippets." That earns nothing, because partitioning is never the whole decision. A LUKS+btrfs root also implies boot.initrd.systemd.enable, an unlock mechanism, boot.supportedFilesystems, services.btrfs.autoScrub, boot.tmp.useTmpfs, an ESP sized for the generation count, and a swap policy that does not contradict profiles.zram.

So: a disk template is a NixOS module that happens to contain a disko.devices attrset. If templates emit only partitions, every host re-derives the tail by hand.

Option surface

profiles.disk = {
  enable  = true;                                # explicit consent to be partitioned
  layout  = "workstation";
  device  = "/dev/disk/by-id/nvme-Samsung_SSD_...";
  swap    = "hibernate";                         # zram | disk | hibernate | none
  unlock  = "tpm2";                              # passphrase | tpm2 | ssh | none
  espSize = "2G";
};

Two rules to bake in hard:

  • by-id paths only. /dev/nvme0n1 reorders across boots and kernel versions. The module should reject a bare /dev/sdX outright; the installer's probe resolves the friendly name the user clicked into a stable one. Cheap now, unfixable later.
  • enable defaults to false, even when a device class sets a layout default. The class supplies the opinion; the host supplies the consent plus the device. Nothing partitions a disk as a side effect of importing myverse.nixosModules.server.

btrfs subvolume policy

Every btrfs template uses subvolumes — they are free and they make snapshots, quota, and (later) impermanence possible.

But the split stays deliberately shallow, because moving a file between subvolumes is a physical copy. rename(2) cannot cross a subvolume boundary, so mv degrades to copy+delete. A layout that looks tidy on paper turns routine file management into gigabytes of pointless I/O.

The rule: split along boundaries nobody moves files across.

Subvol Mount Why it is safe to split
@ / —
@home /home Files move within home, not between home and /
@nix /nix Nothing enters /nix by rename; excluded from snapshots
@log /var/log Write-heavy, should not bloat root snapshots
@snapshots /.snapshots Snapshot target, never a source of moves

Explicitly not doing: a separate @data/@media alongside @home, or splitting /var/lib out of /var. Those are exactly the boundaries users and services move files across.

Mount options: compress=zstd, noatime throughout.

Catalogue

Small on purpose — every template is a VM test to maintain.

Template For Shape
minimal router, VMs ESP + ext4 root. zram only, no swap file. Write-averse.
server server class ESP + btrfs (@, @nix, @log). Optional LUKS. No hibernate.
appliance console, tv ESP + btrfs (@, @nix, @log). No encryption. Tuned for power-cut rather than clean shutdown.
workstation desktop, tablet ESP + LUKS + btrfs (@, @home, @nix, @log, @snapshots). Swapfile sized for hibernate.

impermanent is deliberately not a template — it becomes a flag on the existing ones once a state registry exists. Naming the option now means adding it later does not rename layouts.

Decisions taken

  • Namespace profiles.disk, matching the ~20 existing NixOS modules. (myverse.* stays the flake-parts namespace.)
  • espSize defaults to 2G, flat. NixOS keeps N generations of kernel+initrd in the ESP, and lanzaboote adds signed images on top. The conventional 512M fills up and then boots fail confusingly. profiles/devices/server/default.nix already caps configurationLimit = 5; this is the other half.
  • bcachefs-tiered (nvme cache + HDD backing) is deferred to a follow-up. It is the most distinctly-myverse template and the least proven, and it roughly doubles the test surface.
  • unlock ships with all four values, but only passphrase, ssh and none are implemented in v1. tpm2 depends on secure-boot enrollment happening during provisioning, which does not exist yet. The option surface stays stable so adding it later is not a breaking change.

Traps

Swap is three decisions pretending to be one. zram (profiles.zram, currently defaults enabled), a real swap device (profiles.zswap exists for this), and hibernation (needs a resume device >= RAM, and does not work with swap-on-zram). Those two modules cannot see each other today, so swap = "hibernate" would silently do nothing on a desktop. The disk layer must own the decision and drive profiles.zram.enable from it.

Unlock policy is where disks collide with comin and secure boot. Server comin mode uses operation = "boot", so an encrypted server reboots into its new generation and stops at a passphrase prompt forever. Each template must force an explicit choice rather than defaulting into that deadlock.

These cannot be eval-tested. The existing checks are cheap drvPath evaluations, which catch nothing here — a layout can evaluate perfectly and produce an unbootable machine. disko ships makeDiskoTest; budget a real boot test per template. This is the main cost driver and the main reason the catalogue is four entries.

Scope

In: disko flake input; profiles.disk module; the four templates; per-device-class layout defaults (without enabling); raw-disko escape hatch; VM tests; docs in profiles/README.md.

Out: impermanence and the state registry; secrets/sops; the installer itself (#4); multi-disk mirroring and RAID (single boot disk plus optional single data disk in v1 — exotic pools go through the escape hatch).

Acceptance

  • profiles.disk module with the option surface above, rejecting non-by-id devices
  • Four templates, each emitting both disko.devices and its matching NixOS config
  • Swap policy coordinated with profiles.zram / profiles.zswap
  • A VM test per template that partitions, installs and boots
  • A host can be installed with disko-install --flake .#<host> from a machines/<host>/disk.nix containing nothing but a layout name and a by-id device
  • Reinstalling that host from the same file reproduces the same layout

Blocks

#4 — the installer's disk step consumes this catalogue.

## Problem myverse has no disk story at all. Grepping the tree: zero `fileSystems`, zero `swapDevices`, and no bootloader enablement anywhere except `profiles/features/secure_boot.nix` force-*disabling* systemd-boot. Every host has to hand-write partitioning, mounts, swap and boot config. `hosts/live` only escapes this because the installer-cd module supplies it. Two consequences: 1. A machine cannot be reinstalled from its own declarations. The most important thing about a host — how its storage is laid out — lives only in the residue of whatever someone typed once. 2. **#4 is blocked.** The installer's disk step has nothing to offer the user. "Pick a device and a named layout" requires named layouts to exist. ## Design: a template is a module, not a partition table The tempting version of this is "ship some disko snippets." That earns nothing, because partitioning is never the whole decision. A LUKS+btrfs root also implies `boot.initrd.systemd.enable`, an unlock mechanism, `boot.supportedFilesystems`, `services.btrfs.autoScrub`, `boot.tmp.useTmpfs`, an ESP sized for the generation count, and a swap policy that does not contradict `profiles.zram`. So: **a disk template is a NixOS module that happens to contain a `disko.devices` attrset.** If templates emit only partitions, every host re-derives the tail by hand. ## Option surface ```nix profiles.disk = { enable = true; # explicit consent to be partitioned layout = "workstation"; device = "/dev/disk/by-id/nvme-Samsung_SSD_..."; swap = "hibernate"; # zram | disk | hibernate | none unlock = "tpm2"; # passphrase | tpm2 | ssh | none espSize = "2G"; }; ``` Two rules to bake in hard: - **`by-id` paths only.** `/dev/nvme0n1` reorders across boots and kernel versions. The module should reject a bare `/dev/sdX` outright; the installer's probe resolves the friendly name the user clicked into a stable one. Cheap now, unfixable later. - **`enable` defaults to `false`,** even when a device class sets a `layout` default. The class supplies the opinion; the host supplies the consent plus the device. Nothing partitions a disk as a side effect of importing `myverse.nixosModules.server`. ## btrfs subvolume policy Every btrfs template uses subvolumes — they are free and they make snapshots, quota, and (later) impermanence possible. But the split stays deliberately shallow, because **moving a file between subvolumes is a physical copy.** `rename(2)` cannot cross a subvolume boundary, so `mv` degrades to copy+delete. A layout that looks tidy on paper turns routine file management into gigabytes of pointless I/O. The rule: **split along boundaries nobody moves files across.** | Subvol | Mount | Why it is safe to split | |---|---|---| | `@` | `/` | — | | `@home` | `/home` | Files move *within* home, not between home and `/` | | `@nix` | `/nix` | Nothing enters `/nix` by rename; excluded from snapshots | | `@log` | `/var/log` | Write-heavy, should not bloat root snapshots | | `@snapshots` | `/.snapshots` | Snapshot target, never a source of moves | Explicitly **not** doing: a separate `@data`/`@media` alongside `@home`, or splitting `/var/lib` out of `/var`. Those are exactly the boundaries users and services move files across. Mount options: `compress=zstd`, `noatime` throughout. ## Catalogue Small on purpose — every template is a VM test to maintain. | Template | For | Shape | |---|---|---| | `minimal` | router, VMs | ESP + ext4 root. zram only, no swap file. Write-averse. | | `server` | server class | ESP + btrfs (`@`, `@nix`, `@log`). Optional LUKS. No hibernate. | | `appliance` | console, tv | ESP + btrfs (`@`, `@nix`, `@log`). No encryption. Tuned for power-cut rather than clean shutdown. | | `workstation` | desktop, tablet | ESP + LUKS + btrfs (`@`, `@home`, `@nix`, `@log`, `@snapshots`). Swapfile sized for hibernate. | `impermanent` is deliberately **not** a template — it becomes a flag on the existing ones once a state registry exists. Naming the option now means adding it later does not rename layouts. ## Decisions taken - **Namespace `profiles.disk`**, matching the ~20 existing NixOS modules. (`myverse.*` stays the flake-parts namespace.) - **`espSize` defaults to 2G, flat.** NixOS keeps N generations of kernel+initrd in the ESP, and lanzaboote adds signed images on top. The conventional 512M fills up and then boots fail confusingly. `profiles/devices/server/default.nix` already caps `configurationLimit = 5`; this is the other half. - **`bcachefs-tiered` (nvme cache + HDD backing) is deferred** to a follow-up. It is the most distinctly-myverse template and the least proven, and it roughly doubles the test surface. - **`unlock` ships with all four values, but only `passphrase`, `ssh` and `none` are implemented in v1.** `tpm2` depends on secure-boot enrollment happening during provisioning, which does not exist yet. The option surface stays stable so adding it later is not a breaking change. ## Traps **Swap is three decisions pretending to be one.** zram (`profiles.zram`, currently defaults *enabled*), a real swap device (`profiles.zswap` exists for this), and hibernation (needs a resume device >= RAM, and does not work with swap-on-zram). Those two modules cannot see each other today, so `swap = "hibernate"` would silently do nothing on a desktop. The disk layer must own the decision and drive `profiles.zram.enable` from it. **Unlock policy is where disks collide with comin and secure boot.** Server comin mode uses `operation = "boot"`, so an encrypted server reboots into its new generation and stops at a passphrase prompt forever. Each template must force an explicit choice rather than defaulting into that deadlock. **These cannot be eval-tested.** The existing `checks` are cheap `drvPath` evaluations, which catch nothing here — a layout can evaluate perfectly and produce an unbootable machine. disko ships `makeDiskoTest`; budget a real boot test per template. This is the main cost driver and the main reason the catalogue is four entries. ## Scope **In:** `disko` flake input; `profiles.disk` module; the four templates; per-device-class `layout` defaults (without enabling); raw-disko escape hatch; VM tests; docs in `profiles/README.md`. **Out:** impermanence and the state registry; secrets/sops; the installer itself (#4); multi-disk mirroring and RAID (single boot disk plus optional single data disk in v1 — exotic pools go through the escape hatch). ## Acceptance - [ ] `profiles.disk` module with the option surface above, rejecting non-`by-id` devices - [ ] Four templates, each emitting both `disko.devices` and its matching NixOS config - [ ] Swap policy coordinated with `profiles.zram` / `profiles.zswap` - [ ] A VM test per template that partitions, installs and boots - [ ] A host can be installed with `disko-install --flake .#<host>` from a `machines/<host>/disk.nix` containing nothing but a layout name and a by-id device - [ ] Reinstalling that host from the same file reproduces the same layout ## Blocks #4 — the installer's disk step consumes this catalogue.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
No results found.
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Blocks
#4 Installer GUI
jmarya/myverse
Reference
jmarya/myverse#6
No description provided.