knowledge/technology/applications/backup/borg.md
2024-08-07 15:56:01 +02:00

32 KiB
Raw Blame History

obj website repo rev
application https://www.borgbackup.org https://github.com/borgbackup/borg 2024-08-07

Borg Backup

BorgBackup (short: Borg) is a deduplicating backup program. Optionally, it supports compression and authenticated encryption.

The main goal of Borg is to provide an efficient and secure way to backup data. The data deduplication technique used makes Borg suitable for daily backups since only changes are stored. The authenticated encryption technique makes it suitable for backups to not fully trusted targets.

Usage

Borg has several commands for managing a backup repository.

Options

All Borg commands share these options:

Option Description
--info, -v, --verbose work on log level INFO
-p, --progress show progress information
--log-json Output one JSON object per log line instead of formatted text.
--bypass-lock Bypass locking mechanism
--remote-path PATH use PATH as borg executable on the remote (default: “borg”)
--rsh RSH Use this command to connect to the borg serve process (default: ssh)

borg init

This command initializes an empty repository. A repository is a filesystem directory containing the deduplicated data from zero or more archives.
Usage: borg [common options] init [options] [REPOSITORY]

Options

Option Description
-e MODE, --encryption MODE select encryption key mode (required)

Examples

# Local repository, repokey encryption, BLAKE2b (often faster, since Borg 1.1)
$ borg init --encryption=repokey-blake2 /path/to/repo

# Local repository (no encryption)
$ borg init --encryption=none /path/to/repo

# Remote repository (accesses a remote borg via ssh)
# repokey: stores the (encrypted) key into <REPO_DIR>/config
$ borg init --encryption=repokey-blake2 user@hostname:backup

# Remote repository (accesses a remote borg via ssh)
# keyfile: stores the (encrypted) key into ~/.config/borg/keys/
$ borg init --encryption=keyfile user@hostname:backup

borg create

This command creates a backup archive containing all files found while recursively traversing all paths specified. Paths are added to the archive as they are given, that means if relative paths are desired, the command has to be run from the correct directory.
Usage: borg [common options] create [options] ARCHIVE [PATH...]

Options

Option Description
-n, --dry-run do not create a backup archive
-s, --stats print statistics for the created archive
--list output verbose list of items (files, dirs, …)
--json output stats as JSON. Implies --stats.
--stdin-name NAME use NAME in archive for stdin data (default: stdin)
--stdin-user USER set user USER in archive for stdin data (default: root)
--stdin-group GROUP set group GROUP in archive for stdin data (default: wheel)
--stdin-mode M set mode to M in archive for stdin data (default: 0660)
--content-from-command interpret PATH as command and store its stdout.
--paths-from-stdin read DELIM-separated list of paths to backup from stdin. All control is external: it will back up all files given - no more, no less.
--paths-from-command interpret PATH as command and treat its output as --paths-from-stdin
--paths-delimiter DELIM set path delimiter for --paths-from-stdin and --paths-from-command (default: \n)
-e PATTERN, --exclude PATTERN exclude paths matching PATTERN
--exclude-from EXCLUDEFILE read exclude patterns from EXCLUDEFILE, one per line
--pattern PATTERN include/exclude paths matching PATTERN
--patterns-from PATTERNFILE read include/exclude patterns from PATTERNFILE, one per line
--exclude-caches exclude directories that contain a CACHEDIR.TAG file
--exclude-if-present NAME exclude directories that are tagged by containing a filesystem object with the given NAME
--keep-exclude-tags if tag objects are specified with --exclude-if-present, dont omit the tag objects themselves from the backup archive
--exclude-nodump exclude files flagged NODUMP
-x, --one-file-system stay in the same file system and do not store mount points of other file systems
--comment COMMENT add a comment text to the archive
--timestamp TIMESTAMP manually specify the archive creation date/time (UTC, yyyy-mm-ddThh:mm:ss format). Alternatively, give a reference file/directory.
-c SECONDS, --checkpoint-interval SECONDS write checkpoint every SECONDS seconds (Default: 1800)
-C COMPRESSION, --compression COMPRESSION select compression algorithm, see the output of the “borg help compression” command for details.

Examples

# Backup ~/Documents into an archive named "my-documents"
$ borg create /path/to/repo::my-documents ~/Documents

# same, but list all files as we process them
$ borg create --list /path/to/repo::my-documents ~/Documents

# Backup ~/Documents and ~/src but exclude pyc files
$ borg create /path/to/repo::my-files \
    ~/Documents                       \
    ~/src                             \
    --exclude '*.pyc'

# Backup home directories excluding image thumbnails (i.e. only
# /home/<one directory>/.thumbnails is excluded, not /home/*/*/.thumbnails etc.)
$ borg create /path/to/repo::my-files /home \
    --exclude 'sh:home/*/.thumbnails'

# Backup the root filesystem into an archive named "root-YYYY-MM-DD"
# use zlib compression (good, but slow) - default is lz4 (fast, low compression ratio)
$ borg create -C zlib,6 --one-file-system /path/to/repo::root-{now:%Y-%m-%d} /

# Backup onto a remote host ("push" style) via ssh to port 2222,
# logging in as user "borg" and storing into /path/to/repo
$ borg create ssh://borg@backup.example.org:2222/path/to/repo::{fqdn}-root-{now} /

# Backup a remote host locally ("pull" style) using sshfs
$ mkdir sshfs-mount
$ sshfs root@example.com:/ sshfs-mount
$ cd sshfs-mount
$ borg create /path/to/repo::example.com-root-{now:%Y-%m-%d} .
$ cd ..
$ fusermount -u sshfs-mount

# Make a big effort in fine granular deduplication (big chunk management
# overhead, needs a lot of RAM and disk space, see formula in internals
# docs - same parameters as borg < 1.0 or attic):
$ borg create --chunker-params buzhash,10,23,16,4095 /path/to/repo::small /smallstuff

# Backup a raw device (must not be active/in use/mounted at that time)
$ borg create --read-special --chunker-params fixed,4194304 /path/to/repo::my-sdx /dev/sdX

# Backup a sparse disk image (must not be active/in use/mounted at that time)
$ borg create --sparse --chunker-params fixed,4194304 /path/to/repo::my-disk my-disk.raw

# No compression (none)
$ borg create --compression none /path/to/repo::arch ~

# Super fast, low compression (lz4, default)
$ borg create /path/to/repo::arch ~

# Less fast, higher compression (zlib, N = 0..9)
$ borg create --compression zlib,N /path/to/repo::arch ~

# Even slower, even higher compression (lzma, N = 0..9)
$ borg create --compression lzma,N /path/to/repo::arch ~

# Only compress compressible data with lzma,N (N = 0..9)
$ borg create --compression auto,lzma,N /path/to/repo::arch ~

# Use short hostname, user name and current time in archive name
$ borg create /path/to/repo::{hostname}-{user}-{now} ~
# Similar, use the same datetime format that is default as of borg 1.1
$ borg create /path/to/repo::{hostname}-{user}-{now:%Y-%m-%dT%H:%M:%S} ~
# As above, but add nanoseconds
$ borg create /path/to/repo::{hostname}-{user}-{now:%Y-%m-%dT%H:%M:%S.%f} ~

# Backing up relative paths by moving into the correct directory first
$ cd /home/user/Documents
# The root directory of the archive will be "projectA"
$ borg create /path/to/repo::daily-projectA-{now:%Y-%m-%d} projectA

borg extract

This command extracts the contents of an archive. By default the entire archive is extracted but a subset of files and directories can be selected by passing a list of PATHs as arguments. The file selection can further be restricted by using the --exclude option.
Usage: borg [common options] extract [options] ARCHIVE [PATH...]

Options

Option Description
--list output verbose list of items (files, dirs, …)
-n, --dry-run do not actually change any files
-e PATTERN, --exclude PATTERN exclude paths matching PATTERN
--exclude-from EXCLUDEFILE read exclude patterns from EXCLUDEFILE, one per line
--pattern PATTERN include/exclude paths matching PATTERN
--patterns-from PATTERNFILE read include/exclude patterns from PATTERNFILE, one per line
--strip-components NUMBER Remove the specified number of leading path elements. Paths with fewer elements will be silently skipped.

Examples

# Extract entire archive
$ borg extract /path/to/repo::my-files

# Extract entire archive and list files while processing
$ borg extract --list /path/to/repo::my-files

# Verify whether an archive could be successfully extracted, but do not write files to disk
$ borg extract --dry-run /path/to/repo::my-files

# Extract the "src" directory
$ borg extract /path/to/repo::my-files home/USERNAME/src

# Extract the "src" directory but exclude object files
$ borg extract /path/to/repo::my-files home/USERNAME/src --exclude '*.o'

# Restore a raw device (must not be active/in use/mounted at that time)
$ borg extract --stdout /path/to/repo::my-sdx | dd of=/dev/sdx bs=10M

borg check

The check command verifies the consistency of a repository and its archives.
Usage: borg [common options] check [options] [REPOSITORY_OR_ARCHIVE]

Options

Option Description
--repository-only only perform repository checks
--archives-only only perform archives checks
--verify-data perform cryptographic archive data integrity verification (conflicts with --repository-only)
--repair attempt to repair any inconsistencies found
--save-space work slower, but using less space
--max-duration SECONDS do only a partial repo check for max. SECONDS seconds (Default: unlimited)

borg rename

This command renames an archive in the repository.
Usage: borg [common options] rename [options] ARCHIVE NEWNAME

Examples

$ borg create /path/to/repo::archivename ~
$ borg list /path/to/repo
archivename                          Mon, 2016-02-15 19:50:19

$ borg rename /path/to/repo::archivename newname
$ borg list /path/to/repo
newname                              Mon, 2016-02-15 19:50:19

borg list

This command lists the contents of a repository or an archive.
Usage: borg [common options] list [options] [REPOSITORY_OR_ARCHIVE] [PATH...]

Options

Option Description
--consider-checkpoints Show checkpoint archives in the repository contents list (default: hidden).
--short only print file/directory names, nothing else
--format FORMAT specify format for file or archive listing (default for files: {mode} {user:6} {group:6} {size:8} {mtime} {path}{extra}{NL}; for archives: {archive:<36} {time} [{id}]{NL})
--json Only valid for listing repository contents. Format output as JSON. The form of --format is ignored, but keys used in it are added to the JSON output. Some keys are always present. Note: JSON can only represent text. A “barchive” key is therefore not available.
--json-lines Only valid for listing archive contents. Format output as JSON Lines. The form of --format is ignored, but keys used in it are added to the JSON output. Some keys are always present. Note: JSON can only represent text. A “bpath” key is therefore not available.
-P PREFIX, --prefix PREFIX only consider archive names starting with this prefix. (deprecated)
-a GLOB, --glob-archives GLOB only consider archive names matching the glob. sh: rules apply (without actually using the sh: prefix), see “borg help patterns”.
--sort-by KEYS Comma-separated list of sorting keys; valid keys are: timestamp, archive, name, id; default is: timestamp
--first N consider first N archives after other filters were applied
--last N consider last N archives after other filters were applied
-e PATTERN, --exclude PATTERN exclude paths matching PATTERN
--exclude-from EXCLUDEFILE read exclude patterns from EXCLUDEFILE, one per line
--pattern PATTERN include/exclude paths matching PATTERN
--patterns-from PATTERNFILE read include/exclude patterns from PATTERNFILE, one per line

Examples

$ borg list /path/to/repo
Monday                               Mon, 2016-02-15 19:15:11
repo                                 Mon, 2016-02-15 19:26:54
root-2016-02-15                      Mon, 2016-02-15 19:36:29
newname                              Mon, 2016-02-15 19:50:19
...

$ borg list /path/to/repo::root-2016-02-15
drwxr-xr-x root   root          0 Mon, 2016-02-15 17:44:27 .
drwxrwxr-x root   root          0 Mon, 2016-02-15 19:04:49 bin
-rwxr-xr-x root   root    1029624 Thu, 2014-11-13 00:08:51 bin/bash
lrwxrwxrwx root   root          0 Fri, 2015-03-27 20:24:26 bin/bzcmp -> bzdiff
-rwxr-xr-x root   root       2140 Fri, 2015-03-27 20:24:22 bin/bzdiff
...

$ borg list /path/to/repo::root-2016-02-15 --pattern "- bin/ba*"
drwxr-xr-x root   root          0 Mon, 2016-02-15 17:44:27 .
drwxrwxr-x root   root          0 Mon, 2016-02-15 19:04:49 bin
lrwxrwxrwx root   root          0 Fri, 2015-03-27 20:24:26 bin/bzcmp -> bzdiff
-rwxr-xr-x root   root       2140 Fri, 2015-03-27 20:24:22 bin/bzdiff
...

$ borg list /path/to/repo::archiveA --format="{mode} {user:6} {group:6} {size:8d} {isomtime} {path}{extra}{NEWLINE}"
drwxrwxr-x user   user          0 Sun, 2015-02-01 11:00:00 .
drwxrwxr-x user   user          0 Sun, 2015-02-01 11:00:00 code
drwxrwxr-x user   user          0 Sun, 2015-02-01 11:00:00 code/myproject
-rw-rw-r-- user   user    1416192 Sun, 2015-02-01 11:00:00 code/myproject/file.ext
-rw-rw-r-- user   user    1416192 Sun, 2015-02-01 11:00:00 code/myproject/file.text
...

borg diff

This command finds differences (file contents, user/group/mode) between archives.
Usage: borg [common options] diff [options] REPO::ARCHIVE1 ARCHIVE2 [PATH...]

Examples

$ borg diff testrepo::archive1 archive2
[-rw-r--r-- -> -rwxr-xr-x] file1
   +135 B    -252 B file2

$ borg diff testrepo::archive2 archive3
    +17 B      -5 B file1
added           0 B file4
removed         0 B file3

$ borg diff testrepo::archive1 archive3
    +17 B      -5 B [-rw-r--r-- -> -rwxr-xr-x] file1
   +135 B    -252 B file2
added           0 B file4
removed         0 B file3

$ borg diff --json-lines testrepo::archive1 archive3
{"path": "file1", "changes": [{"type": "modified", "added": 17, "removed": 5}, {"type": "mode", "old_mode": "-rw-r--r--", "new_mode": "-rwxr-xr-x"}]}
{"path": "file2", "changes": [{"type": "modified", "added": 135, "removed": 252}]}
{"path": "file4", "changes": [{"type": "added", "size": 0}]}
{"path": "file3", "changes": [{"type": "removed", "size": 0}]}

borg delete

This command deletes an archive from the repository or the complete repository.
Usage: borg [common options] delete [options] [REPOSITORY_OR_ARCHIVE] [ARCHIVE...]

Options

Option Description
-n, --dry-run do not change repository
--list output verbose list of archives
-s, --stats print statistics for the deleted archive

Examples

# delete a single backup archive:
$ borg delete /path/to/repo::Monday
# actually free disk space:
$ borg compact /path/to/repo

# delete all archives whose names begin with the machine's hostname followed by "-"
$ borg delete --glob-archives '{hostname}-*' /path/to/repo

# delete all archives whose names contain "-2012-"
$ borg delete --glob-archives '*-2012-*' /path/to/repo

# see what would be deleted if delete was run without --dry-run
$ borg delete --list --dry-run -a '*-May-*' /path/to/repo

borg prune

The prune command prunes a repository by deleting all archives not matching any of the specified retention options.
Usage: borg [common options] prune [options] [REPOSITORY]

Options

Option Description
-n, --dry-run do not change repository
--force force pruning of corrupted archives, use --force --force in case --force does not work.
-s, --stats print statistics for the deleted archive
--list output verbose list of archives it keeps/prunes
--keep-within INTERVAL keep all archives within this time interval
--keep-last, --keep-secondly number of secondly archives to keep
--keep-minutely number of minutely archives to keep
-H, --keep-hourly number of hourly archives to keep
-d, --keep-daily number of daily archives to keep
-w, --keep-weekly number of weekly archives to keep
-m, --keep-monthly number of monthly archives to keep
-y, --keep-yearly number of yearly archives to keep

Examples

# Keep 7 end of day and 4 additional end of week archives.
# Do a dry-run without actually deleting anything.
$ borg prune -v --list --dry-run --keep-daily=7 --keep-weekly=4 /path/to/repo

# Same as above but only apply to archive names starting with the hostname
# of the machine followed by a "-" character:
$ borg prune -v --list --keep-daily=7 --keep-weekly=4 --glob-archives='{hostname}-*' /path/to/repo
# actually free disk space:
$ borg compact /path/to/repo

# Keep 7 end of day, 4 additional end of week archives,
# and an end of month archive for every month:
$ borg prune -v --list --keep-daily=7 --keep-weekly=4 --keep-monthly=-1 /path/to/repo

# Keep all backups in the last 10 days, 4 additional end of week archives,
# and an end of month archive for every month:
$ borg prune -v --list --keep-within=10d --keep-weekly=4 --keep-monthly=-1 /path/to/repo

borg compact

This command frees repository space by compacting segments.
Usage: borg [common options] compact [options] [REPOSITORY]

Examples

# compact segments and free repo disk space
$ borg compact /path/to/repo

# same as above plus clean up 17byte commit-only segments
$ borg compact --cleanup-commits /path/to/repo

borg info

This command displays detailed information about the specified archive or repository.
Usage: borg [common options] info [options] [REPOSITORY_OR_ARCHIVE]

Options

Option Description
--json format output as JSON

borg mount

This command mounts an archive as a FUSE filesystem. This can be useful for browsing an archive or restoring individual files. Unless the --foreground option is given the command will run in the background until the filesystem is umounted.
Usage: borg [common options] mount [options] REPOSITORY_OR_ARCHIVE MOUNTPOINT [PATH...]

Options

Option Description
-f, --foreground stay in foreground, do not daemonize
-o Extra mount options
--numeric-ids use numeric user and group identifiers from archive(s)

Examples

# Mounting the repository shows all archives.
# Archives are loaded lazily, expect some delay when navigating to an archive
# for the first time.
$ borg mount /path/to/repo /tmp/mymountpoint
$ ls /tmp/mymountpoint
root-2016-02-14 root-2016-02-15
$ borg umount /tmp/mymountpoint

# Mounting a specific archive is possible as well.
$ borg mount /path/to/repo::root-2016-02-15 /tmp/mymountpoint
$ ls /tmp/mymountpoint
bin  boot  etc      home  lib  lib64  lost+found  media  mnt  opt
root  sbin  srv  tmp  usr  var
$ borg umount /tmp/mymountpoint

borg unmount

This command un-mounts a FUSE filesystem that was mounted with borg mount.
Usage: borg [common options] umount [options] MOUNTPOINT

borg key change-passphrase

The key files used for repository encryption are optionally passphrase protected. This command can be used to change this passphrase.
Usage: borg [common options] key change-passphrase [options] [REPOSITORY]

borg serve

This command starts a repository server process. This command is usually not used manually.
Usage: borg [common options] serve [options]

Options

Option Description
--restrict-to-path PATH restrict repository access to PATH. Can be specified multiple times to allow the client access to several directories. Access to all sub-directories is granted implicitly; PATH doesnt need to directly point to a repository.
--restrict-to-repository PATH restrict repository access. Only the repository located at PATH (no sub-directories are considered) is accessible. Can be specified multiple times to allow the client access to several repositories. Unlike --restrict-to-path sub-directories are not accessible; PATH needs to directly point at a repository location. PATH may be an empty directory or the last element of PATH may not exist, in which case the client may initialize a repository there.
--append-only only allow appending to repository segment files. Note that this only affects the low level structure of the repository, and running delete or prune will still be allowed.
--storage-quota QUOTA Override storage quota of the repository (e.g. 5G, 1.5T). When a new repository is initialized, sets the storage quota on the new repository as well. Default: no quota.