mirror of
https://github.com/freebsd/freebsd-src
synced 2024-10-07 17:10:36 +00:00
9f4ede63d2
As new compression algorithms are added to ZFS, it could be useful for people to recompress data with new algorithms. There is currently no mechanism to do this aside from copying the data manually into a new filesystem with the new algorithm enabled. This tool allows the transformation to happen through zfs send, allowing it to be done efficiently to remote systems and in an incremental fashion. A new zstream command is added that decompresses WRITE records and then recompresses them with a provided algorithm, and then re-emits the modified send stream. It may also be possible to re-compress embedded block pointers, but that was not attempted for the initial version. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Matthew Ahrens <mahrens@delphix.com> Signed-off-by: Paul Dagnelie <pcd@delphix.com> Closes #14106
200 lines
5 KiB
Groff
200 lines
5 KiB
Groff
.\"
|
|
.\" CDDL HEADER START
|
|
.\"
|
|
.\" The contents of this file are subject to the terms of the
|
|
.\" Common Development and Distribution License (the "License").
|
|
.\" You may not use this file except in compliance with the License.
|
|
.\"
|
|
.\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
|
|
.\" or https://opensource.org/licenses/CDDL-1.0.
|
|
.\" See the License for the specific language governing permissions
|
|
.\" and limitations under the License.
|
|
.\"
|
|
.\" When distributing Covered Code, include this CDDL HEADER in each
|
|
.\" file and include the License file at usr/src/OPENSOLARIS.LICENSE.
|
|
.\" If applicable, add the following below this CDDL HEADER, with the
|
|
.\" fields enclosed by brackets "[]" replaced with your own identifying
|
|
.\" information: Portions Copyright [yyyy] [name of copyright owner]
|
|
.\"
|
|
.\" CDDL HEADER END
|
|
.\"
|
|
.\" Copyright (c) 2020 by Delphix. All rights reserved.
|
|
.\"
|
|
.Dd October 4, 2022
|
|
.Dt ZSTREAM 8
|
|
.Os
|
|
.
|
|
.Sh NAME
|
|
.Nm zstream
|
|
.Nd manipulate ZFS send streams
|
|
.Sh SYNOPSIS
|
|
.Nm
|
|
.Cm dump
|
|
.Op Fl Cvd
|
|
.Op Ar file
|
|
.Nm
|
|
.Cm decompress
|
|
.Op Fl v
|
|
.Op Ar object Ns Sy \&, Ns Ar offset Ns Op Sy \&, Ns Ar type Ns ...
|
|
.Nm
|
|
.Cm redup
|
|
.Op Fl v
|
|
.Ar file
|
|
.Nm
|
|
.Cm token
|
|
.Ar resume_token
|
|
.Nm
|
|
.Cm recompress
|
|
.Op Fl l Ar level
|
|
.Ar algorithm
|
|
.
|
|
.Sh DESCRIPTION
|
|
The
|
|
.Sy zstream
|
|
utility manipulates ZFS send streams output by the
|
|
.Sy zfs send
|
|
command.
|
|
.Bl -tag -width ""
|
|
.It Xo
|
|
.Nm
|
|
.Cm dump
|
|
.Op Fl Cvd
|
|
.Op Ar file
|
|
.Xc
|
|
Print information about the specified send stream, including headers and
|
|
record counts.
|
|
The send stream may either be in the specified
|
|
.Ar file ,
|
|
or provided on standard input.
|
|
.Bl -tag -width "-D"
|
|
.It Fl C
|
|
Suppress the validation of checksums.
|
|
.It Fl v
|
|
Verbose.
|
|
Print metadata for each record.
|
|
.It Fl d
|
|
Dump data contained in each record.
|
|
Implies verbose.
|
|
.El
|
|
.Pp
|
|
The
|
|
.Nm zstreamdump
|
|
alias is provided for compatibility and is equivalent to running
|
|
.Nm
|
|
.Cm dump .
|
|
.It Xo
|
|
.Nm
|
|
.Cm token
|
|
.Ar resume_token
|
|
.Xc
|
|
Dumps zfs resume token information
|
|
.It Xo
|
|
.Nm
|
|
.Cm decompress
|
|
.Op Fl v
|
|
.Op Ar object Ns Sy \&, Ns Ar offset Ns Op Sy \&, Ns Ar type Ns ...
|
|
.Xc
|
|
Decompress selected records in a ZFS send stream provided on standard input,
|
|
when the compression type recorded in ZFS metadata may be incorrect.
|
|
Specify the object number and byte offset of each record that you wish to
|
|
decompress.
|
|
Optionally specify the compression type.
|
|
Valid compression types include
|
|
.Sy off ,
|
|
.Sy gzip ,
|
|
.Sy lz4 ,
|
|
.Sy lzjb ,
|
|
.Sy zstd ,
|
|
and
|
|
.Sy zle .
|
|
The default is
|
|
.Sy lz4 .
|
|
Every record for that object beginning at that offset will be decompressed, if
|
|
possible.
|
|
It may not be possible, because the record may be corrupted in some but not
|
|
all of the stream's snapshots.
|
|
Specifying a compression type of
|
|
.Sy off
|
|
will change the stream's metadata accordingly, without attempting decompression.
|
|
This can be useful if the record is already uncompressed but the metadata
|
|
insists otherwise.
|
|
The repaired stream will be written to standard output.
|
|
.Bl -tag -width "-v"
|
|
.It Fl v
|
|
Verbose.
|
|
Print summary of decompressed records.
|
|
.El
|
|
.It Xo
|
|
.Nm
|
|
.Cm redup
|
|
.Op Fl v
|
|
.Ar file
|
|
.Xc
|
|
Deduplicated send streams can be generated by using the
|
|
.Nm zfs Cm send Fl D
|
|
command.
|
|
The ability to send deduplicated send streams is deprecated.
|
|
In the future, the ability to receive a deduplicated send stream with
|
|
.Nm zfs Cm receive
|
|
will be removed.
|
|
However, deduplicated send streams can still be received by utilizing
|
|
.Nm zstream Cm redup .
|
|
.Pp
|
|
The
|
|
.Nm zstream Cm redup
|
|
command is provided a
|
|
.Ar file
|
|
containing a deduplicated send stream, and outputs an equivalent
|
|
non-deduplicated send stream on standard output.
|
|
Therefore, a deduplicated send stream can be received by running:
|
|
.Dl # Nm zstream Cm redup Pa DEDUP_STREAM_FILE | Nm zfs Cm receive No …
|
|
.Bl -tag -width "-D"
|
|
.It Fl v
|
|
Verbose.
|
|
Print summary of converted records.
|
|
.El
|
|
.It Xo
|
|
.Nm
|
|
.Cm recompress
|
|
.Op Fl l Ar level
|
|
.Ar algorithm
|
|
.Xc
|
|
Recompresses a send stream, provided on standard input, using the provided
|
|
algorithm and optional level, and writes the modified stream to standard output.
|
|
All WRITE records in the send stream will be recompressed, unless they fail
|
|
to result in size reduction compared to being left uncompressed.
|
|
The provided algorithm can be any valid value to the
|
|
.Nm compress
|
|
property.
|
|
Note that encrypted send streams cannot be recompressed.
|
|
.Bl -tag -width "-l"
|
|
.It Fl l Ar level
|
|
Specifies compression level.
|
|
Only needed for algorithms where the level is not implied as part of the name
|
|
of the algorithm (e.g. gzip-3 does not require it, while zstd does, if a
|
|
non-default level is desired).
|
|
.El
|
|
.El
|
|
.
|
|
.Sh EXAMPLES
|
|
Heal a dataset that was corrupted due to OpenZFS bug #12762.
|
|
First, determine which records are corrupt.
|
|
That cannot be done automatically; it requires information beyond ZFS's
|
|
metadata.
|
|
If object
|
|
.Sy 128
|
|
is corrupted at offset
|
|
.Sy 0
|
|
and is compressed using
|
|
.Sy lz4 ,
|
|
then run this command:
|
|
.Bd -literal
|
|
.No # Nm zfs Ar send Fl c Ar … | Nm zstream decompress Ar 128,0,lz4 | \
|
|
Nm zfs recv Ar …
|
|
.Ed
|
|
.Sh SEE ALSO
|
|
.Xr zfs 8 ,
|
|
.Xr zfs-receive 8 ,
|
|
.Xr zfs-send 8 ,
|
|
.Lk https://github.com/openzfs/zfs/issues/12762
|