These are based on definitions added to binutils' libctf. Specifically:
- Type IDs are now encoded in 32 bits rather than 16, changing the
layout of ctf_type_t, ctf_array_t, ctf_member_t and ctf_lmember_t.
- Type info is encoded in 32 bits rather than 16. The type "kind" is
extended from 5 bits to 6, and the type "vlen" is extended from 10
bits to 25.
The main upside is that we remove the current limit, imposed by CTFv2,
of 2^{15} distinct types in the main kernel executable. Other limits,
such as that on the number of elements in an enum, imposed by the vlen
limit, are also raised.
This change adds v2 and v3 flavours of macros and type definitions which
differ between the two versions. Compatibility is preserved for now by
having generic names refer to the v2 definitions, so, e.g., ctf_type_t
is still a v2 type.
No functional change intended.
Reviewed by: Domagoj Stolfa
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34360
Use it instead of the existing ctf.h from OpenSolaris. This makes it
easier to use CTF in the core kernel, and to extend the CTF format to
support wider type IDs.
The imported ctf.h is modified to depend only on _types.h, and also to
provide macros which use the "parent" bit of a type ID to refer to types
in a parent CTF container.
No functional change intended.
Reviewed by: Domagoj Stolfa, emaste
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34358
It is unused on FreeBSD and complicates some efforts to modify the CTF
format to permit wider type IDs, so remove it. No functional change
intended.
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
CTFv1 was obsolete before libctf was imported into FreeBSD, and
ctfconvert/ctfmerge can emit only CTFv2. Make ctf.h a bit easier to
maintain by ripping v1 support out. No functional change intended.
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
For consistency with other CTF toolchain code. No functional change
intended.
Fixes: 105fd928b0 ("libctf: Improve check for duplicate SOU definitions in ctf_add_type()")
MFC after: 1 week
This call was added back in the early days of dtrace porting and
no one knows why anymore. The extra flushing causes lots of
unnecessary CPU overhead when a script produces lots of output,
as well as easily losing output because the command can't keep up.
Sponsored by: Netflix
Reviewed by: imp, markj
Differential Revision: https://reviews.freebsd.org/D34216
It was set to the start of the buffer and that can be different from the
start of teh first record because of a misalignment.
This change follows the example of dt_realloc_buf().
Reviewed by: tsoome, markj
MFC after: 4 weeks
Differential Revision: https://reviews.freebsd.org/D33649
Members with such a type will legitimately have a size of zero, so don't
emit a warning.
PR: 260818
Reviewed by: bz
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33700
getf() on FreeBSD calls _sx_slock(), _sx_sunlock() and fget_locked().
Furthermore, it does not set the per-core fault flag, meaning it
usually ends up in a double fault panic once getf() does get called,
especially from fbt.
Reviewing the DTrace Toolkit + a number of other scripts scattered
around FreeBSD, I have not been able to find one use of getf(). Given
how broken the implementation currently is, we disable it until it
can be implemented properly.
Also comment out a test in aggs/tst.subr.d for getf().
Reviewed by: markj
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D33378
When copying a struct or union from one CTF container to another,
ctf_add_type() checks whether it matches an existing type in the
destination container. It does so by looking for a type with the same
name and kind as the new type, and if one exists, it iterates over all
members of the source type and checks whether a member with matching
name and offset exists in the matched destination type. This can
produce false positives, for example because member types are not
compared, but this is not expected to arise in practice. If the match
fails, ctf_add_type() returns an error.
The procedure used for member comparison breaks down in the face of
anonymous struct and union members. ctf_member_iter() visits each
member in the source definition and looks up the corresponding member in
the desination definition by name using ctf_member_info(), but this
function will descend into anonymous members and thus fail to match.
Fix the problem by introducing a custom comparison routine which does
not assume member names are unique. This should also be faster for
types with many members; in the previous scheme, membcmp() would perform
a linear scan of the desination type's members to perform a lookup by
name. The new routine steps through the members of both types in a
single loop.
PR: 258763
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
In dt_cc.c when the provider is an empty string, accessing
strlen(pdp->dtpd_provider) - 1 will result in a pdp->dtpd_provider[-1]
access.
Similarly, in dt_ident.c, if p2 is a NULL pointer, doing a p2++ on it is
undefined behaviour.
Reviewed by: markj
MFC after: 1 week
Sponsored by: Google
Differential Revision: https://reviews.freebsd.org/D30778
When prefixes such as struct, union, etc. are compared with the current
type (e.g. struct foo), a comparison is made with the prefix. The code
currently assumes that every type is a valid C type with a prefix,
however at times, garbage ends up in this function causing an
unpredictable crash with DTrace due to the isspace(*p) call or
subsequent calls. An example that I've seen of this is the letter 's'
being passed in, comparing true with struct as the comparison size was
(q - p) == 1, but then we increment p with the length of "struct",
resulting in an out of bounds read.
Reviewed by: markj
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D29435
After commit 8ba333e02e ("libdtrace: Stop relying on lex
compatibility"), there have been several reports of incremental
buildworlds failing since make does not know that dt_lex.c needs to be
regenerated, and I want to avoid this when merging to stable/13.
MFC with: 8ba333e02e
It does not appear to be required, and as of commit 6b7e592c21
("lex: Do not let input() return 0 when end-of-file is reached") it
causes input to return 0 instead of EOF when end-of-input is reached.
PR: 253440
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Before we did not handle weak symbols correctly, sometimes resulting in
link errors from dtrace -G when processing object files where functions
with weak aliases contain USDT probes.
Reported by: rlibby
Tested by: rlibby
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Note that the public documentation on dtrace.org fails to mention %T and
incorrectly documents %Y. The latter actually uses format "%Y %b %e %T"
where %b is always in C locale.
Discussed with: markj
MFC after: 1 month
Sponsored by: Panzura
encode 0-length (i.e. "") structure and union member names as offset 0.
This ensures that we don't confuse other parts of the CTF code which
expect this encoding.
This resolves a Dtrace error resolving members of anonymous structs/unions
within the (struct mbuf) type which some users were seeing after r366908.
While here, update the code in ctf_add_generic() to encode 0-length type
names as offset 0.
Reviewed by: markj
MFC after: 2 weeks
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D27246
always copy intrinsic data types before copying bitfields which are
based on those types. This ensures the type ordering in the destination
CTF container matches the assumption made elsewhere in the CTF code
that instrinsic data types will always appear before bitfields based on
those types.
This resolves the following error message some users have seen after
r366908:
"/usr/lib/dtrace/ipfw.d", line 121: failed to copy type of 'ip6p':
Conflicting type is already defined
Reviewed by: markj
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D27213
The primary benefit is maintaining a completely shared
code base with the community allowing FreeBSD to receive
new features sooner and with less effort.
I would advise against doing 'zpool upgrade'
or creating indispensable pools using new
features until this change has had a month+
to soak.
Work on merging FreeBSD support in to what was
at the time "ZFS on Linux" began in August 2018.
I first publicly proposed transitioning FreeBSD
to (new) OpenZFS on December 18th, 2018. FreeBSD
support in OpenZFS was finally completed in December
2019. A CFT for downstreaming OpenZFS support in
to FreeBSD was first issued on July 8th. All issues
that were reported have been addressed or, for
a couple of less critical matters there are
pull requests in progress with OpenZFS. iXsystems
has tested and dogfooded extensively internally.
The TrueNAS 12 release is based on OpenZFS with
some additional features that have not yet made
it upstream.
Improvements include:
project quotas, encrypted datasets,
allocation classes, vectorized raidz,
vectorized checksums, various command line
improvements, zstd compression.
Thanks to those who have helped along the way:
Ryan Moeller, Allan Jude, Zack Welch, and many
others.
Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D25872
To define USDT probes, dtrace -G makes use of relocations for undefined
symbols: the target address is overwritten with NOPs and the location is
recorded in the DOF section of the output object file. To avoid link
errors, the original relocation is destroyed. However, this means that
the same input object file cannot be processed multiple times, as
happens during incremental rebuilds. Instead, only set the relocation
type to NONE, so that all information required to reconstruct USDT
probes is preserved.
Reported by: bdrewery
MFC after: 3 weeks
Sponsored by: The FreeBSD Foundation
This file is too complicated as it is and has diverged a fair bit from
illumos due to toolchain differences, so just drop unused code
(including SPARC support).
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
ZFS unmounts a dataset while receiving into it and remounts it afterwards.
But if ZFS is resuming an incomplete receive, it screws up and ends up with
a dataset that is mounted, but returns EIO for every access. This commit
fixes that condition.
While the vulnerable code also exists in OpenZFS, the problem is not
reproducible there. Apparently OpenZFS doesn't unmount the destination
dataset during receive, like FreeBSD does.
PR: 248606
Reviewed by: mmacy
MFC after: 2 weeks
Sponsored by: Axcient
Differential Revision: https://reviews.freebsd.org/D26034
When zsh runs in POSIX sh mode it does not support the -e flag to echo.
Use printf instead of echo to avoid the "-e" characters being printed.
Obtained from: CheriBSD
Reviewed By: markj
Differential Revision: https://reviews.freebsd.org/D26026
Some of the scripts used for libdtrace invoke nawk instead of awk
(for example cddl/contrib/opensolaris/lib/libdtrace/common/mknames.sh).
When bootstrapping all tools, we get the nawk -> awk link while building
usr.bin/awk, but when linking/copying the dependencies from the host we
were only adding awk but not nawk.
This was silently generating invalid files when building libdtrace with
BUILD_WITH_STRICT_TMPPATH=1 since those scripts invoke nawk instead of
awk. In addition to adding the missing link this commit also adds
set -e to those scripts to catch errors like this in the future.
Reviewed By: markj, emaste
Differential Revision: https://reviews.freebsd.org/D26025
This does not appear to matter on FreeBSD or Linux, but when building an
amd64 kernel on macOS I was seeing infinite loops in ctfmerge.
It turns out the loop in wip_save_work() was looping forever due to
pthread_cond_wait() always returning -EINVAL.
Reviewed By: markj, brooks
Differential Revision: https://reviews.freebsd.org/D25973
We are building new bootonce mechanism (previously zfs bootnext) and it is
based on this OpenZFS change. Since this patch is nicely self contained,
I am commiting it as is, and we can stack our changes.
Original patch description follows:
Modern bootloaders leverage data stored in the root filesystem to
enable some of their powerful features. GRUB specifically has a grubenv
file which can store large amounts of configuration data that can be
read and written at boot time and during normal operation. This allows
sysadmins to configure useful features like automated failover after
failed boot attempts. Unfortunately, due to the Copy-on-Write nature
of ZFS, the standard behavior of these tools cannot handle writing to
ZFS files safely at boot time. We need an alternative way to store
data that allows the bootloader to make changes to the data.
This work is very similar to work that was done on Illumos to enable
similar functionality in the FreeBSD bootloader. This patch is different
in that the data being stored is a raw grubenv file; this file can store
arbitrary variables and values, and the scripting provided by grub is
powerful enough that special structures are not required to implement
advanced behavior.
We repurpose the second padding area in each label to store the grubenv
file, protected by an embedded checksum. We add two ioctls to get and
set this data, and libzfs_core and libzfs functions to access them more
easily. There are no direct command line interfaces to these functions;
these will be added directly to the bootloader utilities.
Reviewed-by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Paul Dagnelie <pcd@delphix.com>
Closes#10009
Obtained from: OpenZFS
Sponsored by: Netflix, Klara Inc.
In original implementation, zpool history will read the whole history
before printing anything, causing memory usage goes unbounded. We fix
this by breaking it into read-print iterations.
Reviewed-by: Tom Caputi <tcaputi@datto.com>
Reviewed-by: Matt Ahrens <matt@delphix.com>
Reviewed-by: Igor Kozhukhov <igor@dilos.org>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Chunwei Chen <david.chen@nutanix.com>
Closes#9516
Note, this change changes the libzfs.so ABI by modifying the prototype
of zpool_get_history(). Since libzfs is effectively private to the base
system it is anticipated that this will not be a problem.
PR: 247557
Obtained from: OpenZFS
Reported and tested by: Sam Vaughan <samjvaughan@gmail.com>
Discussed with: freqlabs
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D25745openzfs/zfs@7125a109dc
Since r304321 (-current: Aug 18, 2016) and r328866 (stable/11: Feb 5, 2018)
the FreeBSD loader has supported reading from datasets with the
large_blocks feature active.
PR: 247992
Reported by: Anton Saietskii <vsasjason@gmail.com>
MFC after: 2 weeks
Sponsored by: Klara Inc.
This pattern is used in callbacks with void * data arguments and seems
both relatively uncommon and relatively harmless. Silence the warning
by casting through uintptr_t.
This warning is on by default in Clang 11.
MFC after: 3 days
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D24425
We want newer versions of libzfs_core to run against an existing
zfs kernel module (i.e. a deferred reboot or module reload after
an update).
Programmatically document, via a zfs_ioc_key_t, the valid arguments
for the ioc commands that rely on nvpair input arguments (i.e. non
legacy commands from libzfs_core). Automatically verify the expected
pairs before dispatching a command.
This initial phase focuses on the non-legacy ioctls. A follow-on
change can address the legacy ioctl input from the zfs_cmd_t.
The zfs_ioc_key_t for zfs_keys_channel_program looks like:
static const zfs_ioc_key_t zfs_keys_channel_program[] = {
{"program", DATA_TYPE_STRING, 0},
{"arg", DATA_TYPE_UNKNOWN, 0},
{"sync", DATA_TYPE_BOOLEAN_VALUE, ZK_OPTIONAL},
{"instrlimit", DATA_TYPE_UINT64, ZK_OPTIONAL},
{"memlimit", DATA_TYPE_UINT64, ZK_OPTIONAL},
};
Introduce four input errors to identify specific input failures
(in addition to generic argument value errors like EINVAL, ERANGE,
EBADF, and E2BIG).
ZFS_ERR_IOC_CMD_UNAVAIL the ioctl number is not supported by kernel
ZFS_ERR_IOC_ARG_UNAVAIL an input argument is not supported by kernel
ZFS_ERR_IOC_ARG_REQUIRED a required input argument is missing
ZFS_ERR_IOC_ARG_BADTYPE an input argument has an invalid type
Reviewed by: allanjude
Obtained from: OpenZFS
Sponsored by: Netflix, Klara Inc.
Differential Revision: https://reviews.freebsd.org/D25393
Keep link_map l_addr binary layout compatible, rename l_addr to l_base
where rtld returns map base. Provide relocbase in newly added l_addr.
This effectively reverts the patch to the initial version of D24918.
Reported by: antoine (portmgr)
Reviewed by: jhb, markj
Tested by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D24946
And that should work even (especially) if there is no matching user or
group name. This change allows to see and modify delegations for
deleted groups and users.
The change is originally by Xin Li.
illumos report: https://www.illumos.org/issues/6037
OpenZFS (ZoL) PR: https://github.com/openzfs/zfs/pull/10280
Obtained from: delphij
MFC after: 2 weeks
Currently when the dataset is in use we can't receive snapshots.
zfs send test/1@asd | zfs recv -FM test/2
cannot unmount '/test/2': Device busy
This commits add option 'M' which attempts to forcibly unmount the
dataset. Thanks to this we can enforce receiving snapshots in a
single step.
Note that this functionality is not supported on Linux because the
VFS will prevent active mounted filesystems from being unmounted,
even with the force option. This is the intended VFS behavior.
Discussed-with: Pawel Jakub Dawidek <pjd@FreeBSD.org>
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Allan Jude <allanjude@freebsd.org>
Differential Revision: https://reviews.freebsd.org/D22306openzfs/zfs@a57d3d45d6
zfs create, receive and rename can bypass this hierarchy rule. Update
both userland and kernel module to prevent this issue and use pyzfs
unit tests to exercise the ioctls directly.
Note: this commit slightly changes zfs_ioc_create() ABI. This allow to
differentiate a generic error (EINVAL) from the specific case where we
tried to create a dataset below a ZVOL (ZFS_ERR_WRONG_PARENT).
Reviewed-by: Paul Dagnelie <pcd@delphix.com>
Reviewed-by: Matt Ahrens <mahrens@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Tom Caputi <tcaputi@datto.com>
Signed-off-by: loli10K <ezomori.nozomu@gmail.com>
Approved by: mav (mentor)
MFC after: 2 weeks
Sponsored by: iXsystems, Inc.
openzfs/zfs@d8d418ff0c
We've observed that on some highly fragmented pools, most metaslab
allocations are small (~2-8KB), but there are some large, 128K
allocations. The large allocations are for ZIL blocks. If there is a
lot of fragmentation, the large allocations can be hard to satisfy.
The most common impact of this is that we need to check (and thus load)
lots of metaslabs from the ZIL allocation code path, causing sync writes
to wait for metaslabs to load, which can take a second or more. In the
worst case, we may not be able to satisfy the allocation, in which case
the ZIL will resort to txg_wait_synced() to ensure the change is on
disk.
To provide a workaround for this, this change adds a tunable that can
reduce the size of ZIL blocks.
External-issue: DLPX-61719
Reviewed-by: George Wilson <george.wilson@delphix.com>
Reviewed-by: Paul Dagnelie <pcd@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Matthew Ahrens <mahrens@delphix.com>
Closes#8865openzfs/zfs@b8738257c2
MFC after: 2 weeks
This was the intent of the existing code, but instead it would
unconditionally load dtraceall.ko because of a stale errno value.
Reported by: pho
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
PPC64 ELFv2 acts like a "normal" platform in that it no longer needs
function descriptors. So, ensure we are only enabling them on ELFv1.
Additionally, ELFv2 requires that the ELF header have a nonzero e_flags,
so ensure that the synthesized ELF header in dt_link.c is setting it.
Reviewed by: jhibbits, markj
Approved by: gnn
Differential Revision: https://reviews.freebsd.org/D22403
By default, zpools may not be backed by zvols (that can be changed with the
"vfs.zfs.vol.recursive" sysctl). When that sysctl is set to 0, the kernel
does not attempt to read zvols when looking for vdevs. But the zpool command
still does. This change brings the zpool command into line with the kernel's
behavior. It speeds "zpool import" when an already imported pool has many
zvols, or a zvol with many snapshots.
PR: 241083
Reported by: Martin Birgmeier <d8zNeCFG@aon.at>
Reviewed by: mav, Ryan Moeller <ryan@freqlabs.com>
MFC after: 2 weeks
Sponsored by: Axcient
Differential Revision: https://reviews.freebsd.org/D22077
This will be used in libbe in place of the internal zmount(); libbe only
wants to be able to mount a dataset at an arbitrary mountpoint without
altering dataset/pool properties. The natural way to do this in a portable
way is by creating a zfs_mount_at() interface that's effectively zfs_mount()
+ a mountpoint parameter. zfs_mount() is now a light wrapper around the new
method.
The interface and implementation have already been accepted into ZFS On
Linux, and the next commit to switch libbe() over to this new interface will
solve the last compatibility issue with ZoL. The next sysutils/openzfs
rebase against ZoL should be able to build libbe/bectl with only minor
adjustments to build glue.
Reviewed by: Ryan Moeller <ryan freqlabs com>
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D23132
Previously libdtrace used ftok(3), which hashes the inode number of the
input object file. To increase reproducibility of builds that embed
USDT probes, include a hash of the object file path in the symbol name
instead.
Reported and tested by: bdrewery
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
arm64 is still lacking a fasttrap implementation, which is required to
actually enable userland probes, but this at least allows USDT probes to
be linked into userland applications.
Submitted by: Klaus Küchemann <maciphone2@googlemail.com> (original)
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D22360
gcc9 grew a new warning for unbounded allocas, such as the one in
dt_options_load. Remove both uses of alloca in dt_options.c.
Reviewed by: markj
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D22880
illumos/illumos-gate@555d674d5d555d674d5dhttps://www.illumos.org/issues/10592
This is a collection of recent fixes from ZoL:
8eef997679 Error path in metaslab_load_impl() forgets to drop ms_sync_lock
928e8ad47d Introduce auxiliary metaslab histograms
425d3237ee Get rid of space_map_update() for ms_synced_length
6c926f426a Simplify log vdev removal code
21e7cf5da8 zdb -L should skip leak detection altogether
df72b8bebe Rename range_tree_verify to range_tree_verify_not_present
75058f3303 Remove unused vdev_t fields
Portions contributed by: Jerry Jelinek <jerry.jelinek@joyent.com>
Author: Serapheim Dimitropoulos <serapheim@delphix.com>
MFC after: 4 weeks
illumos/illumos-gate@663207adb1663207adb1
10601 Pool allocation classes
https://www.illumos.org/issues/10601
illumos port of ZoL Pool allocation classes. Includes at least these two
commits:
441709695 Pool allocation classes misplacing small file blocks
cc99f275a Pool allocation classes
10757 Add -gLp to zpool subcommands for alt vdev names
https://www.illumos.org/issues/10757
Port from ZoL of
d2f3e292d Add -gLp to zpool subcommands for alt vdev names
Note that a subsequent ZoL commit changed -p to -P
a77f29f93 Change full path subcommand flag from -p to -P
Portions contributed by: Jerry Jelinek <jerry.jelinek@joyent.com>
Portions contributed by: Håkan Johansson <f96hajo@chalmers.se>
Portions contributed by: Richard Yao <ryao@gentoo.org>
Portions contributed by: Chunwei Chen <david.chen@nutanix.com>
Portions contributed by: loli10K <ezomori.nozomu@gmail.com>
Author: Don Brady <don.brady@delphix.com>
11541 allocation_classes feature must be enabled to add log device
illumos/illumos-gate@c1064fd7cec1064fd7cehttps://www.illumos.org/issues/11541
After the allocation_classes feature was integrated, one can no longer add a
log device to a pool unless that feature is enabled. There is an explicit check
for this, but it is unnecessary in the case of log devices, so we should handle
this better instead of forcing the feature to be enabled.
Author: Jerry Jelinek <jerry.jelinek@joyent.com>
FreeBSD notes.
I faithfully added the new -g, -L, -P flags, but only -g does something:
vdev GUIDs are displayed instead of device names. -L, resolve symlinks,
and -P, display full disk paths, do nothing at the moment.
The use of special vdevs is backward compatible for read-only access, so
root pools should be bootable, but exercise caution.
MFC after: 4 weeks
Somehow I managed to commit the manual page with unresolved conflicts in
it.
While here, I also replaced .sp with .Pp.
MFC after: 3 weeks
X-MFC with: r354804
The DOF file output by dtrace -A contains only the loadable sections.
However, as it was created by a call to dtrace_dof_create() without
flags, the original DOF was created with the loadable sections. The
result is that the DOF includes the section headers for the unloadable
sections (COMMENTS and UTSNAME) without these sections actually being
present. This is inconsistent.
A simple change to anon_prog() ensures that the missing sections are
present in the outputted DOF. Alternatively, the call to
dtrace_dof_create() could pass the DTRACE_D_STRIP flag stripping out the
loadable sections. As the unloadable sections contain info useful for
debugging purposes they haven't been stripped.
Submitted by: Graeme Jenkinson <graeme.jenkinson@cl.cam.ac.uk>
MFC after: 1 week
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D21875
10499 Multi-modifier protection (MMP)
illumos/illumos-gate@e0f1c0afa4e0f1c0afa4https://www.illumos.org/issues/10499
Port the following ZFS commits from ZoL to illumos.
379ca9cf2 Multi-modifier protection (MMP)
bbffb59ef Fix multihost stale cache file import
0d398b256 Do not initiate MMP writes while pool is suspended
10701 Correct lock ASSERTs in vdev_label_read/write
illumos/illumos-gate@58447f688d58447f688dhttps://www.illumos.org/issues/10701
Port of ZoL commit:
0091d66f4e Correct lock ASSERTs in vdev_label_read/write
At a minimum, this fixes a blown assert during an MMP test run when running on
a DEBUG build.
11770 additional mmp fixes
illumos/illumos-gate@4348eb90124348eb9012https://www.illumos.org/issues/11770
Port a few additional MMP fixes from ZoL that came in after our
initial MMP port.
4ca457b065 ZTS: Fix mmp_interval failure
ca95f70dff zpool import progress kstat
(only minimal changes from above can be pulled in right now)
060f0226e6 MMP interval and fail_intervals in uberblock
Note from the committer (me).
I do not have any use for this feature and I have not tested it. I only
did smoke testing with multihost=off.
Please be aware.
I merged the code only to make future merges easier.
Portions contributed by: Jerry Jelinek <jerry.jelinek@joyent.com>
Portions contributed by: Tim Chase <tim@chase2k.com>
Portions contributed by: sanjeevbagewadi <sanjeev.bagewadi@gmail.com>
Portions contributed by: John L. Hammond <john.hammond@intel.com>
Portions contributed by: Giuseppe Di Natale <dinatale2@llnl.gov>
Portions contributed by: Prakash Surya <surya1@llnl.gov>
Portions contributed by: Brian Behlendorf <behlendorf1@llnl.gov>
Author: Olaf Faaland <faaland1@llnl.gov>
MFC after: 4 weeks
This change is based on
r354380 8899 zpool list property documentation doesn't match actual behaviour
There is no "used" pool property, "alloc" is actually spelled
"allocated".
MFC after: 5 days
illumos/illumos-gate@9c2acf00e29c2acf00e2https://www.illumos.org/issues/10554
During the port of MMP (illumos bug 10499) from ZoL, I found this
earlier ZoL project is a prerequisite. Here is the original
description. This addition will enable us to sync an open TXG to the
main pool on demand. The functionality is similar to 'sync(2)' but
'zpool sync' will return when data has hit the main storage instead of
potentially just the ZIL as is the case with the 'sync(2)' cmd.
Portions contributed by: Jerry Jelinek <jerry.jelinek@joyent.com>
Author: Alek Pinchuk <apinchuk@datto.com>
MFC after: 3 weeks
Relnotes: possibly
This patch modifies the zfs_ioc_snapshot_list_next() ioctl to enable it
to take input parameters that alter the way looping through the list of
snapshots is performed. The idea here is to restrict functions that
throw away some of the snapshots returned by the ioctl to a range of
snapshots that these functions actually use. This improves efficiency
and execution speed for some rollback and send operations.
Reviewed-by: Tom Caputi <tcaputi@datto.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed by: Matt Ahrens <mahrens@delphix.com>
Signed-off-by: Alek Pinchuk <apinchuk@datto.com>
Closes#8077zfsonlinux/zfs@4c0883fb4a
MFC after: 2 weeks
illumos/illumos-gate@7931524763
FreeBSD note: some tweaking was needed to avoid a conflict with
sys/rangelock.h.
Author: Matthew Ahrens <mahrens@delphix.com>
Obtained from: illumos
MFC after: 3 weeks
Not sure if we need anything better.
Maybe we should try to port illumos libfakekernel or provide something
similar natively.
MFC after: 4 weeks
X-MFC with: r353618
illumos/illumos-gate@a0b03b161ca0b03b161chttps://www.illumos.org/issues/10330
3 recent ZoL changes in the vdev and metaslab code which we can pull over:
PR 8324 c853f382db 8324 Change target size of metaslabs from 256GB to 16GB
PR 8290 b194fab0fb 8290 Factor metaslab_load_wait() in metaslab_load()
PR 8286 419ba59145 8286 Update vdev_is_spacemap_addressable() for new spacemap
encoding
Author: Serapheim Dimitropoulos <serapheimd@gmail.com>
Obtained from: illumos, ZoL
MFC after: 2 weeks
8423 8199 7432 Implement large_dnode pool feature
7432 Large dnode pool feature
8199 multi-threaded dmu_object_alloc()
8423 Implement large_dnode pool feature
10406 large_dnode changes broke zfs recv of legacy stream
llumos/illumos-gate@54811da5ac54811da5achttps://www.illumos.org/issues/8423https://www.illumos.org/issues/8199https://www.illumos.org/issues/7432illumos/illumos-gate@811964cd9f811964cd9fhttps://www.illumos.org/issues/10406
ZoL issues:
Improved dnode allocation #6564
Clean up large dnode code #6262
Fix dnode_hold() freeing dnode behavior #8172
Fix dnode allocation race #6414, #6439
Partial: Raw sends must be able to decrease nlevels #6821, #6864
Remove unnecessary txg syncs from receive_object() Closes#7197
This updates FreeBSD large_dnode code (that was imported from ZoL) to a
version that was committed to illumos. It has some cleanups,
improvements and fixes comparing to what we have in FreeBSD now.
I think that the most significant update is 8199 multi-threaded
dmu_object_alloc().
This commit reverts r351077 that was a revert of r351074 and r351076 and
restores those changes. Required atomic operations should be available
now on all platforms where we build ZFS.
Obtained from: illumos
MFC after: 3 weeks
The feature is implemented as an extension of the existing
ZFS_IOC_RENAME ioctl. Both the userland and the DSL interfaces support
renaming only a single bookmark at a time. As of now, there is no ZCP
interface to the new functionality. I am going to add it once the DSL
interface passes a test of time.
This change picks up support for zfs_ioc_namecheck_t::ENTITY_NAME that
was added to ZoL as part of Redacted Send/Receive feature by Paul
Dagnelie <pcd@delphix.com>. This is needed to allow a bookmark name in
zc_name.
Discussed with: mahrens
Reviewed by: bcr (man page)
Sponsored by: CyberSecure
Differential Revision: https://reviews.freebsd.org/D21795
Add a small wrapper around libzfs_core's lzc_send_space() to libzfs so
that every legacy ZFS_IOC_SEND consumer, along with their userland
counterpart estimate_ioctl(), can leverage ZFS_IOC_SEND_SPACE to
request send space estimation.
The legacy functionality in zfs_ioc_send() is left untouched for
compatibility purposes.
Obtained from: ZoL
Obtained from: zfsonlinux/zfs@cf7684bc8d
Author: loli10K <ezomori.nozomu@gmail.com>
MFC after: 2 weeks
Although there is always a single stream and the total size in the
summary is always equal to the size reported for the stream, it's nice
to follow the usual output format.
MFC after: 3 days
zfsonlinux/zfs@835db58592
We have long supported estimating a size of an incremental stream from a
snapshot. We should do the same for bookmarks as well.
Obtained from: ZoL
Author: loli10K <ezomori.nozomu@gmail.com>
MFC after: 3 days
This fixes a hole in the situation where the resume state is left from
receiving a new dataset and, so, the state is set on the dataset itself
(as opposed to %recv child).
Additionally, distinguish incremental and resume streams in error
messages.
This was also committed to ZoL:
zfsonlinux/zfs@ebeb6f23bf
MFC after: 2 weeks
Sponsored by: CyberSecure
In compare(), all error cases set the error code to EPIPE, so when an
error is set, the correct assertion to make is that the error is EPIPE,
not EINVAL.
Reviewed-by: Richard Elling <Richard.Elling@RichardElling.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ryan Moeller <ryan@freqlabs.com>
Closes#8743zfsonlinux/zfs@9dc41a769d
Submitted by: Ryan Moeller <ryan@freqlabs.com>
MFC after: 1 week
Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D20118
They follow the conventions set by rw and sx lock probes. There is
an additional lockstat:::lockmgr-disown probe.
Update lockstat(1) to report on contention and hold events for
lockmgr locks. Document the new probes in dtrace_lockstat.4, and
deduplicate some of the existing probe descriptions.
Reviewed by: mjg
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D21355
