Commit graph

277 commits

Author SHA1 Message Date
Martin Matuska 1f1e2261e3 zfs: merge openzfs/zfs@deb121309
Notable upstream pull request merges:
  #12918 Introduce BLAKE3 checksums as an OpenZFS feature
  #13553 Reduce ZIO io_lock contention on sorted scrub
  #13537 Improve sorted scan memory accounting
  #13540 AVL: Remove obsolete branching optimizations
  #13563 FreeBSD: Improve crypto_dispatch() handling

Obtained from:	OpenZFS
OpenZFS commit:	deb1213098
2022-06-23 17:49:33 +02:00
Martin Matuska c03c5b1c80 zfs: merge openzfs/zfs@a86e08941 (master) into main
Notable upstream pull request merges:
  #9078:  log xattr=sa create/remove/update to ZIL
  #11919: Cross-platform xattr user namespace compatibility
  #13014: Report dnodes with faulty bonuslen
  #13016: FreeBSD: Fix zvol_cdev_open locking
  #13019: spl: Don't check FreeBSD rwlocks for double initialization
  #13027: Fix clearing set-uid and set-gid bits on a file when
          replying a write
  #13031: Add enumerated vdev names to 'zpool iostat -v' and
          'zpool list -v'
  #13074: Enable encrypted raw sending to pools with greater ashift
  #13076: Receive checks should allow unencrypted child datasets
  #13098: Avoid dirtying the final TXGs when exporting a pool
  #13172: Fix ENOSPC when unlinking multiple files from full pool

Obtained from:	OpenZFS
OpenZFS commit:	a86e089415
2022-03-08 18:53:02 +01:00
Mark Johnston 2d5d2a986c ctf: Import ctf.h from OpenBSD
Use it instead of the existing ctf.h from OpenSolaris.  This makes it
easier to use CTF in the core kernel, and to extend the CTF format to
support wider type IDs.

The imported ctf.h is modified to depend only on _types.h, and also to
provide macros which use the "parent" bit of a type ID to refer to types
in a parent CTF container.

No functional change intended.

Reviewed by:	Domagoj Stolfa, emaste
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D34358
2022-03-07 10:43:18 -05:00
Dimitry Andric 5f2aca8394 Disable clang 14 warning about bitwise operators in zstd
Parts of zstd, used in openzfs and other places, trigger a new clang 14
-Werror warning:

```
sys/contrib/zstd/lib/decompress/huf_decompress.c:889:25: error: use of bitwise '&' with boolean operands [-Werror,-Wbitwise-instead-of-logical]
                        (BIT_reloadDStreamFast(&bitD1) == BIT_DStream_unfinished)
                        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```

While the warning is benign, it should ideally be fixed upstream and
then vendor-imported, but for now silence it selectively.

MFC after:	3 days
2022-02-08 21:46:08 +01:00
Martin Matuska e92ffd9b62 zfs: merge openzfs/zfs@17b2ae0b2 (master) into main
Notable upstream pull request merges:
  #12766 Fix error propagation from lzc_send_redacted
  #12805 Updated the lz4 decompressor
  #12851 FreeBSD: Provide correct file generation number
  #12857 Verify dRAID empty sectors
  #12874 FreeBSD: Update argument types for VOP_READDIR
  #12896 Reduce number of arc_prune threads
  #12934 FreeBSD: Fix zvol_*_open() locking
  #12947 lz4: Cherrypick fix for CVE-2021-3520
  #12961 FreeBSD: Fix leaked strings in libspl mnttab
  #12964 Fix handling of errors from dmu_write_uio_dbuf() on FreeBSD
  #12981 Introduce a flag to skip comparing the local mac when raw sending
  #12985 Avoid memory allocations in the ARC eviction thread

Obtained from:	OpenZFS
OpenZFS commit:	17b2ae0b24
2022-01-22 23:05:15 +01:00
Greg V 9e9c651cac cddl: fix missing ZFS library dependencies
In 9fae47666 zfsd got a libspl dependency to avoid undefined references.
However that workaround did not help external consumers of libzfs_core.

Fix all missing dependencies lld 13 and the rtld complain about.

Reviewed by:	freqlabs, markj
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D32521
2021-11-15 09:42:07 -05:00
Martin Matuska 6ba2210ee0 zfs: merge openzfs/zfs@ec64fdb93 (master) into main
Notable upstream pull request merges:
  #12392 Avoid panic in case of pool errors and missing L2ARC
  #12448 skip snapshot in zfs_iter_mounted()
  #12516 Fix NFS and large reads on older kernels
  #12533 Fail invalid incremental recursive send gracefully
  #12569 FreeBSD: Really zero the zero page
  #12575 Reject zfs send -RI with nonexistent fromsnap
  #12602 Correct refcount_add in dmu_zfetch
  #12650 zpool should call zfs_nicestrtonum() with non-NULL handle

Obtained from:	OpenZFS
OpenZFS commit:	ec64fdb93d
2021-10-21 15:06:06 +02:00
Emmanuel Vadot 79ac3c12a7 pkgbase: Put dtrace in its own package
While dtrace is usefull some people might not want it.

Differential Revision:	https://reviews.freebsd.org/D30752
Sponsored by:	Diablotin Systems
2021-06-19 17:49:31 +02:00
Jessica Clarke d5c1296234 spl: Unbreak crossbuild after f20893853e
Summary:
f20893853e made getexecname no longer guarded by BOOTSTRAPPING, but it
does not build on non-FreeBSD as it tries to use KERN_PROC_PATHNAME and
related constants.

Reviewers: mm, imp

Reviewed By: imp

Subscribers: imp

Differential Revision: https://reviews.freebsd.org/D30763
2021-06-15 01:23:04 +01:00
Warner Losh 7aa2e90176 spl: Fix gcc6 build
-Wno-error= is only a clang flag, restrict its use to only clang.

Sponsored by:		Netflix
2021-06-10 21:28:52 -06:00
Martin Matuska 47ddbfae27 zfs: unbreak build with clang 12
Change -Wno-error-atomic-alignment to -Wno-error=atomic-alignment
in the Makefile of libspl.

Reported by:	Ed Maste <emaste@FreeBSD.org>
Fix by:		Alexander Richardson <arichardson@FreeBSD.org>
2021-06-11 01:34:46 +02:00
Martin Matuska f20893853e zfs: unbreak build broken in 160388166 2021-06-08 22:21:11 +02:00
Emmanuel Vadot b5fc9e350c pkgbase: Move libicp in utilities
libicp is used by zdb zhack zinject zstream ztest libzpool.so.2 which
are all in FreeBSD-utilities.

Reviewed by:	bapt, emaste
Differential Revision:	https://reviews.freebsd.org/D29164
MFC after:	2 weeks
2021-03-16 07:12:40 +01:00
Greg V ee21ee1572 openzfs: attach pam_zfs_key to build
This PAM module allows unlocking encrypted user home datasets when
logging in (and changing passphrase when changing the account password),
see https://github.com/openzfs/zfs/pull/9903

Also supposed to unload the key when the last session for the user is
done, but there are EBUSY issues:
https://github.com/openzfs/zfs/issues/11222#issuecomment-731897858

Submitted by:	Greg V <greg_unrelenting.technology>
Reviewed by:	mm
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D28018
2021-03-02 07:26:59 -05:00
Martin Matuska ba27dd8be8 zfs: merge OpenZFS master-9312e0fd1
Notable upstream changes:
  778869fa1 Fix reporting of mount progress
  e7adccf7f Disable use of hardware crypto offload drivers on FreeBSD
  03e02e5b5 Fix checksum errors not being counted on repeated repair
  64e0fe14f Restore FreeBSD resource usage accounting
  11f2e9a49 Fix panic if scrubbing after removing a slog device

MFC after:	2 weeks
2021-02-22 13:01:17 +01:00
Martin Matuska 10f57cb98f zfs: bump version and install new share files
- bump version to 2.0.0-FreeBSD_gbf156c966
- install definition files for the new "-o compatibility" option
  to "zpool create"

MFC after:	2 weeks
2021-02-18 15:21:10 +01:00
Mark Johnston 8ba333e02e libdtrace: Stop relying on lex compatibility
It does not appear to be required, and as of commit 6b7e592c21
("lex: Do not let input() return 0 when end-of-file is reached") it
causes input to return 0 instead of EOF when end-of-input is reached.

PR:		253440
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2021-02-17 10:57:19 -05:00
Martin Matuska 184c1b9439 zfs: merge OpenZFS master-436ab35a5
- speed up writing to ZFS pools without ZIL devices (aa755b3)
- speed up importing ZFS pools (2d8f72d, a0e0199, cf0977a)
...

MFC after:		2 weeks
Reviewed by:		mjg (partial)
Tested by:		pho
Differential Revision:	https://reviews.freebsd.org/D28677
2021-02-16 02:46:28 +01:00
Kyle Evans 123ae3045d build: remove LIBPTHREAD/LIBTHR build options
WITHOUT_LIBTHR has been broken for a little over five years now, since the
xz 5.2.0 update introduced a hard liblzma dependency on libthr, and building
a useful system without threading support is becoming increasingly more
difficult.

Additionally, in the five plus years that it's been broken more reverse
dependencies have cropped up in libzstd, libsqlite3, and libcrypto (among
others) that make it more and more difficult to reconcile the effort needed
to fix these options.

Remove the broken options.

PR:		252760
Reviewed by:	brooks, emaste, kib
Differential Revision:	https://reviews.freebsd.org/D28263
2021-01-22 12:33:27 -06:00
Matt Macy b363d3d52a OpenZFS: Fix issues caused by reversed commits + rebase 2021-01-07 17:42:10 -08:00
Alex Richardson 1f6612b444 Install dtrace.h and dependencies
This makes the minimum amount of changes to allow inclusion of dtrace.h
without all the solaris compatibility headers. Installing dtrace.h allows
compiling consumers of libdtrace (e.g. https://github.com/tmetsch/python-dtrace)
without requiring a copy of the source tree.
For python-dtrace I worked around this in 58019c9a12
but being able to build the library without installed sources would be
extremely useful.

Reviewed By:	gnn
Differential Revision: https://reviews.freebsd.org/D27884
2021-01-07 09:26:21 +00:00
Toomas Soome e307eb94ae loader: zfs should support bootonce an nextboot
bootonce feature is temporary, one time boot, activated by
"bectl activate -t BE", "bectl activate -T BE" will reset the bootonce flag.

By default, the bootonce setting is reset on attempt to boot and the next
boot will use previously active BE.

By setting zfs_bootonce_activate="YES" in rc.conf, the bootonce BE will
be set permanently active.

bootonce dataset name is recorded in boot pool labels, bootenv area.

in case of nextboot, the nextboot_enable boolean variable is recorded in
freebsd:nvstore nvlist, also stored in boot pool label bootenv area.
On boot, the loader will process /boot/nextboot.conf if nextboot_enable
is "YES", and will set nextboot_enable to "NO", preventing /boot/nextboot.conf
processing on next boot.

bootonce and nextboot features are usable in both UEFI and BIOS boot.

To use bootonce/nextboot features, the boot loader needs to be updated on disk;
if loader.efi is stored on ESP, then ESP needs to be updated and
for BIOS boot, stage2 (zfsboot or gptzfsboot) needs to be updated
(gpart or other tools).

At this time, only lua loader is updated.

Sponsored by:	Netflix, Klara Inc.
Differential Revision:	https://reviews.freebsd.org/D25512
2020-09-21 09:01:10 +00:00
Alex Richardson 79e02149fc Fix dtrace tools bootstrap on non-FreeBSD after OpenZFS import
This required surprisingly few build system changes and only two changes to the
openZFS compat headers which have been upstreamed as
https://github.com/openzfs/zfs/pull/10863

Reviewed By:	#zfs, freqlabs
Differential Revision: https://reviews.freebsd.org/D26193
2020-09-19 12:08:16 +00:00
Matt Macy 2c48331d28 MFV 2.0-rc2
- Fixes divide by zero for unusual hz
- remove cryptodev dependency
2020-09-18 23:21:24 +00:00
Andrew Turner 2a6803de1c Use MACHINE_CPUARCH when checking for arm64
Use MACHINE_CPUARCH with arm64 (aarch64) when we build code that could run
on any 64-bit Arm instruction set. This will simplify checks in downstream
consumers targeting prototype instruction sets.

The only place we check for MACHINE_ARCH == aarch64 is when building the
device tree blobs. As these are targeting current generation ISAs.

Sponsored by:	Innovate UK
Differential Revision:	https://reviews.freebsd.org/D26370
2020-09-14 16:12:28 +00:00
Alex Richardson e17ebfd3b7 Remove -I flag for include path that doesn't exist
Found this while trying to get macOS bootstrap to work again after OpenZFS merge.

Reviewed By:	#zfs, freqlabs
Differential Revision: https://reviews.freebsd.org/D26192
2020-09-10 15:37:07 +00:00
Mark Johnston 8d4b1e3b76 Correct the D definition for EINTEGRITY.
MFC after:	1 week
Sponsored by:	NetApp, Inc.
Sponsored by:	Klara, Inc.
2020-09-01 15:15:22 +00:00
Ryan Moeller 40d0fd2875 libzfs: Add missing crypto dependency
libzfs_crypto.c uses PKCS5_PBKDF2_HMAC_SHA1 from libcrypto.

Reported by:	John Kennedy
Sponsored by:	iXsystems, Inc.
2020-08-27 13:05:41 +00:00
Ryan Moeller 507cf10ad5 Move zstd sources from libzfs to libzpool
zstd is kernel code that was not supposed to be in libzfs.

libzpool provides userland shims for kernel code and is where the
zstd code needs to be included.

Reported by:	John Kennedy
Discussed with:	mmacy
Sponsored by:	iXsystems, Inc.
2020-08-26 19:03:15 +00:00
Ryan Moeller 67c97ec2d6 Tidy up libzpool Makefile
Sponsored by:	iXsystems, Inc.
2020-08-26 19:00:17 +00:00
Brandon Bergren 89509c956c [PowerPC] Apply the ppc32 GOT overflow fix to powerpcspe
powerpcspe is also a 32 bit ppc platform, and also needs to be -fPIC to
avoid overflowing the GOT.

Sponsored by:	Tag1 Consulting, Inc.
2020-08-25 18:54:10 +00:00
Matt Macy b3fc6d6612 Fix libzfs build failures for some
In some environments adding zstd/include to the include path
would cause zstd's stdlib.h to hide the system one.
2020-08-25 03:43:52 +00:00
Matt Macy 9e5787d228 Merge OpenZFS support in to HEAD.
The primary benefit is maintaining a completely shared
code base with the community allowing FreeBSD to receive
new features sooner and with less effort.

I would advise against doing 'zpool upgrade'
or creating indispensable pools using new
features until this change has had a month+
to soak.

Work on merging FreeBSD support in to what was
at the time "ZFS on Linux" began in August 2018.
I first publicly proposed transitioning FreeBSD
to (new) OpenZFS on December 18th, 2018. FreeBSD
support in OpenZFS was finally completed in December
2019. A CFT for downstreaming OpenZFS support in
to FreeBSD was first issued on July 8th. All issues
that were reported have been addressed or, for
a couple of less critical matters there are
pull requests in progress with OpenZFS. iXsystems
has tested and dogfooded extensively internally.
The TrueNAS 12 release is based on OpenZFS with
some additional features that have not yet made
it upstream.

Improvements include:
  project quotas, encrypted datasets,
  allocation classes, vectorized raidz,
  vectorized checksums, various command line
  improvements, zstd compression.

Thanks to those who have helped along the way:
Ryan Moeller, Allan Jude, Zack Welch, and many
others.

Sponsored by:	iXsystems, Inc.
Differential Revision:	https://reviews.freebsd.org/D25872
2020-08-25 02:21:27 +00:00
Yoshihiro Takahashi 8f11c99715 - Cleanups related to sparc64 removal.
- Remove remains of sparc64 files.

Reviewed by:	imp
Differential Revision:	https://reviews.freebsd.org/D25831
2020-07-28 10:58:37 +00:00
Michael Tuexen a357466592 sack_newdata and snd_recover hold the same value. Therefore, use only
a single instance: use snd_recover also where sack_newdata was used.

Submitted by:		Richard Scheffenegger
Differential Revision:	https://reviews.freebsd.org/D18811
2020-02-13 15:14:46 +00:00
Simon J. Gerraty 2c9a9dfc18 Update Makefile.depend files
Update a bunch of Makefile.depend files as
a result of adding Makefile.depend.options files

Reviewed by:	 bdrewery
MFC after:	1 week
Sponsored by:   Juniper Networks
Differential Revision:  https://reviews.freebsd.org/D22494
2019-12-11 17:37:53 +00:00
Toomas Soome b0ad855d40 libzpool: use CFLAGS.lz4.c to build lz4.o and lz4.pico
Clean up this Makefile a bit.
2019-12-02 15:37:06 +00:00
Andriy Gapon a8c08e008a MFV r354378,r354379,r354386: 10499 Multi-modifier protection (MMP)
10499 Multi-modifier protection (MMP)
illumos/illumos-gate@e0f1c0afa4
e0f1c0afa4
https://www.illumos.org/issues/10499
  Port the following ZFS commits from ZoL to illumos.
  379ca9cf2 Multi-modifier protection (MMP)
  bbffb59ef Fix multihost stale cache file import
  0d398b256 Do not initiate MMP writes while pool is suspended

10701 Correct lock ASSERTs in vdev_label_read/write
illumos/illumos-gate@58447f688d
58447f688d
https://www.illumos.org/issues/10701
  Port of ZoL commit:
  0091d66f4e Correct lock ASSERTs in vdev_label_read/write
  At a minimum, this fixes a blown assert during an MMP test run when running on
  a DEBUG build.

11770 additional mmp fixes
illumos/illumos-gate@4348eb9012
4348eb9012
https://www.illumos.org/issues/11770
  Port a few additional MMP fixes from ZoL that came in after our
  initial MMP port.
  4ca457b065 ZTS: Fix mmp_interval failure
  ca95f70dff zpool import progress kstat
  (only minimal changes from above can be pulled in right now)
  060f0226e6 MMP interval and fail_intervals in uberblock

Note from the committer (me).
I do not have any use for this feature and I have not tested it.  I only
did smoke testing with multihost=off.
Please be aware.
I merged the code only to make future merges easier.

Portions contributed by: Jerry Jelinek <jerry.jelinek@joyent.com>
Portions contributed by: Tim Chase <tim@chase2k.com>
Portions contributed by: sanjeevbagewadi <sanjeev.bagewadi@gmail.com>
Portions contributed by: John L. Hammond <john.hammond@intel.com>
Portions contributed by: Giuseppe Di Natale <dinatale2@llnl.gov>
Portions contributed by: Prakash Surya <surya1@llnl.gov>
Portions contributed by: Brian Behlendorf <behlendorf1@llnl.gov>
Author: Olaf Faaland <faaland1@llnl.gov>

MFC after:	4 weeks
2019-11-18 09:38:35 +00:00
Toomas Soome 24e1a7ac77 r354253 did miss the fact that libzpool is built as fake kernel
We build libzpool as kernel like, use _FAKE_KERNEL check to include
kernel api in libzpool.
2019-11-02 21:02:54 +00:00
Toomas Soome e499793e76 Remove duplicate lz4 implementations
Port illumos change: https://www.illumos.org/issues/11667

Move lz4.c out of zfs tree to opensolaris/common/lz4, adjust it to be
usable from kernel/stand/userland builds, so we can use just one single
source. Add lz4.h to declare lz4_compress() and lz4_decompress().

MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D22037
2019-11-02 12:28:04 +00:00
Emmanuel Vadot c2a7c3bedd pkgbase: Force zfs(8) and zpool(8) to be in the runtime package
Those commands are needed to repair a FreeBSD installation so add them
to the runtime package

Reviewed by:	bapt, gjb
Differential Revision:	https://reviews.freebsd.org/D21498
2019-09-05 14:07:49 +00:00
Kirk McKusick 88640c0e8b Create new EINTEGRITY error with message "Integrity check failed".
An integrity check such as a check-hash or a cross-correlation failed.
The integrity error falls between EINVAL that identifies errors in
parameters to a system call and EIO that identifies errors with the
underlying storage media. EINTEGRITY is typically raised by intermediate
kernel layers such as a filesystem or an in-kernel GEOM subsystem when
they detect inconsistencies. Uses include allowing the mount(8) command
to return a different exit value to automate the running of fsck(8)
during a system boot.

These changes make no use of the new error, they just add it. Later
commits will be made for the use of the new error number and it will
be added to additional manual pages as appropriate.

Reviewed by:    gnn, dim, brueffer, imp
Discussed with: kib, cem, emaste, ed, jilles
Differential Revision: https://reviews.freebsd.org/D18765
2019-01-17 06:35:45 +00:00
Michael Tuexen 1e88cc8b59 Add support for send, receive and state-change DTrace providers for
SCTP. They are based on what is specified in the Solaris DTrace manual
for Solaris 11.4.

Reviewed by:		0mp, dteske, markj
Relnotes:		yes
Differential Revision:	https://reviews.freebsd.org/D16839
2018-08-22 21:23:32 +00:00
Matt Macy d12e91d584 Make dnode definition uniform on !x86
gcc4 requires -fms-extensions to accept anonymous union members
2018-08-21 03:45:09 +00:00
Kyle Evans 7920ad944b libbe(3): Move build goop back out of cddl/
Some background: in the GSoC project, libbe/Makefile lived in lib/libbe. I
created projects/bectl branch, maintained the above for all of five
minutes before I misread Makefile.inc1 and decided that it couldn't possibly
build outside of cddl/, so I kicked the Makefile out into the cddl/ build
and all was good. The misreading was of the bit where .WAIT is added to
SUBDIR after lib, libexec but prior to building bin and cddl *only during
the install targets*, which is the critical part.

Fast forward- buildworld was still broken in my branch unbeknownst to me
because I didn't nuke my OBJDIR. Combing through Makefile.inc1 eventually
revealed the necessary magic to make sure that libbe's dependencies are
specified well enough, and it becomes clear what needs done to make a
non-cddl/ build work. This is an interesting prospect, because the build
split is kind of annoying to work with.

IGNORE_PRAGMA is added to avoid dropping WARNS by one more. This was
previously pulled in via cddl/Makefile.inc.
2018-08-18 03:20:59 +00:00
Kyle Evans f25a4e58ec libbe(3): Remove -v from LDFLAGS
-v is clearly not needed for linking, and it adds extra verbose information
that is not necessary.
2018-08-18 03:08:54 +00:00
Kyle Evans f2fdf2a1dc libbe(3)/bectl(8): Remove now-redundant include paths
These were previously necessary because the libnvpair and libzfs_core
includes were not installed into the SYSROOT, being a part of the copies
target in include/Makefile rather than being installed with the library.

This was fixed in r337696 and the headers are now installed properly, so we
may let go of the cruft.
2018-08-13 05:01:19 +00:00
Kyle Evans ce33c57d6c Use INCS for non-sys/ libnvpair and libzfs_core includes
While nothing was wrong with libnvpair.h, libzfs_core.h was only guarded by
MK_CDDL rather than MK_CDDL && MK_ZFS. Rather than ugl'if'ying
include/Makefile to impose the extra restriction, just move the non-sys/
includes into INCS with the respect lib builds.

This has the added bonus of allowing third party packagers to try and split
these libs out of the FreeBSD-runtime package, if they are so inclined.

The sys/ include was left alone- generally userland libraries shouldn't
install kernel headers.

MFC after:	1 week
2018-08-13 03:38:32 +00:00
Kyle Evans 35d2028fb8 libbe(3)/bectl(8): More SYSROOT/GCC build fixes
- Missing include path
- Fully specify libzfs's dependencies (except for deps pulled in by other
  deps) in Makefile.inc1
- Drop WARNS back down to 2 for libbe(3). I do this with much hesitation,
  but the libzfs headers are apparently a hot warning-filled mess as far as
  GCC 4.2 is concerned.
2018-08-11 22:45:39 +00:00
Kyle Evans 14b841d4a8 MFH @ r337607, in preparation for boarding 2018-08-11 04:26:29 +00:00
Alexander Motin 7fca1b93c4 Do not blindly include illumos kernel headers instead of user-space.
It is not needed now, and I doubt it much helped at all, creating more
confusions then good.
2018-08-02 18:55:55 +00:00
Michael Tuexen 7bda966394 Add a dtrace provider for UDP-Lite.
The dtrace provider for UDP-Lite is modeled after the UDP provider.
This fixes the bug that UDP-Lite packets were triggering the UDP
provider.
Thanks to dteske@ for providing the dwatch module.

Reviewed by:		dteske@, markj@, rrs@
Relnotes:		yes
Differential Revision:	https://reviews.freebsd.org/D16377
2018-07-31 22:56:03 +00:00
Kyle Evans b29bf2f84e libbe(3)/be(8): Drop WARNS overrides, fix all fallout
Based on the idea that we shouldn't have all-new library and utility going
into base that need WARNS=1...

- Decent amount of constification
- Lots of parentheses
- Minor other nits
2018-07-25 15:14:35 +00:00
Kyle Evans 268af06d3e Normalize bectl(8)/libbe(3) Makefiles, remove Makefile copyright/license
Approved by:	hselaskey
2018-07-24 19:55:02 +00:00
Kyle Evans 70a11a8eea libbe(3): Add to cddl build, adjust src.libnames.mk as needed 2018-07-24 15:42:23 +00:00
Michael Tuexen d073fd606c Add support for TCP state names used by Solaris.
For compatibility, add the TCP state names used by Solaris
and given in the Dtrace Guide available at
https://docs.oracle.com/cd/E37838_01/html/E61035/glhgu.html#OSDTGglhmv

Reviewed by:		markj@
Sponsored by:		Netflix, Inc.
Differential Revision:	https://reviews.freebsd.org/D16269
2018-07-14 17:12:04 +00:00
Michael Tuexen 7575d3df5b The IP, TCP, and UDP provider report IP addresses as strings.
In some cases, the required information is not available and the
UDP provider reported an empty string in this case and the IP
and TCP provider reported a NULL pointer.

This patch changes the value provided in this case to the string
"<unknown>". This make the behaviour consistent and in-line with
the behaviour of Solaris.

Reviewed by:		markj@, dteske@, gnn@
Differential Revision:	https://reviews.freebsd.org/D15855
2018-06-18 18:35:29 +00:00
Sean Bruno 8d7181d1e0 Unbreak dtrace runtime for udp after svn r334719 SO_REUSEPORT commit.
Submitted by:	Johannes Lundberg <johalun0@gmail.com>
Sponsored by:	Limeligght Networks
2018-06-07 15:27:07 +00:00
Sean Bruno 1a43cff92a Load balance sockets with new SO_REUSEPORT_LB option.
This patch adds a new socket option, SO_REUSEPORT_LB, which allow multiple
programs or threads to bind to the same port and incoming connections will be
load balanced using a hash function.

Most of the code was copied from a similar patch for DragonflyBSD.

However, in DragonflyBSD, load balancing is a global on/off setting and can not
be set per socket. This patch allows for simultaneous use of both the current
SO_REUSEPORT and the new SO_REUSEPORT_LB options on the same system.

Required changes to structures:
Globally change so_options from 16 to 32 bit value to allow for more options.
Add hashtable in pcbinfo to hold all SO_REUSEPORT_LB sockets.

Limitations:
As DragonflyBSD, a load balance group is limited to 256 pcbs (256 programs or
threads sharing the same socket).

This is a substantially different contribution as compared to its original
incarnation at svn r332894 and reverted at svn r332967.  Thanks to rwatson@
for the substantive feedback that is included in this commit.

Submitted by:	Johannes Lundberg <johalun0@gmail.com>
Obtained from:	DragonflyBSD
Relnotes:	Yes
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D11003
2018-06-06 15:45:57 +00:00
Sean Bruno 7875017ca9 Revert r332894 at the request of the submitter.
Submitted by:	Johannes Lundberg <johalun0_gmail.com>
Sponsored by:	Limelight Networks
2018-04-24 19:55:12 +00:00
Sean Bruno 7b7796eea5 Load balance sockets with new SO_REUSEPORT_LB option
This patch adds a new socket option, SO_REUSEPORT_LB, which allow multiple
programs or threads to bind to the same port and incoming connections will be
load balanced using a hash function.

Most of the code was copied from a similar patch for DragonflyBSD.

However, in DragonflyBSD, load balancing is a global on/off setting and can not
be set per socket. This patch allows for simultaneous use of both the current
SO_REUSEPORT and the new SO_REUSEPORT_LB options on the same system.

Required changes to structures
Globally change so_options from 16 to 32 bit value to allow for more options.
Add hashtable in pcbinfo to hold all SO_REUSEPORT_LB sockets.

Limitations
As DragonflyBSD, a load balance group is limited to 256 pcbs
(256 programs or threads sharing the same socket).

Submitted by:	Johannes Lundberg <johanlun0@gmail.com>
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D11003
2018-04-23 19:51:00 +00:00
Devin Teske 16e39c48a3 Consistent casing for fallback SIGCHLD (s/Unknown/unknown/) 2018-02-26 00:04:21 +00:00
Devin Teske 4b9c94e576 Updates and enhancements to signal.d to aid DTrace scripting
+ Add missing signals SIGTHR (32) and SIGLIBRT (33)
+ Add inline for converting SIG* int to string
+ Add inline for converting CLD_* int to string

Reviewed by:	markj
Sponsored by:	Smule, Inc.
Differential Revision:	https://reviews.freebsd.org/D14497
2018-02-25 23:59:47 +00:00
Devin Teske 03a3981b89 Updates and enhancements to io.d to aid DTrace scripting
+ Add dev_type do devinfo_t
+ Add b_cmd to bufinfo_t
+ Add constants for BIO_* and DEVSTAT_TYPE_*
+ Add inline for converting BIO_* int to string
+ Add inline for converting DEVSTAT_TYPE_* int to string
+ Add mask for dev_type & DEVSTAT_TYPE_MASK to string
+ Add mask for dev_type & DEVSTAT_TYPE_IF_MASK to string

Reviewed by:	markj
Sponsored by:	Smule, Inc.
Differential Revision:	https://reviews.freebsd.org/D14396
2018-02-24 17:13:15 +00:00
Devin Teske 492494d92f Add inline to errno.d for translating int to string
Gives DTrace scripts strerror(3) functionality.

Reviewed by:	markj
Sponsored by:	Smule, Inc.
Differential Revision:	https://reviews.freebsd.org/D14386
2018-02-16 04:22:29 +00:00
Devin Teske 85ef133db3 Add the following errno definitions to /usr/lib/dtrace/errno.d
ENOTCAPABLE (93)
ECAPMODE (94)
ENOTRECOVERABLE (95)
EOWNERDEAD (96)
ERELOOKUP (-5)

and update ELAST (92 -> 96)

Reviewed by:	markj
Sponsored by:	Smule, Inc.
Differential Revision:	https://reviews.freebsd.org/D14340
2018-02-15 18:37:32 +00:00
Devin Teske 980b68eaa2 Use tabs in io.d, fix alignment issues, remove extraneous newlines 2018-02-12 23:53:38 +00:00
Mark Johnston 3ddcfe1e46 Remove uneeded parentheses.
MFC after:	1 week
2018-01-25 15:31:56 +00:00
Bryan Drewery ea825d0274 DIRDEPS_BUILD: Update dependencies.
Sponsored by:	Dell EMC Isilon
2017-10-31 00:07:04 +00:00
Andriy Gapon 1c05a6ea6b MFV r325013,r325034: 640 number_to_scaled_string is duplicated in several commands
illumos/illumos-gate@0a0551200e
0a0551200e

https://www.illumos.org/issues/640
  du(1), df(1m), ls(1), and swap(1m) all include a copy (it appears literally
  copied) of the 'number_to_scaled_string' function in their source. This should
  be moved to a shared library and all 4 commands should use this instead.

FreeBSD note: of all libcmdutils functionality ZFS (and other illumos
contrib code) currently uses only nicenum() function (which is similar
to humanize_number but has some formatting differences).  For this
reason I decided to not port the whole library.  As a result, nicenum.c
from libcmdutils is compiled into libzfs and libzpool.  This is a bit
ugly, but works.  If one day we are forced to create libillumos, then
the file should be moved to that library.

Reviewed by: Sebastian Wiedenroth <wiedi@frubar.net>
Reviewed by: Robert Mustacchi <rm@joyent.com>
Reviewed by: Yuri Pankov <yuripv@gmx.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author: Jason King <jason.brian.king@gmail.com>

MFC after:	2 weeks
2017-10-27 12:37:22 +00:00
Andriy Gapon bda88d07d9 MFV r323530,r323533,r323534: 7431 ZFS Channel Programs, and followups
7431 ZFS Channel Programs

illumos/illumos-gate@dfc115332c
dfc115332c

https://www.illumos.org/issues/7431
  ZFS channel programs (ZCP) adds support for performing compound ZFS
  administrative actions via Lua scripts in a sandboxed environment (with time
  and memory limits).
  This initial commit includes both base support for running ZCP scripts, and a
  small initial library of API calls which support getting properties and
  listing, destroying, and promoting datasets.
  Testing: in addition to the included unit tests, channel programs have been in
  use at Delphix for several months for batch destroying filesystems. The
  dsl_destroy_snaps_nvl() call has also been replaced with

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: John Kennedy <john.kennedy@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Approved by: Garrett D'Amore <garrett@damore.org>
Author: Chris Williamson <chris.williamson@delphix.com>

8552 ZFS LUA code uses floating point math

illumos/illumos-gate@916c8d8811
916c8d8811

https://www.illumos.org/issues/8552
  In the LUA interpreter used by "zfs program", the lua format() function
  accidentally includes support for '%f' and friends, which can cause compilation
  problems when building on platforms that don't support floating-point math in
  the kernel (e.g. sparc). Support for '%f' friends (%f %e %E %g %G) should be
  removed, since there's no way to supply a floating-point value anyway (all
  numbers in ZFS LUA are int64_t's).

Reviewed by: Yuri Pankov <yuripv@gmx.com>
Reviewed by: Igor Kozhukhov <igor@dilos.org>
Approved by: Dan McDonald <danmcd@joyent.com>
Author: Matthew Ahrens <mahrens@delphix.com>

8590 memory leak in dsl_destroy_snapshots_nvl()

illumos/illumos-gate@e6ab4525d1
e6ab4525d1

https://www.illumos.org/issues/8590
  In dsl_destroy_snapshots_nvl(), "snaps_normalized" is not freed after it is
  added to "arg".

Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Steve Gonczi <steve.gonczi@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author: Matthew Ahrens <mahrens@delphix.com>

FreeBSD notes:
- zfs-program.8 manual page is taken almost as is from the vendor repository,
  no FreeBSD-ification done
- fixed multiple instances of NULL being used where an integer is expected
- replaced ETIME and ECHRNG with ETIMEDOUT and EDOM respectively

This commit adds a modified version of Lua 5.2.4 under
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/lua, mirroring the
upstream.  See README.zfs in that directory for the description of Lua
customizations.
See zfs-program.8 on how to use the new feature.

MFC after:	5 weeks
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D12528
2017-10-01 16:11:07 +00:00
Jonathan Anderson bab31140a2 Remove redundant source and object files.
Reviewed by:	bdrewery, ngie
MFC after:	1 week
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D12208
2017-09-09 13:18:32 +00:00
George V. Neville-Neil 958f4928e6 Add D definitions for the named values in socket.h
Reviewed by:	markj
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D12241
2017-09-07 03:05:16 +00:00
Enji Cooper 4b330699f8 Convert traditional ${MK_TESTS} conditional idiom for including test
directories to SUBDIR.${MK_TESTS} idiom

This is being done to pave the way for future work (and homogenity) in
^/projects/make-check-sandbox .

No functional change intended.

MFC after:	1 weeks
2017-08-02 08:35:51 +00:00
Konstantin Belousov 6992112349 Commit the 64-bit inode project.
Extend the ino_t, dev_t, nlink_t types to 64-bit ints.  Modify
struct dirent layout to add d_off, increase the size of d_fileno
to 64-bits, increase the size of d_namlen to 16-bits, and change
the required alignment.  Increase struct statfs f_mntfromname[] and
f_mntonname[] array length MNAMELEN to 1024.

ABI breakage is mitigated by providing compatibility using versioned
symbols, ingenious use of the existing padding in structures, and
by employing other tricks.  Unfortunately, not everything can be
fixed, especially outside the base system.  For instance, third-party
APIs which pass struct stat around are broken in backward and
forward incompatible ways.

Kinfo sysctl MIBs ABI is changed in backward-compatible way, but
there is no general mechanism to handle other sysctl MIBS which
return structures where the layout has changed. It was considered
that the breakage is either in the management interfaces, where we
usually allow ABI slip, or is not important.

Struct xvnode changed layout, no compat shims are provided.

For struct xtty, dev_t tty device member was reduced to uint32_t.
It was decided that keeping ABI compat in this case is more useful
than reporting 64-bit dev_t, for the sake of pstat.

Update note: strictly follow the instructions in UPDATING.  Build
and install the new kernel with COMPAT_FREEBSD11 option enabled,
then reboot, and only then install new world.

Credits: The 64-bit inode project, also known as ino64, started life
many years ago as a project by Gleb Kurtsou (gleb).  Kirk McKusick
(mckusick) then picked up and updated the patch, and acted as a
flag-waver.  Feedback, suggestions, and discussions were carried
by Ed Maste (emaste), John Baldwin (jhb), Jilles Tjoelker (jilles),
and Rick Macklem (rmacklem).  Kris Moore (kris) performed an initial
ports investigation followed by an exp-run by Antoine Brodin (antoine).
Essential and all-embracing testing was done by Peter Holm (pho).
The heavy lifting of coordinating all these efforts and bringing the
project to completion were done by Konstantin Belousov (kib).

Sponsored by:	The FreeBSD Foundation (emaste, kib)
Differential revision:	https://reviews.freebsd.org/D10439
2017-05-23 09:29:05 +00:00
Enji Cooper acc37ca1c1 cddl: normalize paths using SRCTOP-relative paths or :H when possible
This simplifies make logic/output

While here, remove bogus CFLAGS which look for headers in cddl/lib/libumem.
There aren't any source files there (just Makefiles)

MFC after:	1 month
Sponsored by:	Dell EMC Isilon
2017-03-04 11:30:04 +00:00
George V. Neville-Neil 82988b50a1 Add an mbuf to ipinfo_t translator to finish cleanup of mbuf passing to TCP probes.
Reviewed by:	markj
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D9401
2017-02-01 19:33:00 +00:00
Andrey V. Elsukov 4349a2145f Convert ipv4_flags and ipv4_offset fields into host byte order.
Also save only high bits in the ipv4_flags, because it is defined
as uint8_t. So now it will show DF and MF flags as 0x40 and 0x20.

Reviewed by:	markj@
MFC after:	1 week
2016-12-29 20:27:54 +00:00
Mark Johnston b043b5dc6b libproc: Add support for some proc_attach() flags.
This change adds some handling for the equivalent of Solaris' PGRAB_*
flags. In particular, support for PGRAB_RDONLY is needed to avoid a
nasty deadlock: dtrace(1) may otherwise stop the master process for its
pseudo-terminal and end up blocking while writing to standard output.
2016-12-06 04:22:38 +00:00
Mark Johnston d42df2a447 libproc: Match prefixes when looking up mapped object by name.
When looking up an object by name, allow prefix matches if no direct match
is found. This allows one to, for example, match libc entry probes with:

 # dtrace -n 'pid$target:libc.so::entry' -c ./foo

instead of requiring "libc.so.7" or a glob.

Also remove proc_obj2map() as it currently just duplicates the
functionality of proc_name2map(). It's supposed to take a Solaris
link-map ID as a paramter, but support for this isn't implemented and
isn't required to support DTrace's pid provider.
2016-12-06 04:20:32 +00:00
Mark Johnston e20727061f Define dependencies for some auto-generated source files in libdtrace.
Remove an unneeded beforedepend rule.

Reported by:	emaste, jhb
Reviewed by:	bdrewery
MFC after:	1 week
2016-11-17 18:13:42 +00:00
George V. Neville-Neil 11e72c3b25 Unlike Solaris, in FreeBSD p_args can be 0 so check for that
instead of walking down to ar_args blindly.

Reported by:	Amanda Strnad
Reviewed by:	markj, jhb
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
2016-08-25 23:24:57 +00:00
Mark Johnston 650f66ac9a MFV r304057:
7085 add support for "if" and "else" statements in dtrace

illumos/illumos-gate@c3bd3abd88

Add syntactic sugar to dtrace: "if" and "else" statements. The sugar is
baked down to standard dtrace features by adding additional clauses with
the appropriate predicates.

Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: Sebastien Roy <sebastien.roy@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Bryan Cantrill <bryan@joyent.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
Author: Matthew Ahrens <mahrens@delphix.com>

MFC after:	2 weeks
Relnotes:	yes
2016-08-16 02:30:19 +00:00
George V. Neville-Neil 15dbc1605b On FreeBSD there is a setsockopt option SO_USER_COOKIE which allows
setting a 32 bit value on each socket. This can be used by applications
and DTrace as a rendezvous point so that an applicaton's data can
more easily be captured at run time. Expose the user cookie via
DTrace by updating the translator in tcp.d and add a quick test
program, a TCP server, that sets the cookie on each connection
accepted.

Reviewed by:	hiren
MFC after:	1 week
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D7152
2016-07-08 23:44:09 +00:00
Alan Somers 58406fff52 Raise the WARNS level in cddl/lib
cddl/lib/libavl/Makefile
cddl/lib/libctf/Makefile
cddl/lib/libnvpair/Makefile
cddl/lib/libumem/Makefile
cddl/lib/libuutil/Makefile
	Increase WARNS to the highest working level for each of these
	libraries

Approved by:	re (gjb, hrs)
MFC after:	4 weeks
Sponsored by:	Spectra Logic Corp
2016-06-23 15:02:57 +00:00
Ruslan Bukin fed1ca4b71 Add initial DTrace support for RISC-V.
Sponsored by:	DARPA, AFRL
Sponsored by:	HEIF5
2016-05-24 16:41:37 +00:00
Enji Cooper 430f7286a5 Merge ^/user/ngie/release-pkg-fix-tests to unbreak how test files are installed
after r298107

Summary of changes:

- Replace all instances of FILES/TESTS with ${PACKAGE}FILES. This ensures that
  namespacing is kept with FILES appropriately, and that this shouldn't need
  to be repeated if the namespace changes -- only the definition of PACKAGE
  needs to be changed
- Allow PACKAGE to be overridden by callers instead of forcing it to always be
  `tests`. In the event we get to the point where things can be split up
  enough in the base system, it would make more sense to group the tests
  with the blocks they're a part of, e.g. byacc with byacc-tests, etc
- Remove PACKAGE definitions where possible, i.e. where FILES wasn't used
  previously.
- Remove unnecessary TESTSPACKAGE definitions; this has been elided into
  bsd.tests.mk
- Remove unnecessary BINDIRs used previously with ${PACKAGE}FILES;
  ${PACKAGE}FILESDIR is now automatically defined in bsd.test.mk.
- Fix installation of files under data/ subdirectories in lib/libc/tests/hash
  and lib/libc/tests/net/getaddrinfo
- Remove unnecessary .include <bsd.own.mk>s (some opportunistic cleanup)

Document the proposed changes in share/examples/tests/tests/... via examples
so it's clear that ${PACKAGES}FILES is the suggested way forward in terms of
replacing FILES. share/mk/bsd.README didn't seem like the appropriate method
of communicating that info.

MFC after: never probably
X-MFC with: r298107
PR: 209114
Relnotes: yes
Tested with: buildworld, installworld, checkworld; buildworld, packageworld
Sponsored by: EMC / Isilon Storage Division
2016-05-04 23:20:53 +00:00
Glen Barber 49dae58b28 Fix including Kyuafile in packaged base system.
Fix a related typo while here.

Note, this change results in the Kyuafile inclusion in the runtime
package, which needs to be fixed, however addresses the PR as far
as I can tell in my tests.

PR:		209114
Submitted by:	ngie
Sponsored by:	The FreeBSD Foundation
2016-04-29 05:28:40 +00:00
Glen Barber 0edd2576c0 MFH
Sponsored by:	The FreeBSD Foundation
2016-04-16 02:32:12 +00:00
Hiren Panchasara 2dbc593176 Fix the 'type' for a few variables from tcpcb.
Reviewed by:		markj
Sponsored by:		Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D5973
2016-04-15 20:27:36 +00:00
Glen Barber 7d536dc855 MFH
Sponsored by:	The FreeBSD Foundation
2016-03-10 21:16:01 +00:00
Bryan Drewery 15c433351f DIRDEPS_BUILD: Connect MK_TESTS.
Sponsored by:	EMC / Isilon Storage Division
2016-03-09 22:46:01 +00:00
Glen Barber b655ec9752 MFH
Sponsored by:	The FreeBSD Foundation
2016-03-06 04:13:17 +00:00
George V. Neville-Neil 342af4d5ef fix tcpdebug: - assign to "flags" in each probe, not only debug-input
compute "len" in the same way in each probe

Submitted by:	Hannes Mehnert
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D5524
2016-03-03 02:46:12 +00:00
Glen Barber 52259a98ad MFH
Sponsored by:	The FreeBSD Foundation
2016-03-02 16:14:46 +00:00
Bryan Drewery bd18fd57db DIRDEPS_BUILD: Regenerate without local dependencies.
These are no longer needed after the recent 'beforebuild: depend' changes
and hooking DIRDEPS_BUILD into a subset of FAST_DEPEND which supports
skipping 'make depend'.

Sponsored by:	EMC / Isilon Storage Division
2016-02-24 17:20:11 +00:00
Glen Barber 43faedc133 First pass to fix the 'tests' packages.
Sponsored by:	The FreeBSD Foundation
2016-02-02 22:26:49 +00:00
Ruslan Bukin 402ecb32f1 We don't support libdtrace for RISC-V yet. 2016-01-25 10:18:41 +00:00
Bryan Drewery eacae6dc66 Fix LDADD/DPADD that should be LIBADD.
Sponsored by:	EMC / Isilon Storage Division
2015-12-04 03:17:47 +00:00
Bryan Drewery fe893d53e4 Simplify the LIBRARIES_ONLY hacks so that install: is not needed here.
This uses the same pattern that I applied to the other cases of this in
the csu directories.

Sponsored by:	EMC / Isilon Storage Division
2015-12-04 00:29:40 +00:00