libnvmf provides APIs for transmitting and receiving Command and
Response capsules along with data associated with NVMe commands.
Capsules are represented by 'struct nvmf_capsule' objects.
Capsules are transmitted and received on queue pairs represented by
'struct nvmf_qpair' objects.
Queue pairs belong to an association represented by a 'struct
nvmf_association' object.
libnvmf provides additional helper APIs to assist with constructing
command capsules for a host, response capsules for a controller,
connecting queue pairs to a remote controller and optionally
offloading connected queues to an in-kernel host, accepting queue pair
connections from remote hosts and optionally offloading connected
queues to an in-kernel controller, constructing controller data
structures for local controllers, etc.
libnvmf also includes an internal transport abstraction as well as an
implementation of a userspace TCP transport.
libnvmf is primarily intended for ease of use and low-traffic use cases
such as establishing connections that are handed off to the kernel.
As such, it uses a simple API built on blocking I/O.
For a host, a consumer first populates an 'struct
nvmf_association_params' with a set of parameters shared by all queue
pairs for a single association such as whether or not to use SQ flow
control and header and data digests and creates a 'struct
nvmf_association' object. The consumer is responsible for
establishing a TCP socket for each queue pair. This socket is
included in the 'struct nvmf_qpair_params' passed to 'nvmf_connect' to
complete transport-specific negotiation, send a Fabrics Connect
command, and wait for the Connect reply. Upon success, a new 'struct
nvmf_qpair' object is returned. This queue pair can then be used to
send and receive capsules. A command capsule is allocated, populated
with an SQE and optional data buffer, and transmitted via
nvmf_host_transmit_command. The consumer can then wait for a reply
via nvmf_host_wait_for_response. The library also provides some
wrapper functions such as nvmf_read_property and nvmf_write_property
which send a command and wait for a response synchronously.
For a controller, a consumer uses a single association for a set of
incoming connections. A consumer can choose to use multiple
associations (e.g. a separate association for connections to a
discovery controller listening on a different port than I/O
controllers). The consumer is responsible for accepting TCP sockets
directly, but once a socket has been accepted it is passed to
nvmf_accept to perform transport-specific negotiation and wait for the
Connect command. Similar to nvmf_connect, nvmf_accept returns a newly
construct nvmf_qpair. However, in contrast to nvmf_connect,
nvmf_accept does not complete the Fabrics negotiation. The consumer
must explicitly send a response capsule before waiting for additional
command capsules to arrive. In particular, in the kernel offload
case, the Connect command and data are provided to the kernel
controller and the Connect response capsule is sent by the kernel once
it is ready to handle the new queue pair.
For userspace controller command handling, the consumer uses
nvmf_controller_receive_capsule to wait for a command capsule.
nvmf_receive_controller_data is used to retrieve any data from a
command (e.g. the data for a WRITE command). It can be called
multiple times to split the data transfer into smaller sizes.
nvmf_send_controller_data is used to send data to a remote host in
response to a command. It also sends a response capsule indicating
success, or an error if an internal error occurs. nvmf_send_response
is used to send a response without associated data. There are also
several convenience wrappers such as nvmf_send_success and
nvmf_send_generic_error.
Reviewed by: imp
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D44710
As noted in bug 277096, when building a pkgbase repository using
WITHOUT_CROSS_COMPILER and WITHOUT_TOOLCHAIN (which sets WITHOUT_CLANG),
the following residual files are left over:
/usr/lib/clang/18/lib/freebsd/libclang_rt.asan-x86_64.so
/usr/lib/clang/18/share/asan_ignore_list.txt
/usr/lib/clang/18/share/cfi_ignore_list.txt
/usr/lib/clang/18/share/msan_ignore_list.txt
This is because the lib/libclang_rt directory is still descended into,
even if WITHOUT_CLANG is used. Fix it by not descending into the
libclang_rt directory in that case.
PR: 277096
Reported by: Siva Mahadevan <me@svmhdvn.name>
MFC after: 3 days
This adds support for two new diff algorithms, Myers diff and Patience
diff.
These algorithms perform a different form of search compared to the
classic Stone algorithm and support escapes when worst case scenarios
are encountered.
Add the -A flag to allow selection of the algorithm, but default to
using the new Myers diff implementation.
The libdiff implementation currently only supports a subset of input and
output options supported by diff. When these options are used, but the
algorithm is not selected, automatically fallback to the classic Stone
algorithm until support for these modes can be added.
Based on work originally done by thj@ with contributions from kevans@.
Sponsored by: Klara, Inc.
Reviewed by: thj
Differential Revision: https://reviews.freebsd.org/D44302
This will be used by bhyve to build a device tree when booting arm64
guests.
Reviewed by: corvink, jhb
MFC after: 1 week
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D40994
At runtime, when rtld loads libc it will also load libsys. For each
symbol that is present in both, the libsys one will override the libc
one. It continues to be the case that program need only link against
libc (usually implicitly). The linkage to libsys is automatic.
Reviewed by: kib, emaste, imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/908
libsys provides the FreeBSD kernel interface (auxargs, system calls,
vdso). It can be linked directly for programs using a non-standard
libc and will later be linked as a filter library to libc providing
the actual system call implementation.
Reviewed by: kib, emaste, imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/908
Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl script.
Sponsored by: Netflix
Enable LIB32 option on aarch64, defaulting to YES; it had defaulted
to "broken". Add required variables for how to compile lib32 on
arm. Use /usr/include/arm for armv7 (32-bit) headers, analogous to
/usr/include/i386 on amd64. Omit libomp from lib32; it is not
supported on armv7.
Reviewed by: jrtc27
Differential Revision: https://reviews.freebsd.org/D40945
In the process, delete a COMPAT_SOFTFP remnant that was missed in
previous sweeps.
See commit 8fad2cda93 ("bsd.compat.mk: Provide new CPP and sub-make
variables") for the context behind this change.
Reviewed by: emaste, brooks, jhb
Differential Revision: https://reviews.freebsd.org/D40931
We no longer have COMPAT_32BIT hacks for libusb, instead supporting the
normal 32-bit ioctls for freebsd32 processes, so we can enable these for
the lib32 build.
Reviewed by: emaste, brooks, jhb
Differential Revision: https://reviews.freebsd.org/D40920
Most ATM support was removed prior to FreeBSD 12. The netgraph support
was kept as it was less intrusive, but it is presumed to be unused.
Reviewed by: manu
Relnotes: yes
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D38879
Several important base system components are written in C++, and the
WITHOUT_CXX option produced a system that was not fully functional.
Just accept this, and remove the option to build without C++ support.
This reverts commit adc3c128c6.
Reviewed by: brooks, kevans, jhb (earlier)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33108
The text after .error et al is emitted verbatim.
Reviewed by: sjg
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33904
Move some of the code duplicated between ctld(8) and iscsid(8) into a
libiscsiutil library.
Sharing the low-level PDU code did require having a
'struct connection' base class with a method table to permit separate
initiator vs target behavior (e.g. in handling proxy PDUs).
Reviewed by: mav, emaste
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D33544
COMPILER_TYPE is not set during cleandir and perhaps other non-build
targets, and a build with ASAN or UBSAN enabled failed with an error
reporting that runtime libraries could not be built.
PR: 260099
Reviewed by: dim
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32805
bsddialog is an attempt to write in permissive license a replacement for
libdialog.
While it is still in early stage it is good enough to already be used in
many areas, it is imported as private lib until it matures enough to be
considered as having a stable ABI
In the past we built the sanitizer runtimes when building Clang
(and using Clang as the compiler) but 7676b388ad changed this to
be conditional only on using Clang, to make the runtimes available
for external Clang.
They fail to build when WITHOUT_CXX is set though, so add MK_CXX
as part of the condition.
Reported by: Michael Dexter, Build Option Survey
Reviewed by: imp, jrtc27
Fixes: 7676b388ad ("Always build the sanitizer runtimes...")
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32731
libfido2 requires USB, so disable it if not available.
Reported by: peterj
Fixes: 7b1e19ad78 ("Add libfido2 to the build")
Sponsored by: The FreeBSD Foundation
From https://github.com/Yubico/libfido2:
libfido2 provides library functionality and command-line tools to
communicate with a FIDO device over USB, and to verify attestation
and assertion signatures.
libfido2 supports the FIDO U2F (CTAP 1) and FIDO 2.0 (CTAP 2)
protocols.
libfido2 will be used by ssh to support FIDO/U2F keys. It is currently
intended only for use by ssh, and so is installed as a PRIVATELIB and is
placed in the ssh pkgbase package.
This is currently disabled for the 32-bit library build as libfido2 is
not compatible with the COMPAT_32BIT hack in usb_ioctl.h.
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32448
From https://github.com/PJK/libcbor:
libcbor is a C library for parsing and generating CBOR, the general-
purpose schema-less binary data format.
libcbor will be used by ssh to support FIDO/U2F keys. It is currently
intended only for use by ssh, and so is installed as a PRIVATELIB and is
placed in the ssh pkgbase package.
cbor_export.h and configuration.h were generated by the upstream CMake
build. We could create them with bmake rules instead (as NetBSD has
done) but this is a fine start.
This is currently disabled for the 32-bit library build as libfido2 is
not compatible with the COMPAT_32BIT hack in usb_ioctl.h, and there is
no need for libcbor without libfido2.
Reviewed by: kevans
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32347
When WITH_LLVM_BINUTILS is set, we will install the LLVM binutils as
ar/ranlib/nm/objcopy/etc. instead of the elftoolchain ones.
Having the LLVM binutils instead of the elftoolchain ones allows us to use
features such as LTO that depend on binutils that understand LLVM IR.
Another benefit will be an improved user-experience when compiling with
AddressSanitizer, since ASAN does not symbolize backtraces correctly if
addr2line is elftoolchain addr2line instead of llvm-symbolizer.
See https://lists.freebsd.org/archives/freebsd-toolchain/2021-July/000062.html
for more details.
This is currently off by default but will be turned on by default at some
point in the near future.
Reviewed By: emaste
Differential Revision: https://reviews.freebsd.org/D31060
This adds two new options WITH_ASAN/WITH_UBSAN that can be set to
enable instrumentation of all binaries with AddressSanitizer and/or
UndefinedBehaviourSanitizer. This current patch is almost sufficient
to get a complete buildworld with sanitizer instrumentation but in
order to actually build and boot a system it depends on a few more
follow-up commits.
Reviewed By: brooks, kib, markj
Differential Revision: https://reviews.freebsd.org/D31043
I forgot to include this line in 2eb9ad4274.
Reported by: Jenkins CI
MFC after: 1 week
Fixes: 2eb9ad427475190ei ("Simplify and speed up the kyua build")
This allows instrumenting e.g. test binaries even when compiling with an
external clang (e.g. CROSS_TOOLCHAIN=llvm11). I have some upcoming patches
that allow building the entire base system with ASan/UBSan/etc.
instrumentation and this is required in preparation for this.
Reviewed By: dim, emaste
Differential Revision: https://reviews.freebsd.org/D28532
This option has been equivalent to any form of C++ support since libstdc++
was removed. Therefore, replace all MK_LIBCPLUSPLUS uses with MK_CXX.
Reviewed By: emaste
Differential Revision: https://reviews.freebsd.org/D27974
WITHOUT_LIBTHR has been broken for a little over five years now, since the
xz 5.2.0 update introduced a hard liblzma dependency on libthr, and building
a useful system without threading support is becoming increasingly more
difficult.
Additionally, in the five plus years that it's been broken more reverse
dependencies have cropped up in libzstd, libsqlite3, and libcrypto (among
others) that make it more and more difficult to reconcile the effort needed
to fix these options.
Remove the broken options.
PR: 252760
Reviewed by: brooks, emaste, kib
Differential Revision: https://reviews.freebsd.org/D28263
libjail is pretty small, so it makes for a good proof of concept demonstrating
how a system library can be wrapped to create a loadable Lua module for flua.
* Introduce 3lua section for man pages
* Add libjail module
Reviewed by: kevans, manpages
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D26080
VirtFS allows sharing an arbitrary directory tree between bhyve virtual
machine and the host. Current implementation has a fairly complete support
for 9P2000.L protocol, except for the extended attribute support. It has
been verified to work with the qemu-kvm hypervisor.
Reviewed by: rgrimes, emaste, jhb, trasz
Approved by: trasz (mentor)
MFC after: 1 month
Relnotes: yes
Sponsored by: Conclusive Engineering (development), vStack.com (funding)
Differential Revision: https://reviews.freebsd.org/D10335
This changeset introduces the new libnetmap library for writing
netmap applications.
Before libnetmap, applications could either use the kernel API
directly (e.g. NIOCREGIF/NIOCCTRL) or the simple header-only-library
netmap_user.h (e.g. nm_open(), nm_close(), nm_mmap() etc.)
The new library offers more functionalities than netmap_user.h:
- Support for complex netmap options, such as external memory
allocators or per-buffer offsets. This opens the way to future
extensions.
- More flexibility in the netmap port bind options, such as
non-numeric names for pipes, or the ability to specify the netmap
allocator that must be used for a given port.
- Automatic tracking of the netmap memory regions in use across the
open ports.
At the moment there is no man page, but the libnetmap.h header file
has in-depth documentation.
Reviewed by: hrs
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D26171
In parallel builds, this should allow sqlite to start building earlier and
increase parallelism when building lib/. Looking at htop output during
buildworld/tinderbox, there are long phases where only one CPU is active
optimizing the massive sqlite3.c file since the build of libsqlite3 is
started quite late.
Reviewed By: emaste
Differential Revision: https://reviews.freebsd.org/D26169
In lib/Makefile, we document the dependency with SUBDIR_DEPEND
For buildworld orchestration, just prebuild libregex if GOOGLETEST is
enabled. googletest will get built in a later pass.
r316063 installed pf's embedded libevent as a private lib, with headers
in /usr/include/private/event. Unfortunately we also have a copy of
libevent v2 included in ntp, which needed to be updated for compatibility
with OpenSSL 1.1.
As unadorned 'libevent' generally refers to libevent v2, be explicit that
this one is libevent v1.
Reviewed by: vangyzen (earlier)
Differential Revision: https://reviews.freebsd.org/D17275
Having kyua in the base system will simplify automated testing in CI and
eliminates bootstrapping issues on new platforms.
The build of kyua is controlled by WITH(OUT)_TESTS_SUPPORT.
Reviewed by: emaste
Obtained from: CheriBSD
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D24103
It is added an INTERNALLIB and not installed. It will be used by kyua.
This is a preparatory commit for D24103.
Reviewed by: emaste
Obtained from: CheriBSD
Sponsored by: DARPA
The new liblua will be used in a forthcoming import of kyua.
Reviewed by: kevans
Obtained from: CheriBSD
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D24090
LLVM's libunwind is used on all FreeBSD-supported CPU architectures and
is a required component.
Reviewed by: brooks (earlier)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D23123
For libssp.so, rebuild stack_protector.c with FORTIFY_SOURCE stubs that just
abort built into it.
For libssp_nonshared.a, steal stack_protector_compat.c from
^/lib/libc/secure and massage it to maintain that __stack_chk_fail_local
is a hidden symbol.
libssp is now built unconditionally regardless of {WITH,WITHOUT}_SSP in the
build environment, and the gcclibs version has been disconnected from the
build in favor of this one.
PR: 242950 (exp-run)
Reviewed by: kib, emaste, pfg, Oliver Pinter (earlier version)
Also discussed with: kan
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D22943
Summary:
Enable on powerpc64 and in lib/libclang_rt/Makefile change
MACHINE_CPUARCH to MACHINE_ARCH because on powerpc64
MACHINE_ARCH==MACHINE_CPUARCH so the 32-bit library overwrites 64-bit
library during installworld.
This patch doesn't enable any other libclang_rt libraries because they
need to be separately ported.
I have verified that games/julius (which fails on powerpc64 elfv2
without this change because of no libclang_rt profiling library) builds.
Test Plan: Ship it, test on powerpc and powerpcspe
Submitted by: pkubaj
Reviewed by: dim, jhibbits
Differential Revision: https://reviews.freebsd.org/D22425
MFC after: 1 month
X-MFC-With: r353358
