Commit graph

480 commits

Author SHA1 Message Date
Edward Tomasz Napierala 1a13f2e6b4 Introduce stats(3), a flexible statistics gathering API.
This provides a framework to define a template describing
a set of "variables of interest" and the intended way for
the framework to maintain them (for example the maximum, sum,
t-digest, or a combination thereof).  Afterwards the user
code feeds in the raw data, and the framework maintains
these variables inside a user-provided, opaque stats blobs.
The framework also provides a way to selectively extract the
stats from the blobs.  The stats(3) framework can be used in
both userspace and the kernel.

See the stats(3) manual page for details.

This will be used by the upcoming TCP statistics gathering code,
https://reviews.freebsd.org/D20655.

The stats(3) framework is disabled by default for now, except
in the NOTES kernel (for QA); it is expected to be enabled
in amd64 GENERIC after a cool down period.

Reviewed by:	sef (earlier version)
Obtained from:	Netflix
Relnotes:	yes
Sponsored by:	Klara Inc, Netflix
Differential Revision:	https://reviews.freebsd.org/D20477
2019-10-07 19:05:05 +00:00
Dimitry Andric 6b3555c38e Enable OpenMP for powerpc64
Summary: When powerpc64 switches to LLVM, use this patch to enable
OpenMP as well. OpenMP on PPC is only for 64-bits, so don't make a
32-bit libomp. A change to openmp files is necesssary (under review on
https://reviews.llvm.org/D67190), because it determines ELF format
version based on endianness, which is incorrect.

Reviewed by:	alfredo.junior_eldorado.org.br, #manpages
Differential Revision: https://reviews.freebsd.org/D21532
2019-09-26 18:24:04 +00:00
Dimitry Andric 0f80acb965 Merge ^/head r352436 through r352536. 2019-09-19 19:26:12 +00:00
Baptiste Daroussin d567f909ef Add native support for zstd to libarchive
Note that old pkg will failed to build after this. A recent ports tree (one
providing pkg 1.12+) is required to build. Older already built pkg, should
continue working as expected

PR:		238797
Exp run by:	antoine
Reviewed by:	cem
Approved by:	cem
Differential Revision:	https://reviews.freebsd.org/D20752
2019-09-18 07:57:56 +00:00
Dimitry Andric b0d4288fb1 Update build glue for building libc++. 2019-08-28 20:05:55 +00:00
Warner Losh f5a95d9a07 Remove NAND and NANDFS support
NANDFS has been broken for years. Remove it. The NAND drivers that
remain are for ancient parts that are no longer relevant. They are
polled, have terrible performance and just for ancient arm
hardware. NAND parts have evolved significantly from this early work
and little to none of it would be relevant should someone need to
update to support raw nand. This code has been off by default for
years and has violated the vnode protocol leading to panics since it
was committed.

Numerous posts to arch@ and other locations have found no actual users
for this software.

Relnotes:	Yes
No Objection From: arch@
Differential Revision: https://reviews.freebsd.org/D20745
2019-06-25 04:50:09 +00:00
Dimitry Andric eccf47f57a Enable building libomp.so for 32-bit x86. This is done by selectively
enabling the functions that save and restore MXCSR, since access to this
register requires SSE support.

Note that you may run into other issues with OpenMP on i386, since this
*not* yet supported upstream, and certainly not extensively tested.

PR:		236062, 236582
MFC after:	1 month
X-MFC-With:	r344779
2019-03-18 21:04:28 +00:00
Dimitry Andric 9b3c15c583 Disable lib/libomp build for the 32-bit part of amd64 buildworld, as it
is not supported for that target.

Reported by:	Michael Butler <imb@protected-networks.net>
PR:		236062
MFC after:	1 month
X-MFC-With:	r344779
2019-03-16 17:55:22 +00:00
Dimitry Andric b0840a28f6 Connect lib/libomp to the build.
* Set MK_OPENMP to yes by default only on amd64, for now.
* Bump __FreeBSD_version to signal this addition.
* Ensure gcc's conflicting omp.h is not installed if MK_OPENMP is yes.
* Update OptionalObsoleteFiles.inc to cope with the conflicting omp.h.
* Regenerate src.conf(5) with new WITH/WITHOUT fragments.

Relnotes:	yes
PR:		236062
MFC after:	1 month
X-MFC-With:	r344779
2019-03-16 15:45:15 +00:00
Enji Cooper b18a4ccab9 MFhead@r344786 2019-03-05 01:00:38 +00:00
Simon J. Gerraty a92958df34 Enable build of libbearssl
Reviewed by:	emaste
Sponsored by:	Juniper Networks
Differential Revision:	D16337
2019-02-26 06:11:01 +00:00
Enji Cooper 43e25f47a7 Add MK_GOOGLETEST option for building/installing gmock/gtest
These libraries don't compile on non-C++-11 capable compilers, e.g., g++ 4.2.1
and its corresponding implementation of the c++ library, i.e., libstdc++.

Blacklist compilation on all non-C++-11 capable compilers and give others the
option of opting out of building/installing gmock/gtest via MK_GOOGLETEST.

This option is controlled by MK_CXX and MK_TESTS, as ATF compilation is.
2019-02-19 22:40:32 +00:00
Enji Cooper f3c5273d31 Merge build glue for libraries and tests done on github
I need to doublecheck my work vs the port, but I believe that this covers the
initial integration of all upstream tests.

Ref: https://github.com/ngie-eign/freebsd/tree/googletest-integration
2019-02-13 04:58:15 +00:00
Dimitry Andric 959530cc41 Upgrade our copies of clang, llvm, lld, lldb, compiler-rt and libc++ to
the upstream release_70 branch r348686 (effectively, 7.0.1 rc3).  The
release will follow very soon, but no more functional changes are
expected.

Release notes for llvm, clang and lld 7.0.0 are available here:
<http://releases.llvm.org/7.0.0/docs/ReleaseNotes.html>
<http://releases.llvm.org/7.0.0/tools/clang/docs/ReleaseNotes.html>
<http://releases.llvm.org/7.0.0/tools/lld/docs/ReleaseNotes.html>

PR:		230240, 230355
Relnotes:	yes
MFC after:	2 months
2018-12-11 19:05:28 +00:00
Ed Maste 7c807d7708 retire LINKER_FEATURES filter flag
And build libdl unconditionally.  All supported FreeBSD linkers accept
-F / --filter so there is no need to test for support.

Discussed with:	kib
Sponsored by:	The FreeBSD Foundation
2018-11-12 20:44:22 +00:00
Kyle Evans bc82609ae2 Move pmc* bits behind MK_PMC to fix WITHOUT_PMC build
No objection from:	mmacy
MFC after:	3 days
2018-11-05 00:20:58 +00:00
Matt Macy 3b9156971f re-enable pmcstat, pmccontrol, and pmcannotate for gcc4 builds
I had disabled building of the aforementioned targets due to warnings breaking
tinderbox. This silences the warning and restores them to the build.

Reported by:	jhibbits
Reviewed by:	jhibbits
Approved by:	re (gjb)
2018-09-14 01:11:10 +00:00
Kyle Evans 7920ad944b libbe(3): Move build goop back out of cddl/
Some background: in the GSoC project, libbe/Makefile lived in lib/libbe. I
created projects/bectl branch, maintained the above for all of five
minutes before I misread Makefile.inc1 and decided that it couldn't possibly
build outside of cddl/, so I kicked the Makefile out into the cddl/ build
and all was good. The misreading was of the bit where .WAIT is added to
SUBDIR after lib, libexec but prior to building bin and cddl *only during
the install targets*, which is the critical part.

Fast forward- buildworld was still broken in my branch unbeknownst to me
because I didn't nuke my OBJDIR. Combing through Makefile.inc1 eventually
revealed the necessary magic to make sure that libbe's dependencies are
specified well enough, and it becomes clear what needs done to make a
non-cddl/ build work. This is an interesting prospect, because the build
split is kind of annoying to work with.

IGNORE_PRAGMA is added to avoid dropping WARNS by one more. This was
previously pulled in via cddl/Makefile.inc.
2018-08-18 03:20:59 +00:00
Warner Losh 71cb2dfcbc Remove special cases for armeb in the build.
Differential Revision:  https://reviews.freebsd.org/D16257
2018-07-17 23:23:54 +00:00
Brooks Davis e4b0a90e77 Normalize the g(eom,cache,part,...) build.
Rather then combining hardlink creation for the geom(8) binary with
shared library build, move libraries to src/lib/geom so they are
built and installed normally.  Create a common Makefile.classes
which is included by both lib/geom/Makefile and sbin/geom/Makefile
so the symlink and libraries stay in sync.

The relocation of libraries allows libraries to be build for 32-bit
compat.  This also reduces the number of non-standard builds in
the system.

This commit is not sufficent to run a 32-bit /sbin/geom on a 64-bit
system out of the box as it will look in the wrong place for libraries
unless GEOM_LIBRARY_PATH is set appropriatly in the environment.

Reviewed by:	bdrewery
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D15360
2018-06-25 19:55:15 +00:00
Konstantin Belousov f39bffc62c Rework ofed build.
Aligns the build with the FreeBSD traditional approach to not build in
contrib/, and to track inter-dependencies between libraries.

With help from:	bdrewery
Reviewed by:	bdrewery, hselasky
Sponsored by:	Mellanox Technologies
MFC after:	2 weeks
Differential revision:	https://reviews.freebsd.org/D15648
2018-06-16 15:05:05 +00:00
Matt Macy 456eeabebc pmc: fix logic in skipping riscv 2018-06-09 02:25:18 +00:00
Matt Macy 6159b91c1f pmc: don't build on riscv where there's no kmod support 2018-06-09 02:15:45 +00:00
Matt Macy f992dd4b5c pmc: convert native to jsonl and track TSC value of samples
- add '-j' options to filter to enable converting native pmc
  log format to json lines format to enable the use of scripts
  and external tooling

% pmc filter -j pmc.log pmc.jsonl

- Record the tsc value in sampling interrupts as opposed to
  recording nanotime when the sample is copied to a global log
  in hardclock - potentially many milliseconds later.

- At initialize record the tsc_freq and the time of day to give
  us an offset for translating the tsc values in callchain records
2018-06-07 02:03:22 +00:00
Hans Petter Selasky b00ab7548b MFV r333789: libpcap 1.9.0 (pre-release)
MFC after:	1 month
Sponsored by:	Mellanox Technologies
2018-05-28 08:12:18 +00:00
Ruslan Bukin cde6fa2857 Add new shared library -- libopencsd.
OpenCSD is an ARM CoreSight(tm) trace packets decoder.

- Connect libopencsd to the arm64 build.
- Install opencsd headers to /usr/include/opencsd/

Sponsored by:	DARPA, AFRL
2018-04-04 14:31:56 +00:00
Ruslan Bukin 30b3274fff Add new shared library -- libipt.
libipt is the Intel Processor Trace (Intel PT) packets decoder.

- Include libipt to amd64 build.
- Install libipt headers to /usr/include/libipt/

Sponsored by:	DARPA, AFRL
2018-03-21 14:37:04 +00:00
Kyle Evans b37f6c9805 Add libregex, connect it to the build
libregex is a regex(3) implementation intended to feature GNU extensions and
any other non-POSIX compliant extensions that are deemed worthy.

These extensions are separated out into a separate library for the sake of
not cluttering up libc further with them as well as not deteriorating the
speed (or lack thereof) of the libc implementation.

libregex is implemented as a build of the libc implementation with LIBREGEX
defined to distinguish this from a libc build. The reasons for
implementation like this are two-fold:

1.) Maintenance- This reduces the overhead induced by adding yet another
regex implementation to base.

2.) Ease of use- Flipping on GNU extensions will be as simple as linking
against libregex, and POSIX-compliant compilations can be guaranteed with a
REG_POSIX cflag that should be ignored by libc/regex and disables extensions
in libregex. It is also easier to keep REG_POSIX sane and POSIX pure when
implemented in this fashion.

Tests are added for future functionality, but left disconnected for the time
being while other testing is done.

Reviewed by:	cem (previous version)
Differential Revision:	https://reviews.freebsd.org/D12934
2018-01-22 02:44:41 +00:00
Ed Maste 5f8eed2f42 Install strings unconditionally
Previously it was enabled by WITH_/WITHOUT_TOOLCHAIN, but it is commonly
expected to be available and may have non-toolchain consumers.  As it
is now taken from the BSD-licensed ELF Tool Chain project, just install
it unconditionally.

PR:		213665, 223725
Reviewed by:	bdrewery
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D8398
2017-11-20 20:55:41 +00:00
Bryan Drewery 82fcadcd55 Connect libpathconv, disconnected since import in r309035.
Sponsored by:	Dell EMC Isilon
2017-10-31 00:03:52 +00:00
Mariusz Zaborski ceb36bc93a Introduce caspermocks.
The idea behinds mocks is that we don't need to ifdef a lot of code in
tools itself but those defines are hidden in the casper library.
Right now the mocks are implemented as define/inlines functions.
There was a very long discussion how this should be implemented.
This approach has some advantages like we don't need to link to any additional
libraries. Unfortunately there are also some disadvantages for example it is
easy to get library out of sync between two versions of functions or that we
need extra define to compile program with casper support.
This isn't an ideal solution but it's good enough for now and should simplify
capsicumizing programs. This also doesn't close us any other ways to do those
mocks and this should evolve in time.

Discussed with:	pjd, emaste, ed, rwatson, bapt, cem, bdrewery
Differential Revision:    https://reviews.freebsd.org/D8753
2017-10-28 19:23:57 +00:00
Ruslan Bukin d27927f731 Extract a set of pmcstat functions and interfaces to the new internal
library -- libpmcstat.

This includes PMC logging module, symbols lookup functions,
ELF parsing, process management, PMC attachment, etc.

This allows to reuse code while building new hwpmc(4)-based applications.

Also add pmcstat_symbol_search_by_name() function that allows to find
mapped IP range for a given function name.

Reviewed by:	kib
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D12718
2017-10-24 16:28:00 +00:00
Dimitry Andric fc203cd43e After the import of libc++ 5.0.0, there is no need to disable building
libc++experimental.a on arm (r318654) and mips (r318859) anymore, since
upstream fixed the static assertions which would occur.

Noticed by:	George Abdelmalik <gabdelmalik@uniridge.com.au>
PR:		223119
MFC after:	3 days
2017-10-21 18:21:44 +00:00
Warner Losh c8550231ef Disconnect libstand from the build.
Remove libstand from the src/lib build. Remove LIBSTAND from
bsd.libnames.mk. Add affected files to the obsolete files list.

Sponsored by: Netflix
2017-10-09 22:12:57 +00:00
Konstantin Belousov 99ac8154ff Provide libdl.
Create libdl.so.1 as a filter for libc.so.7 which exports public dl*
functions. The functions are resolved from the rtld instead, the goal
of creating library is to avoid errors from the static linker due to
missed libdl. For static binaries, an empty .o is compiled into
libdl.a so that static binaries still get dl stubs from libc.a.

Right now lld cannot create filter objects, disable libdl on arm64
when binutils are not used.

Reviewed by:	bdrewery, dim (previos version); emaste
Exp run:	PR 220525, done by antoine
Sponsored by:	The FreeBSD Foundation
MFC after:	1 month
Differential revision:	https://reviews.freebsd.org/D11504
2017-07-10 14:59:21 +00:00
Adrian Chadd 7e0699952a [lib] disable libc++experimental on mips platforms for now.
This breaks at least mips32 platform builds.

Reviewed by:	dim
2017-05-25 05:02:43 +00:00
Dimitry Andric 8e69077b05 For now, disable building libc++experimental for arm, since there are a
number of static assertion failures in the time_t related parts.

Reported by:	mmel, kib
2017-05-22 16:13:30 +00:00
Dimitry Andric 6930ca7463 Add libc++experimental.a for std::experimental support
This adds a separate library for supporting std::experimental features.
It is purposefully static, and must be explicitly linked into programs
using -lc++experimental.

PLEASE NOTE: there is NO WARRANTY as to any stability or continuing
existence of the features in the std::experimental parts of the C++
library!

Reviewed by:	ed
Differential Revision: https://reviews.freebsd.org/D10840
2017-05-21 17:07:12 +00:00
Baptiste Daroussin e229090553 Import zstandard 1.1.4 in base
zstandard is a new compression library/tool which is very fast at
compression/decompression

For now import as a private library
2017-04-15 20:05:22 +00:00
Baptiste Daroussin a19eddc34b Import zstd 1.1.4 2017-04-15 19:47:16 +00:00
Enji Cooper bc53c94f82 Fix typo in lib/Makefile
The SUBDIR_DEPEND variable should be for librpcsec_gss, not
liblibrpc_gss

MFC after:	1 week
PR:		216409
Reported by:	mail@fbsd.e4m.org
2017-01-28 23:51:03 +00:00
Ed Maste c7fbd7722d Connect new LLVM-based libgcc_eh & libgcc_s to the build
Compiler-rt and LLVM's libunwind provide a suitable replacement for
libgcc.a, libgcc_eh.a, and libgcc_s.so.

Remove the now-unused LLVM_LIBUNWIND block from gnu/lib/libgcc.

PR:		213480 [exp-run]
Reviewed by:	brooks, ed
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D8189
2016-11-04 19:35:49 +00:00
Ed Maste df283203fc Simplify logic for libproc and librtld_db in lib/Makefile 2016-10-14 17:04:07 +00:00
Ed Maste cd060b8d7a partially convert lib/Makefile to SUBDIR.${MK_FOO} style
Cases other than MK_* (e.g. ${MACHINE_CPUARCH} == "i386") have been left
as is.

Reviewed by:	imp
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D8246
2016-10-14 01:53:15 +00:00
Ed Maste 3ba97c2088 garbage collect _libatm, missed in r179308 2016-10-13 21:35:48 +00:00
Warner Losh b2a7ac4802 Fix building on i386 and arm. But 'public domain' headers on the files
with no creative content. Include "lost" changes from git:
o Use /dev/efi instead of /dev/efidev
o Remove redundant NULL checks.

Submitted by: kib@, dim@, zbb@, emaste@
2016-10-13 06:56:23 +00:00
Warner Losh d49a5ddd04 Create libefivar library. This library aims to provide
the same API as the GPL'd version of this library. It implements the common
Linux API for programatically manipulating UEFI environment varibales using
the UEFI Runtime Services the kernel provides. It replaces the old efi
library since it is programmed to a different interface, but retails the
CHAR16 to UTF-8 and vice versa conversion routines. The new name is to match
Linux program's expectations.

Differential Revision: https://reviews.freebsd.org/D8128
Reviewed by: kib@, wblock@, Ganael Laplanche
2016-10-11 22:30:41 +00:00
Mariusz Zaborski 7a6e3cf83e libcapsicum: introduce Capsicum helpers
Capsicum helpers are a set of inline functions which goal is to reduce
duplicated patterns used to Capsicumize applications.

Reviewed by:	cem, AllanJude, bapt, ed, emaste
Differential Revision:	https://reviews.freebsd.org/D8013
2016-10-03 20:48:18 +00:00
Adrian Chadd 35c05a4fbc Add librss, a simple wrapper around RSS APIs so applications can begin auto-tuning.
I've used this in a handful of RSS test applications.  It is just some
very simple functions to fetch the RSS configuration, query the per-bucket
CPU set, and mark sockets as local to an RSS bucket.  It should be sufficient
for both thread-based and process-based workloads.

(Yes, I wrote a manpage.)

This is based on some early RSS API and wrapper API work I did whilst
I was at Netflix.  Thanks to Netflix for the very original work that
spawned this; thanks to Peter Grehan for his feedback about RSS APIs
and thanks to Jack Vogel and Navdeep Parhar for the NIC-facing side of the
APIs.  These fed into the simple userland API I wrote up here.

Reviewed by:	gallatin
2016-09-30 19:59:56 +00:00
Ed Maste f987297fc9 Add a WITHOUT_DIALOG src.conf(5) knob
It also turns off dependencies (bsdinstall, bsdconfig, dpv, tzsetup).

Reviewed by:	dteske
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D7969
2016-09-27 18:08:38 +00:00
Kristof Provost ec21434933 Renaming libifc to libifconfig in response to feedback on initial commit of
this library. Sticking to 'libifconfig' (and 'ifconfig_' as function prefix)
should reduce chances of namespace collisions, make it more clear what the
library does, and be more in line with existing libraries.

Submitted by: Marie Helene Kvello-Aune <marieheleneka@gmail.com>
Differential Revision:	https://reviews.freebsd.org/D7742
Reviewed by:	cem, kp
2016-09-02 18:33:08 +00:00
Kristof Provost d14b24ef84 Add libifc, a library implementing core functionality that exists in ifconfig(8) today.
libifc (pronounced lib-ifconfig) aims to be a light abstraction layer between
programs and the kernel APIs for managing the network configuration.
This should hopefully make programs easier to maintain, and reduce code
duplication.

Work will begin on making ifconfig(8) use this library in the near future.

This code is still evolving. The interface should not be considered stable until
it is announced as such.

Submitted By: Marie Helene Kvello-Aune <marieheleneka@gmail.com>
Reviewed By: kp
Differential Revision: https://reviews.freebsd.org/D7529
2016-08-25 19:40:25 +00:00
Ed Maste 903b0dd702 Don't build libdevdctl if MK_CXX is no 2016-08-24 17:15:32 +00:00
Bryan Drewery fa6d79e80b Rename ORDERED to BOOTSTRAP since no order is respected in the list.
The directories in SUBDIR_ORDERED are built in parallel, so the name is
misleading.

MFC after:	3 days
Sponsored by:	EMC / Isilon Storage Division
2016-08-22 22:51:10 +00:00
Kurt Lidl c0759dac0d Separate BLACKLIST vs BLACKLIST_SUPPORT properly
Sponsored by:	The FreeBSD Foundation
2016-06-07 16:31:03 +00:00
Bryan Drewery d82362bbce Remove libstdc++ again.
This was mis-merged in r298107 which missed r289389.
2016-06-02 20:31:02 +00:00
Kurt Lidl 95856e1457 Add basic blacklist build support
Reviewed by:	rpaulo
Approved by:	rpaulo
Relnotes:	YES
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D5913
2016-06-02 19:06:04 +00:00
Alan Somers 7a0c41d5d7 zfsd(8), the ZFS fault management daemon
Add zfsd, which deals with hard drive faults in ZFS pools. It manages
hotspares and replements in drive slots that publish physical paths.

cddl/usr.sbin/zfsd
	Add zfsd(8) and its unit tests

cddl/usr.sbin/Makefile
	Add zfsd to the build

lib/libdevdctl
	A C++ library that helps devd clients process events

lib/Makefile
share/mk/bsd.libnames.mk
share/mk/src.libnames.mk
	Add libdevdctl to the build. It's a private library, unusable by
	out-of-tree software.

etc/defaults/rc.conf
	By default, set zfsd_enable to NO

etc/mtree/BSD.include.dist
	Add a directory for libdevdctl's include files

etc/mtree/BSD.tests.dist
	Add a directory for zfsd's unit tests

etc/mtree/BSD.var.dist
	Add /var/db/zfsd/cases, where zfsd stores case files while it's shut
	down.

etc/rc.d/Makefile
etc/rc.d/zfsd
	Add zfsd's rc script

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c
	Fix the resource.fs.zfs.statechange message. It had a number of
	problems:

	It was only being emitted on a transition to the HEALTHY state.
	That made it impossible for zfsd to take actions based on drives
	getting sicker.

	It compared the new state to vdev_prevstate, which is the state that
	the vdev had the last time it was opened.  That doesn't make sense,
	because a vdev can change state multiple times without being
	reopened.

	vdev_set_state contains logic that will change the device's new
	state based on various conditions.  However, the statechange event
	was being posted _before_ that logic took effect.  Now it's being
	posted after.

Submitted by:	gibbs, asomers, mav, allanjude
Reviewed by:	mav, delphij
Relnotes:	yes
Sponsored by:	Spectra Logic Corp, iX Systems
Differential Revision:	https://reviews.freebsd.org/D6564
2016-05-28 17:43:40 +00:00
Glen Barber 538354481e MFH
Sponsored by:	The FreeBSD Foundation
2016-03-14 18:54:29 +00:00
Warner Losh bb52d711d0 Use the newly minted Makefile.libcompat to implement libsoft libraries
for the armv6 ABI switch. This also make WITH_LIBSOFT functional on
the arm platform. As a transition thing, this seems to work even
without switching the ABI (we basically build the same libraries
twice when MK_LIBSOFT=yes until the ABI cut over next
month). MK_LIBSOFT remains default no.
2016-03-12 23:25:05 +00:00
Bryan Drewery b3b5738bfc Revert r296395.
This mistakenly removed the SUBDIR_PARALLEL but even worse is that the install
(and build) order is not correct due to the lack of SUBDIR_DEPEND on the
most critical libraries.  The only reason they build correctly now is because
buildworld's 'make libraries' orders them properly.

Pointyhat to:	bdrewery
Sponsored by:	EMC / Isilon Storage Division
2016-03-05 05:39:59 +00:00
Bryan Drewery 57942836ce Parallel installation has been safe here since r267511 added SUBDIR_DEPEND.
Sponsored by:	EMC / Isilon Storage Division
2016-03-04 22:36:42 +00:00
Glen Barber 52259a98ad MFH
Sponsored by:	The FreeBSD Foundation
2016-03-02 16:14:46 +00:00
Mariusz Zaborski c501d73c7e Convert casperd(8) daemon to the libcasper.
After calling the cap_init(3) function Casper will fork from it's original
process, using pdfork(2). Forking from a process has a lot of advantages:
1. We have the same cwd as the original process.
2. The same uid, gid and groups.
3. The same MAC labels.
4. The same descriptor table.
5. The same routing table.
6. The same umask.
7. The same cpuset(1).
From now services are also in form of libraries.
We also removed libcapsicum at all and converts existing program using Casper
to new architecture.

Discussed with:		pjd, jonathan, ed, drysdale@google.com, emaste
Partially reviewed by:	drysdale@google.com, bdrewery
Approved by:		pjd (mentor)
Differential Revision:	https://reviews.freebsd.org/D4277
2016-02-25 18:23:40 +00:00
Glen Barber 0fe0fe112f MFH
Sponsored by:	The FreeBSD Foundation
2016-02-15 21:58:52 +00:00
Ed Maste c1da297433 Add libpe for elfcopy(1) PE/COFF support
Sponsored by:	The FreeBSD Foundation
2016-02-12 21:12:47 +00:00
Glen Barber bb4355f72d Remove duplicate line, likely result of a mismerge.
Sponsored by:	The FreeBSD Foundation
2016-02-04 17:03:12 +00:00
Glen Barber f9421853ad MFH
Sponsored by:	The FreeBSD Foundation
2016-01-25 14:13:28 +00:00
Ruslan Bukin 5ac86464fc Style. 2016-01-25 10:44:10 +00:00
Ruslan Bukin 194aedf71a Do build libproc and librtld_db for RISC-V as well. 2016-01-25 10:23:36 +00:00
Glen Barber 241984530f Fix svn:mergeinfo, and MFH a commit that was previously missed.
Sponsored by:	The FreeBSD Foundation
2016-01-04 22:32:37 +00:00
Glen Barber b626f5a73a MFH r289384-r293170
Sponsored by:	The FreeBSD Foundation
2016-01-04 19:19:48 +00:00
Ian Lepore 91ee346313 Make the building of libsmb and mount_smbfs unconditional, now that r292552
has eliminated alignment and endian problems that were making it fail on
some platforms.

PR:        180438
PR:        189415
2015-12-21 17:41:08 +00:00
John Baldwin d6fb489498 Start on a new library (libsysdecode) that provides routines for decoding
system call information such as system call arguments.  Initially this
will consist of pulling duplicated code out of truss and kdump though it
may prove useful for other utilities in the future.

This commit moves the shared utrace(2) record parser out of kdump into
the library and updates kdump and truss to use it.  One difference from
the previous version is that the library version treats unknown events
that start with the "RTLD" signature as unknown events.  This simplifies
the interface and allows the consumer to decide how to handle all
non-recognized events.  Instead, this function only generates a string
description for known malloc() and RTLD records.

Reviewed by:	bdrewery
Differential Revision:	https://reviews.freebsd.org/D4537
2015-12-15 00:05:07 +00:00
Bryan Drewery d32e83c26d Move obscure lib/ installation of /usr/lib/include symlink to include/.
This avoids the need for an afterinstall: hook and a check for LIBRARIES_ONLY.
It also now respects INCLUDEDIR.

This came in r249484.

Sponsored by:	EMC / Isilon Storage Division
2015-12-04 03:18:02 +00:00
Adrian Chadd 127be1a706 Add lib80211, a small (but hopefully soon to grow) set of library
routines to interface with net80211.

This is all from the ifconfig program; the duplicate code from ifconfig
will be removed when it starts using this API.

Differential Revision:	https://reviews.freebsd.org/D4290
2015-11-30 04:41:50 +00:00
Bryan Drewery 0d9e60c857 Always build libevent, which fixes the build for usr.sbin/ypldap after r291230.
The dependency for ypldap is only with WITH_NIS, but libevent is small
enough to not warrant the extra logic and maintenance cost here so just
always build it.

Sponsored by:	EMC / Isilon Storage Division
2015-11-26 01:54:09 +00:00
Craig Rodrigues a36eca08bb Rename libohash to libopenbsd.
libopenbsd is an internal library which
to bring in compatibility stuff from OpenBSD.
This will allow us to bring in more
OpenBSD utilities into the FreeBSD base system.

We similarly use libnetbsd for bringing in stuff from NetBSD.

Reviewed by: bapt
Differential Revision: https://reviews.freebsd.org/D4078
2015-11-04 07:20:55 +00:00
Bryan Drewery eb774fcd58 libstdc++ also snook in incorrectly in r267511, despite not being a real
subdir.

Sponsored by:	EMC / Isilon Storage Division
2015-10-15 20:49:20 +00:00
Bryan Drewery d023e38f61 Remove unneeded libg++ reference that came in with r267511 based on a removed
comment.

Sponsored by:	EMC / Isilon Storage Division
2015-10-15 20:46:34 +00:00
Bryan Drewery e0d253678e Fix another ++= parsed as '+=', missed in r289384.
Sponsored by:	EMC / Isilon Storage Division
2015-10-15 20:40:23 +00:00
Bryan Drewery 468a365ddd Let the SUBDIR_DEPEND*c++ variables actually work rather than being parsed
as a +=.  These were safe due to a .WAIT very early on.

Sponsored by:	EMC / Isilon Storage Division
2015-10-15 20:27:15 +00:00
Ed Maste 1bc28ffccd Roll WITHOUT_ELFTOOLCHAIN_TOOLS into WITHOUT_TOOLCHAIN
The option was added only to ease the transition from GNU Binutils to
ELF Tool Chain tools, and that process is now complete (for the viable
replacements). Noting the removal in UPDATING is sufficient as we have
not shipped a release with the option.

Reviewed by:	brooks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D3240
2015-08-13 17:50:47 +00:00
Ruslan Bukin b78ee15e9f First cut of DTrace for AArch64.
Reviewed by:	andrew, emaste
Sponsored by:	ARM Limited
Differential Revision:	https://reviews.freebsd.org/D2738
2015-07-01 15:51:11 +00:00
Enji Cooper 4ed27da932 Remove SUBDIR_DEPEND for non-existent libreadline directory
libreadline lives under gnu/lib

MFC after: 1 week
2015-05-24 23:19:47 +00:00
Baptiste Daroussin 8def216e05 Drop libmandoc and incorporate it into the main mandoc Makefile
This simplifies maintainance of mandoc(1).
Note that the same direction was taken on OpenBSD
2015-05-21 12:51:56 +00:00
Ed Maste 4d37254910 Remove redundant csu subdir logic
The appropriate subdirectories are handled by lib/csu/Makefile. There's
no need to duplicate this logic in Makefile.inc1 and lib/Makefile.

Reviewed by:	imp
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D2523
2015-05-12 17:53:22 +00:00
Kenneth D. Merry 43518607b2 Significant upgrades to sa(4) and mt(1).
The primary focus of these changes is to modernize FreeBSD's
tape infrastructure so that we can take advantage of some of the
features of modern tape drives and allow support for LTFS.

Significant changes and new features include:

 o sa(4) driver status and parameter information is now exported via an
   XML structure.  This will allow for changes and improvements later
   on that will not break userland applications.  The old MTIOCGET
   status ioctl remains, so applications using the existing interface
   will not break.

 o 'mt status' now reports drive-reported tape position information
   as well as the previously available calculated tape position
   information.  These numbers will be different at times, because
   the drive-reported block numbers are relative to BOP (Beginning
   of Partition), but the block numbers calculated previously via
   sa(4) (and still provided) are relative to the last filemark.
   Both numbers are now provided.  'mt status' now also shows the
   drive INQUIRY information, serial number and any position flags
   (BOP, EOT, etc.) provided with the tape position information.
   'mt status -v' adds information on the maximum possible I/O size,
   and the underlying values used to calculate it.

 o The extra sa(4) /dev entries (/dev/saN.[0-3]) have been removed.

   The extra devices were originally added as place holders for
   density-specific device nodes.  Some OSes (NetBSD, NetApp's OnTap
   and Solaris) have had device nodes that, when you write to them,
   will automatically select a given density for particular tape drives.

   This is a convenient way of switching densities, but it was never
   implemented in FreeBSD.  Only the device nodes were there, and that
   sometimes confused users.

   For modern tape devices, the density is generally not selectable
   (e.g. with LTO) or defaults to the highest availble density when
   the tape is rewritten from BOT (e.g. TS11X0).  So, for most users,
   density selection won't be necessary.  If they do need to select
   the density, it is easy enough to use 'mt density' to change it.

 o Protection information is now supported.  This is either a
   Reed-Solomon CRC or CRC32 that is included at the end of each block
   read and written.  On write, the tape drive verifies the CRC, and
   on read, the tape drive provides a CRC for the userland application
   to verify.

 o New, extensible tape driver parameter get/set interface.

 o Density reporting information.  For drives that support it,
   'mt getdensity' will show detailed information on what formats the
   tape drive supports, and what formats the tape drive supports.

 o Some mt(1) functionality moved into a new mt(3) library so that
   external applications can reuse the code.

 o The new mt(3) library includes helper routines to aid in parsing
   the XML output of the sa(4) driver, and build a tree of driver
   metadata.

 o Support for the MTLOAD (load a tape in the drive) and MTWEOFI
   (write filemark immediate) ioctls needed by IBM's LTFS
   implementation.

 o Improve device departure behavior for the sa(4) driver.  The previous
   implementation led to hangs when the device was open.

 o This has been tested on the following types of drives:
	IBM TS1150
	IBM TS1140
	IBM LTO-6
	IBM LTO-5
	HP LTO-2
	Seagate DDS-4
	Quantum DLT-4000
	Exabyte 8505
	Sony DDS-2

contrib/groff/tmac/doc-syms,
share/mk/bsd.libnames.mk,
lib/Makefile,
	Add libmt.

lib/libmt/Makefile,
lib/libmt/mt.3,
lib/libmt/mtlib.c,
lib/libmt/mtlib.h,
	New mt(3) library that contains functions moved from mt(1) and
	new functions needed to interact with the updated sa(4) driver.

	This includes XML parser helper functions that application writers
	can use when writing code to query tape parameters.

rescue/rescue/Makefile:
	Add -lmt to CRUNCH_LIBS.

src/share/man/man4/mtio.4
	Clarify this man page a bit, and since it contains what is
	essentially the mtio.h header file, add new ioctls and structure
	definitions from mtio.h.

src/share/man/man4/sa.4
	Update BUGS and maintainer section.

sys/cam/scsi/scsi_all.c,
sys/cam/scsi/scsi_all.h:
	Add SCSI SECURITY PROTOCOL IN/OUT CDB definitions and CDB building
	functions.

sys/cam/scsi/scsi_sa.c
sys/cam/scsi/scsi_sa.h
	Many tape driver changes, largely outlined above.

	Increase the sa(4) driver read/write timeout from 4 to 32
	minutes.  This is based on the recommended values for IBM LTO
	5/6 drives.  This may also avoid timeouts for other tape
	hardware that can take a long time to do retries and error
	recovery.  Longer term, a better way to handle this is to ask
	the drive for recommended timeout values using the REPORT
	SUPPORTED OPCODES command.  Modern IBM and Oracle tape drives
	at least support that command, and it would allow for more
	accurate timeout values.

	Add XML status generation.  This is done with a series of
	macros to eliminate as much duplicate code as possible.  The
	new XML-based status values are reported through the new
	MTIOCEXTGET ioctl.

	Add XML driver parameter reporting, using the new MTIOCPARAMGET
	ioctl.

	Add a new driver parameter setting interface, using the new
	MTIOCPARAMSET and MTIOCSETLIST ioctls.

	Add a new MTIOCRBLIM ioctl to get block limits information.

	Add CCB/CDB building routines scsi_locate_16, scsi_locate_10,
	and scsi_read_position_10().

	scsi_locate_10 implements the LOCATE command, as does the
	existing scsi_set_position() command.  It just supports
	additional arguments and features.  If/when we figure out a
	good way to provide backward compatibility for older
	applications using the old function API, we can just revamp
	scsi_set_position().  The same goes for
	scsi_read_position_10() and the existing scsi_read_position()
	function.

	Revamp sasetpos() to take the new mtlocate structure as an
	argument.  It now will use either scsi_locate_10() or
	scsi_locate_16(), depending upon the arguments the user
	supplies.  As before, once we change position we don't have a
	clear idea of what the current logical position of the tape
	drive is.

	For tape drives that support long form position data, we
	read the current position and store that for later reporting
	after changing the position.  This should help applications
	like Bacula speed tape access under FreeBSD once they are
	modified to support the new ioctls.

	Add a new quirk, SA_QUIRK_NO_LONG_POS, that is set for all
	drives that report SCSI-2 or older, as well as drives that
	report an Illegal Request type error for READ POSITION with
	the long format.  So we should automatically detect drives
	that don't support the long form and stop asking for it after
	an initial try.

	Add a partition number to the sa(4) softc.

	Improve device departure handling. The previous implementation
	led to hangs when the device was open.

	If an application had the sa(4) driver open, and attempted to
	close it after it went away, the cam_periph_release() call in
	saclose() would cause the periph to get destroyed because that
	was the last reference to it.  Because destroy_dev() was
	called from the sa(4) driver's cleanup routine (sacleanup()),
	and would block waiting for the close to happen, a deadlock
	would result.

	So instead of calling destroy_dev() from the cleanup routine,
	call destroy_dev_sched_cb() from saoninvalidate() and wait for
	the callback.

	Acquire a reference for devfs in saregister(), and release it
	in the new sadevgonecb() routine when all devfs devices for
	the particular sa(4) driver instance are gone.

	Add a new function, sasetupdev(), to centralize setting
	per-instance devfs device parameters instead of repeating the
	code in saregister().

	Add an open count to the softc, so we know how many
	peripheral driver references are a result of open
       	sessions.

	Add the D_TRACKCLOSE flag to the cdevsw flags so
	that we get a 1:1 mapping of open to close calls
	instead of a N:1 mapping.

	This should be a no-op for everything except the
	control device, since we don't allow more than one
	open on non-control devices.

	However, since we do allow multiple opens on the
	control device, the combination of the open count
	and the D_TRACKCLOSE flag should result in an
	accurate peripheral driver reference count, and an
	accurate open count.

	The accurate open count allows us to release all
	peripheral driver references that are the result
	of open contexts once we get the callback from devfs.

sys/sys/mtio.h:
	Add a number of new mt(4) ioctls and the requisite data
	structures.  None of the existing interfaces been removed
	or changed.

	This includes definitions for the following new ioctls:

	MTIOCRBLIM      /* get block limits */
	MTIOCEXTLOCATE	/* seek to position */
	MTIOCEXTGET     /* get tape status */
	MTIOCPARAMGET	/* get tape params */
	MTIOCPARAMSET	/* set tape params */
	MTIOCSETLIST	/* set N params */

usr.bin/mt/Makefile:
	mt(1) now depends on libmt, libsbuf and libbsdxml.

usr.bin/mt/mt.1:
	Document new mt(1) features and subcommands.

usr.bin/mt/mt.c:
	Implement support for mt(1) subcommands that need to
	use getopt(3) for their arguments.

	Implement a new 'mt status' command to replace the old
	'mt status' command.  The old status command has been
	renamed 'ostatus'.

	The new status function uses the MTIOCEXTGET ioctl, and
	therefore parses the XML data to determine drive status.
	The -x argument to 'mt status' allows the user to dump out
	the raw XML reported by the kernel.

	The new status display is mostly the same as the old status
	display, except that it doesn't print the redundant density
	mode information, and it does print the current partition
	number and position flags.

	Add a new command, 'mt locate', that will supersede the
	old 'mt setspos' and 'mt sethpos' commands.  'mt locate'
	implements all of the functionality of the MTIOCEXTLOCATE
	ioctl, and allows the user to change the logical position
	of the tape drive in a number of ways.  (Partition,
	block number, file number, set mark number, end of data.)
	The immediate bit and the explicit address bits are
	implemented, but not documented in the man page.

	Add a new 'mt weofi' command to use the new MTWEOFI ioctl.
	This allows the user to ask the drive to write a filemark
	without waiting around for the operation to complete.

	Add a new 'mt getdensity' command that gets the XML-based
	tape drive density report from the sa(4) driver and displays
	it.  This uses the SCSI REPORT DENSITY SUPPORT command
	to get comprehensive information from the tape drive about
	what formats it is able to read and write.

	Add a new 'mt protect' command that allows getting and setting
	tape drive protection information.  The protection information
	is a CRC tacked on to the end of every read/write from and to
	the tape drive.

Sponsored by:	Spectra Logic
MFC after:	1 month
2015-02-23 21:59:30 +00:00
George V. Neville-Neil fcb5606706 Initial version of DTrace on ARM32.
Submitted by:	Howard Su based on work by Oleksandr Tymoshenko
Reviewed by:	ian, andrew, rpaulo, markj
2015-02-10 19:41:30 +00:00
Rui Paulo 532000256b Merge xz 5.2.0.
This brings support for multi-threaded compression.  This brings close
N times faster compression where N is the number of CPU cores.
Because of this, liblzma now depends on libthr.

Soon libarchive will be modified to use the new lzma API.

Thanks to antoine@ for the exp-run.

Differential Revision:	 https://reviews.freebsd.org/D1786
Reviewed by:	bapt
2015-02-09 06:20:34 +00:00
John Baldwin 64de80195b Add a new device control utility for new-bus devices called devctl. This
allows the user to request administrative changes to individual devices
such as attach or detaching drivers or disabling and re-enabling devices.
- Add a new /dev/devctl2 character device which uses ioctls for device
  requests.  The ioctls use a common 'struct devreq' which is somewhat
  similar to 'struct ifreq'.
- The ioctls identify the device to operate on via a string.  This
  string can either by the device's name, or it can be a bus-specific
  address.  (For unattached devices, a bus address is the only way to
  locate a device.)  Bus drivers register an eventhandler to claim
  unrecognized device names that the driver recognizes as a valid address.
  Two buses currently support addresses: ACPI recognizes any device
  in the ACPI namespace via its full path starting with "\" and
  the PCI bus driver recognizes an address specification of
  'pci[<domain>:]<bus>:<slot>:<func>' (identical to the PCI selector
  strings supported by pciconf).
- To make it easier to cut and paste, change the PnP location string
  in the PCI bus driver to output a full PCI selector string rather
  than 'slot=<slot> function=<func>'.
- Add a devctl(3) interface in libdevctl which provides a wrapper around
  the ioctls and is the preferred interface for other userland code.
- Add a devctl(8) program which is a simple wrapper around the requests
  supported by devctl(3).
- Add a device_is_suspended() function to check DF_SUSPENDED.
- Add a resource_unset_value() function that can be used to remove a
  hint from the kernel environment.  This is used to clear a
  hint.<driver>.<unit>.disabled hint when re-enabling a boot-time
  disabled device.

Reviewed by:	imp (parts)
Requested by:	imp (changing PCI location string)
Relnotes:	yes
2015-02-06 16:09:01 +00:00
Enji Cooper c7b6816fa9 Add MK_FILE to control whether or not to build file(1), libmagic(3), etc
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
2015-02-04 10:24:40 +00:00
Enji Cooper 31a741f473 Conditionalize building radius support into libpam, ppp, etc via
MK_RADIUS_SUPPORT

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
2015-02-04 06:53:45 +00:00
Enji Cooper 02629e469f Add MK_BHYVE knob for building and installing bhyve(4), et al
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
2015-01-26 06:44:48 +00:00
Enji Cooper 3b09bf5e55 Build lib/libgpio if MK_GPIO != no
Fill in corresponding entries for MK_GPIO == no in OptionalObsoleteFiles.inc

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
2015-01-25 05:37:06 +00:00
Dimitry Andric 8028b78dbf Connect libclang_rt to the build, for specific architectures. This
contains the libraries for Address Sanitizer (asan), Undefined Behavior
Sanitizer (ubsan) and Profile Guided Optimization.

ASan is a fast memory error detector. It can detect the following types
of bugs:

    Out-of-bounds accesses to heap, stack and globals
    Use-after-free
    Use-after-return (to some extent)
    Double-free, invalid free
    Memory leaks (experimental)

Typical slowdown introduced by AddressSanitizer is 2x.

UBSan is a fast and compatible undefined behavior checker. It enables a
number of undefined behavior checks that have small runtime cost and no
impact on address space layout or ABI.

PLEASE NOTE: the sanitizers still have some rough edges on FreeBSD,
particularly on i386.  These will hopefully be smoothed out in the
coming time.

Differential Revision: https://reviews.freebsd.org/D1505
2015-01-13 19:54:47 +00:00
Poul-Henning Kamp c05bafc566 Deorbit the IEEE-488/GPIB support. 2014-12-25 20:15:13 +00:00
Ed Maste 3194293903 Build infrastructure for elftoolchain tools
Set WITH_ELFTOOLCHAIN_TOOLS in src.conf to use the elftoolchain version
of the following tools:

 * addr2line
 * elfcopy (strip / mcs)
 * nm
 * size
 * strings

Reviewed by:	bapt (earlier version)
Sponsored by:	The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D1224
2014-12-01 17:49:42 +00:00
Rui Paulo f12128e4df Import libgpio.
This is a thin wrapper around the kernel interface which should make
it easier to write GPIO applications.  gpioctl(8) will be converted to
use this library in a separate commit.

Differential Revision:	https://reviews.freebsd.org/D1183
Reviewed by:	adrian, loos
Discussed on:	arm@, embedded@
Relnotes:	yes
2014-11-24 21:49:40 +00:00
Baptiste Daroussin 362492e53e Promote SQLite3 as a privatelib as it will also be used by mandoc
While here ensure sqlite3 is using pread(2) and enable the suppot for FTS4
2014-11-22 19:44:49 +00:00