Commit graph

367 commits

Author SHA1 Message Date
Andrew Turner 44e446a1b3 Rename the macros to extract a single arm64 ID field.
Because of the previous naming scheme the old ID_AA64PFR0_EL1 macro
collided with a potential macro for the register of the same name. To fix
this collision rename these macros.

Sponsored by:	DARPA, AFRL
2019-10-30 10:06:57 +00:00
Xin LI 21aae72489 Remove zlib 1.0.4 from kernel.
PR:		229763
Reviewed by:	emaste, Yoshihiro Ota <ota j email ne jp>
Differential Revision:	https://reviews.freebsd.org/D21375
2019-08-25 17:13:00 +00:00
Conrad Meyer 2b0ebb77e4 libkern: Implement strchrnul(3) 2019-08-19 22:53:05 +00:00
Xin LI 0ed1d6fb00 Allow Kernel to link in both legacy libkern/zlib and new sys/contrib/zlib,
with an eventual goal to convert all legacl zlib callers to the new zlib
version:

 * Move generic zlib shims that are not specific to zlib 1.0.4 to
   sys/dev/zlib.
 * Connect new zlib (1.2.11) to the zlib kernel module, currently built
   with Z_SOLO.
 * Prefix the legacy zlib (1.0.4) with 'zlib104_' namespace.
 * Convert sys/opencrypto/cryptodeflate.c to use new zlib.
 * Remove bundled zlib 1.2.3 from ZFS and adapt it to new zlib and make
   it depend on the zlib module.
 * Fix Z_SOLO build of new zlib.

PR:		229763
Submitted by:	Yoshihiro Ota <ota j email ne jp>
Reviewed by:	markm (sys/dev/zlib/zlib_kmod.c)
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D19706
2019-08-01 06:35:33 +00:00
Warner Losh 710becdd96 Remove pre-FreeBSD 7.0 compatibility. 2019-07-19 18:38:47 +00:00
Xin LI f89d207279 Separate kernel crc32() implementation to its own header (gsb_crc32.h) and
rename the source to gsb_crc32.c.

This is a prerequisite of unifying kernel zlib instances.

PR:		229763
Submitted by:	Yoshihiro Ota <ota at j.email.ne.jp>
Differential Revision:	https://reviews.freebsd.org/D20193
2019-06-17 19:49:08 +00:00
Mateusz Guzik 5e57adc874 random(4): depessimize arc4random
- __predict_false reseeding on entry as it is almost never true.
- don't blindly atomic_cmpset as on x86 it ends up dirtying the cacheline.
it almost ever succeeds per above
- fetch the timestamp prior to getting the cpu number

Reviewed by:	cem
Approved by:	secteam (delphij)
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D20242
2019-05-12 06:32:46 +00:00
Conrad Meyer 6b97c2e306 Revert r346410 and r346411
libkern in .PATH has too many filename conflicts with libc and my -DNO_CLEAN
tinderbox didn't catch that ahead of time.  Mea culpa.
2019-04-19 22:08:17 +00:00
Conrad Meyer 7deb4b1964 libkern: Bring in arc4random_uniform(9) from libc
It is a useful arc4random wrapper in the kernel for much the same reasons as
in userspace.  Move the source to libkern (because kernel build is
restricted to sys/, but userspace can include any file it likes) and build
kernel and libc versions from the same source file.

Copy the documentation from arc4random_uniform(3) to the section 9 page.

While here, add missing arc4random_buf(9) symlink.

Sponsored by:	Dell EMC Isilon
2019-04-19 20:05:47 +00:00
Conrad Meyer 3782136ff1 random(4): Restore availability tradeoff prior to r346250
As discussed in that commit message, it is a dangerous default.  But the
safe default causes enough pain on a variety of platforms that for now,
restore the prior default.

Some of this is self-induced pain we should/could do better about; for
example, programmatic CI systems and VM managers should introduce entropy
from the host for individual VM instances.  This is considered a future work
item.

On modern x86 and Power9 systems, this may be wholly unnecessary after
D19928 lands (even in the non-ideal case where early /boot/entropy is
unavailable), because they have fast hardware random sources available early
in boot.  But D19928 is not yet landed and we have a host of architectures
which do not provide fast random sources.

This change adds several tunables and diagnostic sysctls, documented
thoroughly in UPDATING and sys/dev/random/random_infra.c.

PR:		230875 (reopens)
Reported by:	adrian, jhb, imp, and probably others
Reviewed by:	delphij, imp (earlier version), markm (earlier version)
Discussed with:	adrian
Approved by:	secteam(delphij)
Relnotes:	yeah
Security:	related
Differential Revision:	https://reviews.freebsd.org/D19944
2019-04-18 20:48:54 +00:00
Conrad Meyer 13774e8228 random(4): Block read_random(9) on initial seeding
read_random() is/was used, mostly without error checking, in a lot of
very sensitive places in the kernel -- including seeding the widely used
arc4random(9).

Most uses, especially arc4random(9), should block until the device is seeded
rather than proceeding with a bogus or empty seed.  I did not spy any
obvious kernel consumers where blocking would be inappropriate (in the
sense that lack of entropy would be ok -- I did not investigate locking
angle thoroughly).  In many instances, arc4random_buf(9) or that family
of APIs would be more appropriate anyway; that work was done in r345865.

A minor cleanup was made to the implementation of the READ_RANDOM function:
instead of using a variable-length array on the stack to temporarily store
all full random blocks sufficient to satisfy the requested 'len', only store
a single block on the stack.  This has some benefit in terms of reducing
stack usage, reducing memcpy overhead and reducing devrandom output leakage
via the stack.  Additionally, the stack block is now safely zeroed if it was
used.

One caveat of this change is that the kern.arandom sysctl no longer returns
zero bytes immediately if the random device is not seeded.  This means that
FreeBSD-specific userspace applications which attempted to handle an
unseeded random device may be broken by this change.  If such behavior is
needed, it can be replaced by the more portable getrandom(2) GRND_NONBLOCK
option.

On any typical FreeBSD system, entropy is persisted on read/write media and
used to seed the random device very early in boot, and blocking is never a
problem.

This change primarily impacts the behavior of /dev/random on embedded
systems with read-only media that do not configure "nodevice random".  We
toggle the default from 'charge on blindly with no entropy' to 'block
indefinitely.'  This default is safer, but may cause frustration.  Embedded
system designers using FreeBSD have several options.  The most obvious is to
plan to have a small writable NVRAM or NAND to persist entropy, like larger
systems.  Early entropy can be fed from any loader, or by writing directly
to /dev/random during boot.  Some embedded SoCs now provide a fast hardware
entropy source; this would also work for quickly seeding Fortuna.  A 3rd
option would be creating an embedded-specific, more simplistic random
module, like that designed by DJB in [1] (this design still requires a small
rewritable media for forward secrecy).  Finally, the least preferred option
might be "nodevice random", although I plan to remove this in a subsequent
revision.

To help developers emulate the behavior of these embedded systems on
ordinary workstations, the tunable kern.random.block_seeded_status was
added.  When set to 1, it blocks the random device.

I attempted to document this change in random.4 and random.9 and ran into a
bunch of out-of-date or irrelevant or inaccurate content and ended up
rototilling those documents more than I intended to.  Sorry.  I think
they're in a better state now.

PR:		230875
Reviewed by:	delphij, markm (earlier version)
Approved by:	secteam(delphij), devrandom(markm)
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D19744
2019-04-15 18:40:36 +00:00
Matt Macy 744799ead2 Add non-sleepable strdup variant strdup_flags
debugfs expects to do non-sleepable allocations

Reviewed by:	hps@
MFC after:	1 week
Sponsored by:	iX Systems
Differential Revision:	https://reviews.freebsd.org/D19259
2019-02-20 20:48:10 +00:00
Ed Maste 05bc9aa78f mcount: tidy up ANSIfication
libc/gmon's mcount was ANSIfied in r124180, with libkern following over
a decade later, in r325988, but some minor discrepancies remained.
Update libc/gmon's mexitcount to an ANSI C function definition, and use
(void) for libkern-only functions that take no arguments.

Reported by:	bde
2018-10-20 22:39:35 +00:00
Conrad Meyer e38b830780 Sync strlcpy with userland version, again
No functional change.

Please remember to update libkern copies of libc functions when you update
libc.

Sponsored by:	Dell EMC Isilon
2018-06-21 17:35:13 +00:00
Conrad Meyer 3d825c42ca str(r)chr: Replace union abuse with __DECONST
Writing one union member and reading another is technically illegal C,
although we do it in many places in the tree.  Use the __DECONST macro
instead, which is (technically) a valid C construct.

Trivial style(9) cleanups to touched lines while here.

Sponsored by:	Dell EMC Isilon
2018-06-04 18:47:14 +00:00
Mateusz Guzik ba96f37758 Use __builtin for various mem* and b* (e.g. bzero) routines.
Some of the routines were using artificially limited builtin already,
drop the explicit limit.

The use of builtins allows quite often allows the compiler to elide the call
or most zeroing to begin with. For instance, if the target object is 32 bytes
in size and gets zeroed + has 16 bytes initialized, the compiler can just
add code to zero out the rest.

Note not all the primites have asm variants and some of the existing ones
are not optimized. Maintaines are strongly encourage to take a look
(regardless of this change).
2018-06-02 18:03:35 +00:00
Mateusz Guzik 97e8984893 libkern: tidy up memset
1. Remove special-casing of 0 as it just results in an extra function call.
This is clearly pessimal.
2. Drop the inline stuff. For the most part it is much better served with
__builtin_memset (coming later).
3. Move the declaration to systm.h to match other funcs.

Archs are encouraged to implement the variant for their own platform so that
this implementation can be dropped.
2018-06-02 17:57:09 +00:00
Bruce Evans dbe3061729 Fix recent breakages of kernel profiling, mostly on i386 (high resolution
kernel profiling remains broken).

memmove() was broken using ALTENTRY().  ALTENTRY() is only different from
ENTRY() in the profiling case, and its use in that case was sort of
backwards.  The backwardness magically turned memmove() into memcpy()
instead of completely breaking it.  Only the high resolution parts of
profiling itself were broken.  Use ordinary ENTRY() for memmove().
Turn bcopy() into a tail call to memmove() to reduce complications.
This gives slightly different pessimizations and profiling lossage.
The pessimizations are minimized by not using a frame pointer() for
bcopy().

Calls to profiling functions from exception trampolines were not
relocated.  This caused crashes on the first exception.  Fix this using
function pointers.

Addresses of exception handlers in trampolines were not relocated.  This
caused unknown offsets in the profiling data.  Relocate by abusing
setidt_disp as for pmc although this is slower than necessary and
requires namespace pollution.  pmc seems to be missing some relocations.
Stack traces and lots of other things in debuggers need similar relocations.

Most user addresses were misclassified as unknown kernel addresses and
then ignored.  Treat all unknown addresses as user. Now only user
addresses in the kernel text range are significantly misclassified (as
known kernel addresses).

The ibrs functions didn't preserve enough registers.  This is the only
recent breakage on amd64.  Although these functions are written in
asm, in the profiling case they call profiling functions which are
mostly for the C ABI, so they only have to save call-used registers.
They also have to save arg and return registers in some cases and
actually save them in all cases to reduce complications.  They end up
saving all registers except %ecx on i386 and %r10 and %r11 on amd64.
Saving these is only needed for 1 caller on each of amd64 and i386.
Save them there.  This is slightly simpler.

Remove saving %ecx in handle_ibrs_exit on i386.  Both handle_ibrs_entry
and handle_ibrs_exit use %ecx, but only the latter needed to or did
save it.  But saving it there doesn't work for the profiling case.

amd64 has more automatic saving of the most common scratch registers
%rax, %rcx and %rdx (its complications for %r10 are from unusual use
of %r10 by SYSCALL).  Thus profiling of handle_ibrs_exit_rs() was not
broken, and I didn't simplify the saving by moving the saving of these
registers from it to the caller.
2018-06-02 04:25:09 +00:00
Warner Losh c6f7141fc0 Protect bzero call against macro expansion
Shortly, we'll be moving to defining bzero and memset in terms of
__builting_memset. To do that, we can't have macro calls to bzero in
the fallback impelmentation of memset. Normal calls to bzero are fine.
All 4 architectures that use this have their own copies of bzero, so
there's no mutual recursion issue between memset and bcopy.
2018-05-24 23:20:10 +00:00
Warner Losh 509b94387a This is no unreferenced, so retire it.
Differential Revision: https://reviews.freebsd.org/D15374
2018-05-24 21:11:38 +00:00
Xin LI b6f7731dba Remove "All rights reserved" from my files.
See r333391 for the rationale.

MFC after:	1 week
2018-05-10 06:41:08 +00:00
Warner Losh baaa3c4d60 Simplify things a little
Rather than include a copy for memmove to call bcopy to call memcpy
(which handles overlapping copies), make memmove a strong reference to
memcpy to save the two calls.

Differential Revision: https://reviews.freebsd.org/D15374
2018-05-10 02:31:48 +00:00
Warner Losh 5aa07b053a Move MI-ish bcopy routine to libkern
riscv and powerpc have nearly identical bcopy.c that's
supposed to be mostly MI. Move it to the MI libkern.

Differential Revision: https://reviews.freebsd.org/D15374
2018-05-10 02:31:38 +00:00
David Bright 2b08b42bae iconv uses strlen directly on user supplied memory
`iconv_sysctl_add` from `sys/libkern/iconv.c` incorrectly limits the
size of user strings, such that several out of bounds reads could have
been possible.

static int
iconv_sysctl_add(SYSCTL_HANDLER_ARGS)
{
	struct iconv_converter_class *dcp;
	struct iconv_cspair *csp;
	struct iconv_add_in din;
	struct iconv_add_out dout;
	int error;

	error = SYSCTL_IN(req, &din, sizeof(din));
	if (error)
		return error;
	if (din.ia_version != ICONV_ADD_VER)
		return EINVAL;
	if (din.ia_datalen > ICONV_CSMAXDATALEN)
		return EINVAL;
	if (strlen(din.ia_from) >= ICONV_CSNMAXLEN)
		return EINVAL;
	if (strlen(din.ia_to) >= ICONV_CSNMAXLEN)
		return EINVAL;
	if (strlen(din.ia_converter) >= ICONV_CNVNMAXLEN)
		return EINVAL;
...

Since the `din` struct is directly copied from userland, there is no
guarantee that the strings supplied will be NULL terminated. The
`strlen` calls could continue reading past the designated buffer
sizes.

Declaration of `struct iconv_add_in` is found in `sys/sys/iconv.h`:

struct iconv_add_in {
	int	ia_version;
	char	ia_converter[ICONV_CNVNMAXLEN];
	char	ia_to[ICONV_CSNMAXLEN];
	char	ia_from[ICONV_CSNMAXLEN];
	int	ia_datalen;
	const void *ia_data;
};

Our strings are followed by the `ia_datalen` member, which is checked
before the `strlen` calls:

if (din.ia_datalen > ICONV_CSMAXDATALEN)

Since `ICONV_CSMAXDATALEN` has value `0x41000` (and is `unsigned`),
this ensures that `din.ia_datalen` contains at least 1 byte of 0, so
it is not possible to trigger a read out of bounds of the `struct`
however, this code is fragile and could introduce subtle bugs in the
future if the `struct` is ever modified.

PR:		207302
Submitted by:	CTurt <cturt@hardenedbsd.org>
Reported by:	CTurt <cturt@hardenedbsd.org>
Reviewed by:	jhb, vangyzen
MFC after:	1 week
Sponsored by:	Dell EMC
Differential Revision:	https://reviews.freebsd.org/D14521
2018-02-26 18:23:36 +00:00
Ed Maste 85b648927a libkern: use nul for terminating char rather than 0
Akin to the change made in r188080 for lib/libc/string/.

Reported by:	bde
Sponsored by:	The FreeBSD Foundation
2018-02-13 19:17:48 +00:00
Pedro F. Giffuni 2cc7a55540 SPDX: fix wrong license ID tag in libkern. 2017-12-28 01:20:30 +00:00
Pedro F. Giffuni 8a36da99de sys/kern: adoption of SPDX licensing ID tags.
Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
2017-11-27 15:20:12 +00:00
Pedro F. Giffuni 51369649b0 sys: further adoption of SPDX licensing ID tags.
Mainly focus on files that use BSD 3-Clause license.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.
2017-11-20 19:43:44 +00:00
Ed Maste 4a8dea8cf9 ANSIfy sys/libkern
PR:		223641
Submitted by:	ota@j.email.ne.jp
MFC after:	1 week
2017-11-19 00:31:13 +00:00
Pedro F. Giffuni df57947f08 spdx: initial adoption of licensing ID tags.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.

Initially, only tag files that use BSD 4-Clause "Original" license.

RelNotes:	yes
Differential Revision:	https://reviews.freebsd.org/D13133
2017-11-18 14:26:50 +00:00
Kirk McKusick 75e3597abb Continuing efforts to provide hardening of FFS, this change adds a
check hash to cylinder groups. If a check hash fails when a cylinder
group is read, no further allocations are attempted in that cylinder
group until it has been fixed by fsck. This avoids a class of
filesystem panics related to corrupted cylinder group maps. The
hash is done using crc32c.

Check hases are added only to UFS2 and not to UFS1 as UFS1 is primarily
used in embedded systems with small memories and low-powered processors
which need as light-weight a filesystem as possible.

Specifics of the changes:

sys/sys/buf.h:
    Add BX_FSPRIV to reserve a set of eight b_xflags that may be used
    by individual filesystems for their own purpose. Their specific
    definitions are found in the header files for each filesystem
    that uses them. Also add fields to struct buf as noted below.

sys/kern/vfs_bio.c:
    It is only necessary to compute a check hash for a cylinder
    group when it is actually read from disk. When calling bread,
    you do not know whether the buffer was found in the cache or
    read. So a new flag (GB_CKHASH) and a pointer to a function to
    perform the hash has been added to breadn_flags to say that the
    function should be called to calculate a hash if the data has
    been read. The check hash is placed in b_ckhash and the B_CKHASH
    flag is set to indicate that a read was done and a check hash
    calculated. Though a rather elaborate mechanism, it should
    also work for check hashing other metadata in the future. A
    kernel internal API change was to change breada into a static
    fucntion and add flags and a function pointer to a check-hash
    function.

sys/ufs/ffs/fs.h:
    Add flags for types of check hashes; stored in a new word in the
    superblock. Define corresponding BX_ flags for the different types
    of check hashes. Add a check hash word in the cylinder group.

sys/ufs/ffs/ffs_alloc.c:
    In ffs_getcg do the dance with breadn_flags to get a check hash and
    if one is provided, check it.

sys/ufs/ffs/ffs_vfsops.c:
    Copy across the BX_FFSTYPES flags in background writes.
    Update the check hash when writing out buffers that need them.

sys/ufs/ffs/ffs_snapshot.c:
    Recompute check hash when updating snapshot cylinder groups.

sys/libkern/crc32.c:
lib/libufs/Makefile:
lib/libufs/libufs.h:
lib/libufs/cgroup.c:
    Include libkern/crc32.c in libufs and use it to compute check
    hashes when updating cylinder groups.

Four utilities are affected:

sbin/newfs/mkfs.c:
    Add the check hashes when building the cylinder groups.

sbin/fsck_ffs/fsck.h:
sbin/fsck_ffs/fsutil.c:
    Verify and update check hashes when checking and writing cylinder groups.

sbin/fsck_ffs/pass5.c:
    Offer to add check hashes to existing filesystems.
    Precompute check hashes when rebuilding cylinder group
    (although this will be done when it is written in fsutil.c
    it is necessary to do it early before comparing with the old
    cylinder group)

sbin/dumpfs/dumpfs.c
    Print out the new check hash flag(s)

sbin/fsdb/Makefile:
    Needs to add libufs now used by pass5.c imported from fsck_ffs.

Reviewed by: kib
Tested by: Peter Holm (pho)
2017-09-22 12:45:15 +00:00
Ryan Libby 9de5f67de2 x86/crc32_sse42.c: quiet unused function warning
Reviewed by:	cem
Approved by:	markj (mentor)
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D11980
2017-08-11 17:05:31 +00:00
Ed Maste c2aa86d19c arm64: add ".arch armv8-a+crc" to allow use of crc instructions
With Clang 5.0 the .arch directive is required, otherwise Clang
complains "error: instruction requires: crc".

This was reported in D10499 but not added initially, because clang 3.8
available on a ref machine reported unknown directive.  Clang 4.0 allows
but does not require the directive.

Submitted by:	andrew
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2017-06-08 20:06:09 +00:00
Xin LI da31cbbca0 Sync qsort.c with userland r318515.
(Note that MIN macro is removed in favor of sys/param.h's version).

PR:		213922
2017-05-19 06:37:16 +00:00
Ed Maste 3e85b721d6 Remove register keyword from sys/ and ANSIfy prototypes
A long long time ago the register keyword told the compiler to store
the corresponding variable in a CPU register, but it is not relevant
for any compiler used in the FreeBSD world today.

ANSIfy related prototypes while here.

Reviewed by:	cem, jhb
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D10193
2017-05-17 00:34:34 +00:00
Michael Tuexen d7f27102b5 armv8 has support for optional CRC32C instructions. This patch checks if they are
available and if that is true make use of them.
Thank you very much to Andrew Turner for providing help and review the patch!
Reviewed by:		andrew
MFC after:		1 week
Differential Revision:	https://reviews.freebsd.org/D10499
2017-04-27 17:53:05 +00:00
Mark Murray 150890b0c6 Replace the RC4 algorithm for generating in-kernel secure random
numbers with Chacha20. Keep the API, though, as that is what the
other *BSD's have done.

Use the boot-time entropy stash (if present) to bootstrap the
in-kernel entropy source.

Reviewed by: delphij,rwatson
Approved by: so(delphij)
MFC after: 2 months
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D10048
2017-04-16 09:11:02 +00:00
Bruce Evans 864c28cf81 Use inline asm instead of unportable intrinsics for the SSE4 crc32
optimization.

This fixes building with gcc-4.2.1 (it doesn't support SSE4).
gas-2.17.50 [FreeBSD] supports SSE4 instructions, so this doesn't
need using .byte directives.

This fixes depending on host user headers in the kernel.

Fix user includes (don't depend on namespace pollution in <nmmintrin.h>
that is not included now).

The instrinsics had no advantages except to sometimes avoid compiler
pessimixations.  clang understands them a bit better than inline asm,
and generates better looking code which also runs better for cem, but
for me it just at the same speed or slower by doing excessive
unrollowing in all the wrong places.  gcc-4.2.1 also doesn't understand
what it is doing with unrolling, but with -O3 somehow it does more
unrolling that helps.

Reduce 1 of the the compiler pessimizations (copying a variable which
already satisfies an "rm" constraint in a good way by being in memory
and not used again, to different memory and accessing it there.  Force
copying it to a register instead).

Try to optimize the inner loops significantly, so as to run at full
speed on smaller inputs.  The algorithm is already very MD, and was
tuned for the throughput of 3 crc32 instructions per cycle found on
at least Sandybridge through Haswell.  Now it is even more tuned for
this, so depends more on the compiler not rearranging or unrolling
things too much.  The main inner loop for should have no difficulty
runing at full speed on these CPUs unless the compiler unrolls it too
much.  However, the main inner loop wasn't even used for buffers smaller
than 24K.  Now it is used for buffers larger than 384 bytes.  Now it
is not so long, and the main outer loop is used more.  The new
optimization is to try to arrange that the outer loop runs in parallel
with the next inner loop except for the final iteration; then reduce
the loop sizes significantly to take advantage of this.

Approved by:	cem
Not tested in production by:	bde
2017-03-26 10:31:48 +00:00
Xin LI 91868665a9 Discard first 3072 bytes of RC4 keystream, this is a bandaid
that allows us to work on switching to a more modern PRNG.

Submitted by:	Steven Chamberlain <steven pyro eu org>
Approved by:	so
2017-03-14 06:00:44 +00:00
Warner Losh 59b34b3404 strstr.c was inadvertently blasted with a copy of isa_nmi.c. Revert
and remove clause 3 while I'm here.
2017-03-01 02:07:51 +00:00
Warner Losh fbbd9655e5 Renumber copyright clause 4
Renumber cluase 4 to 3, per what everybody else did when BSD granted
them permission to remove clause 3. My insistance on keeping the same
numbering for legal reasons is too pedantic, so give up on that point.

Submitted by:	Jan Schaumann <jschauma@stevens.edu>
Pull Request:	https://github.com/freebsd/freebsd/pull/96
2017-02-28 23:42:47 +00:00
Pedro F. Giffuni e099b90b80 sys: Replace zero with NULL for pointers.
Found with:	devel/coccinelle
MFC after:	3 weeks
Differential Revision:	https://reviews.freebsd.org/D9694
2017-02-22 02:35:59 +00:00
Eric van Gyzen 922193e7ff Remove inet_ntoa() from the kernel
inet_ntoa() cannot be used safely in a multithreaded environment
because it uses a static local buffer.  Remove it from the kernel.

Suggested by:	glebius, emaste
Reviewed by:	gnn
MFC after:	never
Sponsored by:	Dell EMC
Differential Revision:	https://reviews.freebsd.org/D9625
2017-02-16 20:50:01 +00:00
Conrad Meyer 6be2ff7d3e calculate_crc32c: Add SSE4.2 implementation on x86
Derived from an implementation by Mark Adler.

The fast loop performs three simultaneous CRCs over subsets of the data
before composing them.  This takes advantage of certain properties of
the CRC32 implementation in Intel hardware.  (The CRC instruction takes 1
cycle but has 2-3 cycles of latency.)

The CRC32 instruction does not manipulate FPU state.

i386 does not have the crc32q instruction, so avoid it there.  Otherwise
the implementation is identical to amd64.

Add basic userland tests to verify correctness on a variety of inputs.

PR:		216467
Reported by:	Ben RUBSON <ben.rubson at gmail.com>
Reviewed by:	kib@, markj@ (earlier version)
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D9342
2017-01-31 03:26:32 +00:00
Conrad Meyer 90a79ac576 Use time_t for intermediate values to avoid overflow in clock_ts_to_ct
Add additionally safety and overflow checks to clock_ts_to_ct and the
BCD routines while we're here.

Perform a safety check in sys_clock_settime() first to avoid easy local
root panic, without having to propagate an error value back through
dozens of APIs currently lacking error returns.

PR:		211960, 214300
Submitted by:	Justin McOmie <justin.mcomie at gmail.com>, kib@
Reported by:	Tim Newsham <tim.newsham at nccgroup.trust>
Reviewed by:	kib@
Sponsored by:	Dell EMC Isilon, FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D9279
2017-01-24 18:05:29 +00:00
Conrad Meyer 484820d442 libkern: Remove obsolete 'register' keyword
Sponsored by:	Dell EMC Isilon
2017-01-12 17:02:29 +00:00
Fabien Thomas cdaf963483 Update r309143 to prevent false sharing.
Reported by:	mjg
Approved by:	so
MFC after:	1 month
2016-11-25 17:20:23 +00:00
Fabien Thomas dcf3302859 In a dual processor system (2*6 cores) during IPSec throughput tests,
we see a lot of contention on the arc4 lock, used to generate the IV
of the ESP output packets.

The idea of this patch is to split this mutex in order to reduce the
contention on this lock.

Reviewed by:	delphij, markm, ache
Approved by:	so
Obtained from: emeric.poupon@stormshield.eu
MFC after: 1 month
Sponsored by:	Stormshield
Differential Revision:	https://reviews.freebsd.org/D8130
2016-11-25 13:49:33 +00:00
Pedro F. Giffuni 4ed3c0e713 sys: Make use of our rounddown() macro when sys/param.h is available.
No functional change.
2016-04-30 14:41:18 +00:00
Jung-uk Kim f8146b882b Merge ACPICA 20160422. 2016-04-27 19:09:21 +00:00
Pedro F. Giffuni b85f65af68 kern: for pointers replace 0 with NULL.
These are mostly cosmetical, no functional change.

Found with devel/coccinelle.
2016-04-15 16:10:11 +00:00
Conrad Meyer 65327c21ed libkern: ffs, fls: s/4/3/ the 3rd BSD clause
Approved by:	emaste
Sponsored by:	EMC / Isilon Storage Division
2015-10-22 21:04:47 +00:00
Conrad Meyer 3d3e385e75 Add libkern ffsll() for parity with flsll()
Sponsored by:	EMC / Isilon Storage Division
Differential Revision:	https://reviews.freebsd.org/D3962
2015-10-22 20:28:37 +00:00
Andrew Turner 756508c7f6 Add the __aeabi_memclr8 symbol, clang 3.7 uses this.
MFC after:	1 week
Sponsored by:	ABT Systems Lts
2015-09-21 18:35:32 +00:00
Andrew Turner 3303004f1a Remove checks for __ARM_EABI__, we only build for EABI now.
Sponsored by:	ABT Systems Ltd
2015-07-09 21:02:40 +00:00
Andrew Turner 6c50960be6 Add support for __aeabi_memclr4, clang 3.7 calls it.
Sponsored by:	ABT Systems Ltd
2015-07-09 20:54:38 +00:00
Mark Murray d1b06863fb Huge cleanup of random(4) code.
* GENERAL
- Update copyright.
- Make kernel options for RANDOM_YARROW and RANDOM_DUMMY. Set
  neither to ON, which means we want Fortuna
- If there is no 'device random' in the kernel, there will be NO
  random(4) device in the kernel, and the KERN_ARND sysctl will
  return nothing. With RANDOM_DUMMY there will be a random(4) that
  always blocks.
- Repair kern.arandom (KERN_ARND sysctl). The old version went
  through arc4random(9) and was a bit weird.
- Adjust arc4random stirring a bit - the existing code looks a little
  suspect.
- Fix the nasty pre- and post-read overloading by providing explictit
  functions to do these tasks.
- Redo read_random(9) so as to duplicate random(4)'s read internals.
  This makes it a first-class citizen rather than a hack.
- Move stuff out of locked regions when it does not need to be
  there.
- Trim RANDOM_DEBUG printfs. Some are excess to requirement, some
  behind boot verbose.
- Use SYSINIT to sequence the startup.
- Fix init/deinit sysctl stuff.
- Make relevant sysctls also tunables.
- Add different harvesting "styles" to allow for different requirements
  (direct, queue, fast).
- Add harvesting of FFS atime events. This needs to be checked for
  weighing down the FS code.
- Add harvesting of slab allocator events. This needs to be checked for
  weighing down the allocator code.
- Fix the random(9) manpage.
- Loadable modules are not present for now. These will be re-engineered
  when the dust settles.
- Use macros for locks.
- Fix comments.

* src/share/man/...
- Update the man pages.

* src/etc/...
- The startup/shutdown work is done in D2924.

* src/UPDATING
- Add UPDATING announcement.

* src/sys/dev/random/build.sh
- Add copyright.
- Add libz for unit tests.

* src/sys/dev/random/dummy.c
- Remove; no longer needed. Functionality incorporated into randomdev.*.

* live_entropy_sources.c live_entropy_sources.h
- Remove; content moved.
- move content to randomdev.[ch] and optimise.

* src/sys/dev/random/random_adaptors.c src/sys/dev/random/random_adaptors.h
- Remove; plugability is no longer used. Compile-time algorithm
  selection is the way to go.

* src/sys/dev/random/random_harvestq.c src/sys/dev/random/random_harvestq.h
- Add early (re)boot-time randomness caching.

* src/sys/dev/random/randomdev_soft.c src/sys/dev/random/randomdev_soft.h
- Remove; no longer needed.

* src/sys/dev/random/uint128.h
- Provide a fake uint128_t; if a real one ever arrived, we can use
  that instead. All that is needed here is N=0, N++, N==0, and some
  localised trickery is used to manufacture a 128-bit 0ULLL.

* src/sys/dev/random/unit_test.c src/sys/dev/random/unit_test.h
- Improve unit tests; previously the testing human needed clairvoyance;
  now the test will do a basic check of compressibility. Clairvoyant
  talent is still a good idea.
- This is still a long way off a proper unit test.

* src/sys/dev/random/fortuna.c src/sys/dev/random/fortuna.h
- Improve messy union to just uint128_t.
- Remove unneeded 'static struct fortuna_start_cache'.
- Tighten up up arithmetic.
- Provide a method to allow eternal junk to be introduced; harden
  it against blatant by compress/hashing.
- Assert that locks are held correctly.
- Fix the nasty pre- and post-read overloading by providing explictit
  functions to do these tasks.
- Turn into self-sufficient module (no longer requires randomdev_soft.[ch])

* src/sys/dev/random/yarrow.c src/sys/dev/random/yarrow.h
- Improve messy union to just uint128_t.
- Remove unneeded 'staic struct start_cache'.
- Tighten up up arithmetic.
- Provide a method to allow eternal junk to be introduced; harden
  it against blatant by compress/hashing.
- Assert that locks are held correctly.
- Fix the nasty pre- and post-read overloading by providing explictit
  functions to do these tasks.
- Turn into self-sufficient module (no longer requires randomdev_soft.[ch])
- Fix some magic numbers elsewhere used as FAST and SLOW.

Differential Revision: https://reviews.freebsd.org/D2025
Reviewed by: vsevolod,delphij,rwatson,trasz,jmg
Approved by: so (delphij)
2015-06-30 17:00:45 +00:00
Andrew Turner 97f8390e7d Add more __aeabi_memcpy functions, later versions of clang generate calls
to these functions.
2015-05-31 07:31:20 +00:00
Craig Rodrigues d9db52256e Move zlib.c from net to libkern.
It is not network-specific code and would
be better as part of libkern instead.
Move zlib.h and zutil.h from net/ to sys/
Update includes to use sys/zlib.h and sys/zutil.h instead of net/

Submitted by:		Steve Kiernan stevek@juniper.net
Obtained from:		Juniper Networks, Inc.
GitHub Pull Request:	https://github.com/freebsd/freebsd/pull/28
Relnotes:		yes
2015-04-22 14:38:58 +00:00
Ryan Stone 3cd4533107 Implement asprintf in libkern
Differential Revision:		https://reviews.freebsd.org/D1877
Reviewed by:			pjd, jfv
MFC After:			1 month
Sponsored by:			Sandvine Inc.
2015-03-01 00:22:16 +00:00
Dimitry Andric 0b77a417c8 Similar to r277901, fix more -Wcast-qual warnings in libkern's strtoq(),
strtoul() and strtouq(), by using __DECONST.  No functional change.

MFC after:	3 days
2015-01-29 21:54:01 +00:00
Dimitry Andric b94d92e5c8 Fix a -Wcast-qual warning in libkern's strtol(), by using __DECONST. No
functional change.

MFC after:	3 days
2015-01-29 21:16:45 +00:00
John-Mark Gurney 08fca7a56b Add some new modes to OpenCrypto. These modes are AES-ICM (can be used
for counter mode), and AES-GCM.  Both of these modes have been added to
the aesni module.

Included is a set of tests to validate that the software and aesni
module calculate the correct values.  These use the NIST KAT test
vectors.  To run the test, you will need to install a soon to be
committed port, nist-kat that will install the vectors.  Using a port
is necessary as the test vectors are around 25MB.

All the man pages were updated.  I have added a new man page, crypto.7,
which includes a description of how to use each mode.  All the new modes
and some other AES modes are present.  It would be good for someone
else to go through and document the other modes.

A new ioctl was added to support AEAD modes which AES-GCM is one of them.
Without this ioctl, it is not possible to test AEAD modes from userland.

Add a timing safe bcmp for use to compare MACs.  Previously we were using
bcmp which could leak timing info and result in the ability to forge
messages.

Add a minor optimization to the aesni module so that single segment
mbufs don't get copied and instead are updated in place.  The aesni
module needs to be updated to support blocked IO so segmented mbufs
don't have to be copied.

We require that the IV be specified for all calls for both GCM and ICM.
This is to ensure proper use of these functions.

Obtained from:	p4: //depot/projects/opencrypto
Relnotes:	yes
Sponsored by:	FreeBSD Foundation
Sponsored by:	NetGate
2014-12-12 19:56:36 +00:00
Andrew Turner 524bca9008 Use the unified syntax in a few more assembly files
MFC after:	1 week
Sponsored by:	ABT Systems Ltd
2014-12-05 19:08:36 +00:00
Xin LI 4facd36ca0 Sync with userland variant.
MFC after:	2 weeks
2014-11-17 21:01:35 +00:00
Dag-Erling Smørgrav 99e9de871a Add a complete implementation of MurmurHash3. Tweak both implementations
so they match the established idiom.  Document them in hash(9).

MFC after:	1 month
MFC with:	r272906
2014-10-18 22:15:11 +00:00
Xin LI d427f43c25 Remove duplicated code.
Suggested by:	jmg
2014-10-16 22:04:07 +00:00
George V. Neville-Neil 1d2baefc13 Change the PF hash from Jenkins to Murmur3. In forwarding tests
this showed a conservative 3% incrase in PPS.

Differential Revision:	https://reviews.freebsd.org/D461
Submitted by:	des
Reviewed by:	emaste
MFC after:	1 month
2014-10-10 19:26:26 +00:00
Xin LI 78b59024b5 Add explicit_bzero(3) and its kernel counterpart.
Obtained from:	OpenBSD
MFC after:	2 weeks
2014-10-07 04:54:11 +00:00
Bjoern A. Zeeb afa0a6efe0 Use __DECONST to avoid compiler warnings (and thus build failures)
with gcc on sparc64, mips, and powerpc after r271173.
2014-09-08 08:12:09 +00:00
Benno Rice 27ecc2adbc Add support for gdb's memory searching capabilities to our in-kernel gdb
server.

Submitted by:	Daniel O'Connor <daniel.oconnor@isilon.com>
Reviewed by:	jhb
Sponsored by:	EMC Isilon Storage Division
2014-09-05 16:40:47 +00:00
Edward Tomasz Napierala 3914ddf8a7 Bring in the new automounter, similar to what's provided in most other
UNIX systems, eg. MacOS X and Solaris.  It uses Sun-compatible map format,
has proper kernel support, and LDAP integration.

There are still a few outstanding problems; they will be fixed shortly.

Reviewed by:	allanjude@, emaste@, kib@, wblock@ (earlier versions)
Phabric:	D523
MFC after:	2 weeks
Relnotes:	yes
Sponsored by:	The FreeBSD Foundation
2014-08-17 09:44:42 +00:00
Ian Lepore 25166187e3 Fix unwind-info errors in our hand-written arm assembler code.
We have functions nested within functions, and places where we start a
function then never end it, we just jump to the middle of something else.
We tried to express this with nested ENTRY()/END() macros (which result
in .fnstart and .fnend directives), but it turns out there's no way to
express that nesting in ARM EHABI unwind info, and newer tools treat
multiple .fnstart directives without an intervening .fnend as an error.

These changes introduce two new macros, EENTRY() and EEND().  EENTRY()
creates a global label you can call/jump to just like ENTRY(), but it
doesn't emit a .fnstart.  EEND() is a no-op that just documents the
conceptual endpoint that matches up with the same-named EENTRY().

This is based on patches submitted by Stepan Dyatkovskiy, but I made some
changes and added the EEND() stuff, so blame any problems on me.

Submitted by:	Stepan Dyatkovskiy <stpworld@narod.ru>
2014-08-01 18:24:44 +00:00
Marcel Moolenaar e7d939bda2 Remove ia64.
This includes:
o   All directories named *ia64*
o   All files named *ia64*
o   All ia64-specific code guarded by __ia64__
o   All ia64-specific makefile logic
o   Mention of ia64 in comments and documentation

This excludes:
o   Everything under contrib/
o   Everything under crypto/
o   sys/xen/interface
o   sys/sys/elf_common.h

Discussed at: BSDcan
2014-07-07 00:27:09 +00:00
John Baldwin 0cbce0671e Use strcasecmp() instead of strcmp() when checking user-supplied encoding
names so that encoding names are treated as case-insensitive.  This allows
the use of 'utf-8' instead of 'UTF-8' for example and matches the behavior
of iconv(1).

PR:		167977
Submitted by:	buganini@gmail.com
MFC after:	1 week
2014-06-09 19:27:47 +00:00
Eitan Adler 04660ed15f Fix typo
Reported by:	emaste
2013-11-29 20:14:26 +00:00
Eitan Adler 5c41c9d246 Fix typo
Reported by:	swildner@DragonFlyBSD.org
2013-11-29 20:12:02 +00:00
Andrew Turner ca49624410 Fix ixp425 boot2 with ARM EABI:
- libkern is missing __aeabi_llsl, implement this by calling __ashldi3.
 - Because of how the asm entry macros are defined the boot2 code
   requires the unwind symbols to exist, include them in boot2.

Approved by:	re (marius)
2013-09-29 15:19:34 +00:00
Alfred Perlstein 3d30404f83 Fix watchdog pretimeout.
The original API calls for pow2ns, however the new APIs from
Linux call for seconds.

We need to be able to convert to/from 2^Nns to seconds in both
userland and kernel to fix this and properly compare units.
2013-07-27 20:47:01 +00:00
Andrey A. Chernov 2b50ce65be Attempt to mitigate poor initialization of arc4 by one-shot
reinitialization from yarrow right after good entropy is harvested.

Approved by:    secteam (delphij)
MFC after:      1 week
2013-04-19 00:30:52 +00:00
Andrew Turner 69d758b4b8 Add __aeabi_memset to libkern, implemented using memset, as clang may
generate calls to it.
2013-03-16 23:11:55 +00:00
Andrew Turner a2dee2ad37 Add END to ARM libkern assembly functions 2013-03-16 04:08:01 +00:00
Andrew Turner 5cc8d54ca0 Move the __aeabi_unwind_cpp_pr{0,1,2} functions to libkern so they can be
referenced in a non-debug kernel.
2013-03-16 04:06:49 +00:00
Andrew Turner 530bff3f19 Add __aeabi_memcpy to libkern as clang may generate calls to it. 2013-03-10 07:55:40 +00:00
Andrew Turner 31735bf308 Add __aeabi_ulcmp to allow building of the LINT kernel. 2013-01-23 09:18:18 +00:00
Andrew Turner 46d741dc7f Add the required __aeabi_* functions the kernel uses when built for ARM EABI 2013-01-17 09:37:42 +00:00
Xin LI 299ff0d7cc Sync strlcpy with userland version.
MFC after:	1 month
2012-11-03 04:29:24 +00:00
Xin LI 6afdae4139 Sync strlen with userland implementation.
MFC after:	1 month
2012-11-03 04:28:53 +00:00
Eitan Adler 0af1b47258 s/ is is / is /g
s/ a a / a /g

Approved by:	cperciva
MFC after:	3 days
2012-09-14 22:00:03 +00:00
Gleb Smirnoff 62208ca5d2 - Move jenkins.h to jenkins_hash.c
- Provide missing function that can do hashing of arbitrary sized buffer.
- Refetch lookup3.c and do only minimal edits to it, so that diff between
  our jenkins_hash.c and lookup3.c is minimal.
- Add declarations for jenkins_hash(), jenkins_hash32() to sys/hash.h.
- Document these functions in hash(9)

Obtained from:	http://burtleburtle.net/bob/c/lookup3.c
2012-09-04 12:07:33 +00:00
Mateusz Guzik 414d9b2ed0 Fix unloading of libiconv module.
Previously it would either loop infinitely or exit with error leaking a lock.

Reported by:	Will DeVries
Approved by:	trasz (mentor)
MFC after:	1 week
2012-06-11 17:42:39 +00:00
Kevin Lo fa27760e2e Use strcmp that I replaced by accident. 2012-05-21 02:45:47 +00:00
Kevin Lo 040b0e4eda Fix broken ref count
Submitted by:	gcooper
2012-05-21 02:41:15 +00:00
Kevin Lo f529372e8a Fix improper handling of variadic args with ICDEBUG
PR:	kern/168095
Submitted by:	gcooper
2012-05-21 02:30:22 +00:00
Kevin Lo bc6f23e0ad Remove dead code 2012-05-21 02:23:20 +00:00
Marius Strobl f7b9ae0882 Remove second consts in r233288 in order to appease C++ compilers.
While at it, remove some style(9) bugs in libkern.h.

Submitted by:	kan
2012-03-26 18:22:04 +00:00
Marius Strobl cc0c154ffb Declare the CRC lookup-tables const as they hardly should change at
run-time.
2012-03-21 20:55:21 +00:00
Ed Schouten 22fec34a57 Implement extensions on top of standards instead of the other way around.
Now that index() and rindex() have become unused, simply turn them into
wrappers around strchr() and strrchr(), respectively.
2012-01-03 07:05:30 +00:00
Ed Schouten dc15eac046 Use strchr() and strrchr().
It seems strchr() and strrchr() are used more often than index() and
rindex(). Therefore, simply migrate all kernel code to use it.

For the XFS code, remove an empty line to make the code identical to
the code in the Linux kernel.
2012-01-02 12:12:10 +00:00
Ed Schouten 41adcfa1d0 Remove the now unused skpc() function.
It was only used by ufs and ext2 and I have really strong doubts that
there are other pieces of code that also use this function. If it turns
out that external drivers use this code as well, I'd be happy to migrate
or revert.

Bump __FreeBSD_version while there.
2012-01-01 20:54:44 +00:00