Commit graph

577 commits

Author SHA1 Message Date
Gleb Smirnoff 2bab0c5535 New sendfile(2) syscall. A joint effort of NGINX and Netflix from 2013 and
up to now.

The new sendfile is the code that Netflix uses to send their multiple tens
of gigabits of data per second. The new implementation features asynchronous
I/O, when I/O operations are launched, but not awaited to be complete. An
explanation of why such behavior is beneficial compared to old one is
going to be too long for a commit message, so we will skip it here.

Additional features of new syscall are extra flags, which provide an
application more control over data sent. The SF_NOCACHE flag tells
kernel that data shouldn't be cached after it was sent. The SF_READAHEAD()
macro allows to specify readahead size in pages.

The new syscalls is a drop in replacement. No modifications are required
to applications. One can take nginx binary for stable/10 and run it
successfully on head. Although SF_NODISKIO lost its original sense, as now
sendfile doesn't block, and now means something completely different (tm),
using the new sendfile the old way is absolutely safe.

Celebrates:	Netflix global launch!
Sponsored by:	Nginx, Inc.
Sponsored by:	Netflix
Relnotes:	yes
2016-01-08 20:34:57 +00:00
George V. Neville-Neil 9d2d8e7bec Switch the IPsec related statistics to using the built in sysctl
variable set rather than reading from kernel memory.
This also makes the -z (zero) flag work correctly

MFC after:	1 week
Sponsored by: Rubicon Communications, LLC (Netgate)
Differential Revision:	https://reviews.freebsd.org/D4591
2015-12-17 02:02:09 +00:00
Craig Rodrigues 06691045ba Add more text to explain --libxo flag. 2015-12-01 19:18:53 +00:00
Hajimu UMEMOTO 4fc31adf6a At the time a destination or a gateway of `netstat -r'
protrudes its field, narrow the next field to raise
readability bit.
2015-12-01 16:04:50 +00:00
Bryan Drewery cf990407e1 Update dependencies after r291406 added libelf to libkvm.
Unfortunately filemon/meta mode tracks all indirect dependencies here
since ld(1) is reading libelf when linking in libkvm.  Churn would be
reduced if this was able to be limited to direct dependencies.

Sponsored by:	EMC / Isilon Storage Division
2015-12-01 05:18:48 +00:00
Bryan Drewery 114350b9de Replace DPSRCS that work fine in SRCS.
This is so that 'make depend' is not a required build step in these
files.

DPSRCS is overall unneeded.  DPSRCS already contains SRCS, so anything
which can safely be in SRCS should be.  DPSRCS is mostly just a way to
generate files that should not be linked into the final PROG/LIB.  For
headers and grammars it is safe for them to be in SRCS since they will
be excluded during linking and installation.

The only remaining uses of DPSRCS are for generating .c or .o files that
must be built before 'make depend' can run 'mkdep' on the SRCS c files
list.  A semi-proper example is in tests/sys/kern/acct/Makefile where a
checked-in .c file has an #include on a generated .c file.  The
generated .c file should not be linked into the final PROG though since
it is #include'd.  The more proper way here is just to build/link it in
though without DPSRCS.  Another example is in sys/modules/linux/Makefile
where a shell script runs to parse a DPSRCS .o file that should not be
linked into the module.  Beyond those, the need for DPSRCS is largely
unneeded, redundant, and forces 'make depend' to be ran.  Generally,
these Makefiles should avoid the need for DPSRCS and define proper
dependencies for their files as well.

An example of an improper usage and why this matters is in usr.bin/netstat.
nl_defs.h was only in DPSRCS and so was not generated during 'make all',
but only during 'make depend'.  The files including it lacked proper
depenencies on it, which forced running 'make depend' to workaround that
bug.  The 'make depend' target should mostly be used for incremental build
help, not to produce a working build.  This specific example was broken in
the meta build until r287905 since it does not run 'make depend'.

The gnu/lib/libreadline/readline case is fine since bsd.lib.mk has 'OBJS:
SRCS:M*.h' when there is no .depend file.

Sponsored by:	EMC / Isilon Storage Division
MFC after:	1 week
2015-11-25 20:38:17 +00:00
Hajimu UMEMOTO 0eaa116e1e Fix udp entry of `netstat -TW'. 2015-11-25 11:20:54 +00:00
Hajimu UMEMOTO 046aad399a Correct alignment of the addresses in the `netstat -aW' output. 2015-11-24 14:25:40 +00:00
Hajimu UMEMOTO 06ff7ccb5f Add missing error check after xo_parse_args() in netstat(8).
Submitted by:	Oliver Pinter
Differential Revision:	https://reviews.freebsd.org/D4233
2015-11-24 11:07:37 +00:00
Hajimu UMEMOTO 857357b6c9 Don't truncate an interface name when -W option is specified.
Spotted by:	Jim Thompson <jim__at__netgate.com>
MFC after:	1 week
2015-11-20 12:32:49 +00:00
Hajimu UMEMOTO b1a302a533 Avoid core dump when output style is html. 2015-11-20 12:15:58 +00:00
Hajimu UMEMOTO d48d68fa9e JSON doesn't permit a hexadecimal notation of an integer. 2015-11-17 12:09:57 +00:00
Hajimu UMEMOTO b670e89ac2 Do not truncate addresses when printing in encoded format. 2015-11-06 14:50:23 +00:00
Hajimu UMEMOTO fb3ab86f32 - Fix alignment for padding link address.
- Trim whitespace of link address.
2015-11-06 14:35:22 +00:00
Enji Cooper f9b3502ce1 Fix compiling netstat after r290367 by substituting sys/types.h for
sys/param.h, as sys/param.h defines the MAX(..) macro

Reported by: O. Hartmann <ohartman@zedat.fu-berlin.de>
Pointyhat to: ume
2015-11-06 08:43:12 +00:00
Hajimu UMEMOTO cca052c621 Give enough room for addresses when -W option is specified. 2015-11-05 11:06:46 +00:00
Hajimu UMEMOTO 011ae9c907 Fix alignment of `Drop' header. 2015-11-05 11:04:43 +00:00
Hajimu UMEMOTO f3ffc9fdf9 Use returned network name from getnetbyaddr() correctly. 2015-11-05 11:02:28 +00:00
Hajimu UMEMOTO 6f53a03868 Revert previous workaround. This problem was fixed
by r290318.
2015-11-05 10:58:19 +00:00
Hajimu UMEMOTO 6ad5f7ca01 Since sa->sa_len doesn't match sizeof(struct sockaddr_dl),
getnameinfo() fails against sockaddr_dl.  This commit is workaround
for this problem.
2015-11-04 19:09:42 +00:00
Hajimu UMEMOTO 3e687cd867 Fix alignment of AF_LINK address. 2015-11-04 19:05:04 +00:00
Hajimu UMEMOTO 86a598950f Simplify r290367 using asterisk for a field width
and precision.
2015-11-04 16:59:12 +00:00
Hajimu UMEMOTO 38f16ca658 Unify AF_INET case and AF_INET6 case. 2015-11-04 15:05:15 +00:00
Hajimu UMEMOTO bd4fcecaca Change to not truncate an interface name when -W option
is specified.
2015-11-04 14:57:46 +00:00
Hajimu UMEMOTO 38507af333 Mask an IPv6 network address. 2015-11-04 14:47:10 +00:00
Hajimu UMEMOTO 8e2d18a7c5 Restore an interface name field to left align. 2015-11-04 13:59:13 +00:00
Bryan Drewery 42c4cf86d4 Update META_MODE dependencies. 2015-09-17 05:06:34 +00:00
Mark Johnston 9eddb899d9 Use a common subroutine to fetch and zero protocol stats instead of
duplicating roughly similar code for each protocol.

MFC after:	2 weeks
2015-09-11 04:37:01 +00:00
Mark Johnston 2bdd6ea2ad Remove prototypes for undefined functions from netstat.h. 2015-09-11 04:02:05 +00:00
Hiroki Sato 81dacd8beb Simplify kvm symbol resolution and error handling. The symbol table
nl_symbols will eventually be organized into several modules depending
on MK_* variables.
2015-09-02 18:51:36 +00:00
Hiroki Sato a261e36179 Divide statistics in the number of packets with 1000 instead of 1024
in human-readable form.

PR:	183598
2015-09-02 18:42:35 +00:00
Hiroki Sato 10d5269ff9 - Add -W flag support for network column in intpr() (-i flag) and
routepr() (-r flag).  It is too narrow to show an IPv6 prefix
  in most cases.

- Accept "local" as a synonym of "unix" in protocol family name.

- Show a prefix length in CIDR notation when name resolution failed in
  netname().

- Make routename() and netname() AF-independent and remove
  unnecessary typecasting from struct sockaddr.

- Use getnameinfo(3) to format L2 addr in intpr().

- Fix a bug which showed "Address" when -A flag is specfied in pr_rthdr().

- Replace cryptic GETSA() macro with SA_SIZE().

- Fix declarations shadowing local variables with the same names.

- Add more static, remove unused header files and variables.

MFC after:	1 week
2015-09-01 08:42:04 +00:00
Xin LI a4a889a0ef Failure of dropping privilege should be fatal, so test and bail out
when setgid() fails.

Reported by:	clang static analyzer
MFC after:	2 weeks
2015-09-01 06:05:43 +00:00
Michael Tuexen 0835304fea Use the userland exported states and handle them completely. 2015-08-29 09:27:29 +00:00
John-Mark Gurney e7ae88eeda add documentation for timers that silby added in r197244, almost 6 years
ago...
2015-08-27 19:12:42 +00:00
Marcel Moolenaar d1a0d267b7 Upgrade libxo to 0.4.5.
Local changes incorporated by 0.4.5: r284340
Local changes retained: r276260, r282117

Obtained from:	https://github.com/Juniper/libxo
2015-08-24 16:26:20 +00:00
Mark Johnston 794c57d3c5 Fix counter reads on platforms where sizeof(uint64_t) != sizeof(uint64_t *).
In the kernel, structs such as tcpstat are manipulated as an array of
counter_u64_t (uint64_t *), but made visible to userland as an array of
uint64_t. kread_counters() was previously copying the counter array into
user space and sequentially overwriting each counter with its value. This
mostly affects IPsec counters, as other counters are exported via sysctl.

PR:		201700
Tested by:	Jason Unovitch
MFC after:	1 week
2015-07-21 23:57:38 +00:00
Mark Johnston 4ffc6e480d Fix some libxo format string errors in the pfkey stats code.
PR:	201700
2015-07-20 02:38:51 +00:00
George V. Neville-Neil 987de84445 New AES modes for IPSec, user space components.
Update setkey and libipsec to understand aes-gcm-16 as an
encryption method.

A partial commit of the work in review D2936.

Submitted by:	eri
Reviewed by:	jmg
MFC after:	2 weeks
Sponsored by:	Rubicon Communications (Netgate)
2015-07-03 20:09:14 +00:00
Simon J. Gerraty ccfb965433 Add META_MODE support.
Off by default, build behaves normally.
WITH_META_MODE we get auto objdir creation, the ability to
start build from anywhere in the tree.

Still need to add real targets under targets/ to build packages.

Differential Revision:       D2796
Reviewed by: brooks imp
2015-06-13 19:20:56 +00:00
Simon J. Gerraty 44d314f704 dirdeps.mk now sets DEP_RELDIR 2015-06-08 23:35:17 +00:00
Simon J. Gerraty 98e0ffaefb Merge sync of head 2015-05-27 01:19:58 +00:00
Luiz Otavio O Souza 1947004980 Properly align the header and the data columns for netstat -r with and
without the -W flag.
2015-05-18 18:03:47 +00:00
Luiz Otavio O Souza 587b51331a Adjust the string format to match the actual number of arguments.
This fix a segmentation fault on ARM when netstat -r is used together with
-W.

This issue was introduced in r279122.
2015-05-17 15:12:55 +00:00
Gleb Smirnoff e42e1fcc64 Printing interface names:
o Restore historical behaviour of appending '*' if interface is down,
  and we have enough space to print it (usually we don't). [1]
o Do not truncate interface names when printing in encoded format.
o Report interface flags into encoded format.

PR:		199873 [1]
Sponsored by:	Nginx, Inc.
2015-05-06 16:43:44 +00:00
Gleb Smirnoff 6fe1796015 Like it was already done for 'netstat -i', drop the kvm(3) support
in 'netstat -r'.

The netstat/route.c was the last abuser of struct ifnet and struct
rtentry in the tree. With this change if_var.h can become kernel
only include, _WANT_RTENTRY can go away and projects/ifnet and
projects/routing can go forward.

Differential Revision:	https://reviews.freebsd.org/D2242
Reviewed by:		melifaro, gnn
Sponsored by:		Nginx, Inc.
Sponsored by:		Netflix
2015-04-07 05:50:45 +00:00
Gleb Smirnoff c151f24d08 o Make net.inet6.ip6.mif6table return special API structure, that doesn't
contain kernel pointers, and instead has interface index.
  Bump __FreeBSD_version for that change.
o Now, netstat/mroute6.c no longer needs to kvm_read(3) struct ifnet, and
  no longer needs to include if_var.h

Note that this change is far from being a complete move of IPv6 multicast
routing to a proper API. Other structures are still dumped into their
sysctls as is, requiring userland application to #define _KERNEL when
including ip6_mroute.h and then call kvm_read(3) to gather all bits and
pieces. But fixing this is out of scope of the opaque ifnet project.

Sponsored by:	Nginx, Inc.
Sponsored by:	Netflix
2015-04-06 22:12:18 +00:00
Gleb Smirnoff 56d5e0967c Stop including if_var.h from userland.
Sponsored by:	Nginx, Inc.
2015-04-06 09:42:23 +00:00
Marcel Moolenaar 315ca438f3 Fix typo in dropped-packets attribute (missing s).
Pointed-out by: allanjude (excellent catch!)
2015-03-04 02:56:32 +00:00
Marcel Moolenaar 65f2833127 Fix "netstat -hdw 1" output.
Reported by: np@
2015-02-25 17:06:27 +00:00