timeout(1) is used by /etc/rc.d/zfskeys. Unfortunately, having
timeout(1) installed in /usr/bin causes problems when /usr is an
encrypted ZFS partition.
Implementing timeout(1) in sh(1) is not trivial. A more elegant solution
is to move timeout(1) to /bin so that it is available to early services
in the boot process.
PR: 265221
Reviewed by: allanjude, des, imp
Approved by: allanjude, des, imp
Reported by: Ivan <r4@sovserv.ru>
Fixes: 33ff39796f Add zfskeys rc.d script for auto-loading encryption keys
MFC after: 1 week
Relnotes: yes
Sponsored by: Modirum MDPay
Sponsored by: Klara Inc.
Differential Revision: https://reviews.freebsd.org/D38344
* The allocated buffer is only used in the fallback case, so move it
there. The argument for passing it in from the caller was that if
malloc(3) were to fail, we'd want it to fail before we started
copying anything, but firstly, it was already not in the right place
to ensure that, and secondly, malloc(3) never fails (except in very
contrived circumstances, such as an unreasonable RLIMIT_AS or
RLIMIT_DATA).
* Remove the mmap(2) option. It is almost never beneficial,
especially when the alternative is copy_file_range(2), and it adds
needless complexity and indentation.
MFC after: 1 week
Sponsored by: Klara, Inc.
Reviewed by: rmacklem, mav
Differential Revision: https://reviews.freebsd.org/D38291
While here, complete the libxo conversion and switch return value to standard constants.
MFC after: 1 week
Sponsored by: Klara, Inc.
Differential revision: https://reviews.freebsd.org/D38097
Introduce new prompt format characters:
- '\[' starts the sequence of non-printing chatacters
- '\]' ends the sequence of non-printing characters
Within these sequences, the following characters are now supported:
- '\a' emits ASCII BEL (0x07, 007) character
- '\e' emits ASCII ESC (0x1b, 033) character
- '\r' emits ASCII CR (0x0d, 015) character
- '\n' emits ASCII CRLF sequence
These can be used to embed ANSI sequences into prompt strings.
Example in .shrc:
PS1="\[\e[7m\]\u@\h\[\e[0m\]:\w \\$ "
This tries to maintain some degree of compatibility with GNU bash,
that uses GNU readline library (which behaves slightly different from
BSD editline): It has two "non-printing boundary" characters:
- RL_PROMPT_START_IGNORE (\001)
- RL_PROMPT_END_IGNORE (\002)
while BSD editline only has one (when using EL_PROMPT_ESC setting), so
for this purpose, ASCII \001 was chosen and both \[ and \] emits
this character.
And while here, enlarge PROMPTLEN from 128 to 192 characters.
Reviewed by: jilles
Approved by: jilles
Differential Revision: https://reviews.freebsd.org/D37701
The version 4 UUID is meant for generating UUIDs from truly-random or
pseudo-random numbers. [1]
bin/uuidgen gained the new flag '-r' to create version 4 UUID.
[1] RFC 4122, https://www.rfc-editor.org/rfc/rfc4122#section-4.4
Reviewed by: pstef
Approved by: bapt
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D37695
Previously when using NO_ROOT we recorded METALOG entries for the /.cshrc
hard link with a different file mode than the link target, which is not
permitted.
We cannot just set LINKMODE here as it would also apply to the hard link
for the tcsh binary.
Reviewed by: brooks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D37499
Previously when using NO_ROOT we recorded a METALOG entry for the
/.profile hard link with a different mode than the link target, which is
not permitted.
Reviewed by: bapt
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D37476
Add a -v flag for ls which sorts entries following a natural ordering
using strverscmp(3) (e.g. "bloem1 bloem9 bloem10" as opposed to
"bloem1 bloem10 bloem9").
Update the manual page and add a test case.
Reviewed by: pauamma, bcr
Tested by: pstef
Differential Revision: https://reviews.freebsd.org/D36407
Describe a shell trick to do non-blocking modification of the terminal
settings, by ignoring job control signals with trap built-in.
PR: 266627
With input from: jilles
Reviewed by: pauamma
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D36745
The ls(1) (with -l option) and find(1) (with -ls option) utilties
segment fault when operating on files with very large modification
times. A recent disk corruption set a spurious bit in the mtime
field of one of my files to 0x8000000630b0167 (576460753965089127)
which is in year 18,266,940,962. I discovered the problem when
running fsck_ffs(8) which uses ctime(3) to convert it to a readable
format. Ctime cannot fit the year into its four character field, so
returns ??? ??? ?? ??:??:?? ???? (typically Thu Nov 24 18:22:48 2021).
With the filesystem mounted, I used `ls -l' to see how it would
report the modification time and it segment faulted. The find(1)
program also segment faulted (see script below). Both these utilities
call the localtime(3) function to decode the modification time.
Localtime(3) returns a pointer to a struct tm (which breaks things
out into its component pieces: year, month, day, hour, minute,
second). The ls(1) and find(1) utilities then print out the date
based on the appropriate fields in the returned tm structure.
Although not documented in the localtime(3) manual page, localtime(3)
returns a NULL pointer if the passed in time translates to a year
that will not fit in an "int" (which if "int" is 32-bits cannot
hold the year 18,266,940,962). Since ls(1) and find(1) do not check
for a NULL struct tm * return from localtime(3), they segment fault
when they try to dereference it.
When localtime(3) returns NULL, the attached patches produce a date
string of "bad date val". This string is chosen because it has the
same number of characters (12) and white spaces (2) as the usual
date string, for example "Sep 3 22:06" or "May 15 2017".
The most recent ANSI standard for localtime(3) does say that localtime(3)
can return NULL (see https://pubs.opengroup.org/onlinepubs/9699919799/
and enter localtime in the search box). Our localtime(3) man page should
be updated to indicate that NULL is a possible return. More importantly,
there are over 100 uses of localtime(3) in the FreeBSD source tree (see
Differential Revision D36474 for the list). Most do not check for a NULL
return from localtime(3).
Reported by: Peter Holm
Reviewed by: kib, Chuck Silvers, Warner Losh
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D36474
This is to avoid loading .shrc which may contain commands that would
result in output different than expected.
Reviewed by: jilles
Differential Revision: https://reviews.freebsd.org/D35876
As per Utility Syntax Guidelines, accept both forms: -l -n and -ln.
To do that, anticipate the source string for the next option that will
be parsed by nextopt(). It's not always *argptr, sometimes it is
nextopt_optptr.
To simplify the check for not_fcnumber, slightly modify nextopt() to
always nullify nextopt_optptr in cases where it would have been set
to point to a NUL character.
Reviewed by: jilles
Differential Revision: https://reviews.freebsd.org/D35836
The use of 'package' in this could be understood to mean a FreeBSD
package provided by pkg, rather than the fact that we use data provided
by IANA. Re-word it to clearly identify `tzdata` as the IANA Time Zone
Database on first use, then drop subsequent uses of the word 'package'.
Reviewed by: 0mp, pauamma, philip
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D35966
The previous description was both incorrect and incomplete in its
description -- the 2038 limit doesn't apply on !i386 platforms, and
it didn't note that values above 100 are accepted and interpreted
differently. Further, it didn't note that absolute years are accepted.
Reviewed by: pauamma_gundo.com (manpages)
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D35360
The sleep time is calculated as the sum of all arguments passed.
This makes the FreeBSD version of sleep functionally compatible with
the version in GNU coreutils.
MFC after: 1 week
The coreutils version of this command accepts a unit designation of s,
m, h, or d (for seconds, minutes, hours, days) immediately following
the number of (fractional) units to delay.
The submitted patch has been modified in one detail: the test meant to
detect the presence of the unit modified was not specific (!= 1) and
would have accepted a non-numeric initial element or extra characters
following the union. The committed version accepts only the number
immediately followed by one of the defined unit designators and no
further characters.
PR: 264162
MFC after: 1 week
The correct logic is a lot simpler than the previous iteration. We
record the base fts_name to avoid having to worry about whether we
needed the root symlink name or not (as applicable), then we can simply
shift all of that logic to after path translation to make it less
fragile.
If we're copying to DNE, then we'll have swapped out the NULL root_stat
pointer and then attempted to recurse on it. The previously nonexistent
directory shouldn't exist at all in the new structure, so just back out
from that tree entirely and move on.
The tests have been amended to indicate our expectations better with
subdirectory recursion. If we copy A to A/B, then we expect to copy
everything from A/B/* into A/B/A/B, with exception to the A that we
create in A/B.
Reviewed by: bapt
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D34655
Provide libedit a special function making it always add a space after
the autocompleted command. The default one adds a slash if the word is
also a name of a directory in the current working directory, but this is
wrong for commands.
Reviewed by: bapt, jilles
Differential Revision: https://reviews.freebsd.org/D34544
Before:
devfs 2 2 0 100% 0 0 100% /dev
After:
devfs 2 2 0 100% 0 0 - /dev
The previous behaviour was confusing for end users and many monitoring tools
Note the linux df tools is also using the same syntax '-' for such filesystem
MFC After: 2 weeks
Reviewed by: manu, emaste, imp
Differential Revision: https://reviews.freebsd.org/D34515
According to POSIX, cp should allow the `-P` flag to work whether `-R`
is specified or not. Currently, the `-P` option only works along with
`-R`.
PR: 199466
Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D30012
The traversal was previously not properly honoring -H/-L/-P. Notably,
we should not have been resolving symlinks encountered during traversal
when either -H or -P are specified.
Sponsored by: Klara, Inc.
Reviewed by: bapt
Differential Revision: https://reviews.freebsd.org/D34063
The df command provides a -l option to exclude all non-local file
systems and a -t option with a (positive or negative) list of file
system types to display.
This commit adds support for a combination of -l and -t. If both are
specified, the parameter list of the -t option is applied on top of
the selection of öocal file systems (independently of the order of
the -l and -t options).
E.g., "df -t noprocfs,sysfs -l" will select all local file systems
except those of type procfs and sysfs.
PR: 260921
Approved by: imp
Relnotes: yes
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D33748
As noted in the PR, cp -R has some surprising behavior. Typically, when
you `cp -R foo bar` where both foo and bar exist, foo is cleanly copied
to foo/bar. When you `cp -R foo foo` (where foo clearly exists), cp(1)
goes a little off the rails as it creates foo/foo, then discovers that
and creates foo/foo/foo, so on and so forth, until it eventually fails.
POSIX doesn't seem to disallow this behavior, but it isn't very useful.
GNU cp(1) will detect the recursion and squash it, but emit a message in
the process that it has done so.
This change seemingly follows the GNU behavior, but it currently doesn't
warn about the situation -- the author feels that the final product is
about what one might expect from doing this and thus, doesn't need a
warning. The author doesn't feel strongly about this.
PR: 235438
Reviewed by: bapt
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D33944
Pointer arithmetic on a null pointer is undefined behavior.
The bug can be reproduced by running bin/sh/tests/builtins/wait6.0 with
UBSAN.
Reported by: Mark Millard
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D34011
Now that root uses /bin/sh as the default shell no need to have csh part
of runtime.
Put it in its own package.
MFC after: never
Sponsored by: Beckhoff Automation GmbH & Co. KG
Differential Revision: https://reviews.freebsd.org/D33471
YP is less and less used, split them to users have the choice to not
install them.
MFC after: 2 weeks
Sponsored by: Beckhoff Automation GmbH & Co. KG
Differential Revision: https://reviews.freebsd.org/D33441
This makes it consistent with other date(1) implementations. Also, it
feels more consistent since hours and minutes are already represented as
HH and MM respectively.
MFC after: 3 days
- Use Cm instead of Ar or Sq for command modifiers of the -v flag.
- Remove unnecessary "Ar ..." from the synopsis. It's not clear what it
was referring to.
- Add missing arguments to the -f and -v flags.
- Stylize the dot before "ss" with Cm in the default format in the -f
flag description.
- Set LC_ALL=C in the last example so that the output format of
date(1) always matches the specified format of the -f flag not matter
the locale.
- List the -f flag as optional in all usage lines in the synopsis.
MFC after: 3 days
After an unescaped newline, there may be a here-document. Some places in
case and for did not check for one.
Reviewed by: bdrewery
Differential Revision: https://reviews.freebsd.org/D32628
In single-user mode, all env vars are absent, so exptilde() would not be
able to expand ~ correctly.
Place the lines setting PATH below HOME, so exptilde() would work as
expected.
Sponsored by: The FreeBSD Foundation
MFC after: 3 days
Reviewed by: jilles, emaste
Differential Revision: https://reviews.freebsd.org/D27003
For $reason mobaxterm default on sending unusual sequence from home/del
key, which makes libedit unabel to catch them and bind them correctly.
mobaxterm seems popular on the windows environment, so add proper
keybinding to default shrc configuration so it works out of box.
Reported by: lme
In the default configuration add 2 bindings which has been requested by
many during the HEADSUP discussion:
* csh like arrow history navigation
* ctrl-arrow to jump from word to words
Add an alias to make the history command exist as an alias to fc -l.
Make freebsd-version(1) support jails by adding the -j flag which takes
a jail jid or name as an argument. As with other options, -j
flags stack and display in the order requested.
Reviewed by: bcr (manpages), kevans
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D25705
Add -o verify to sh to make it use O_VERIFY when
sourcing scripts and reading profiles.
Useful in conjunction with mac_veriexec to help protect at
least some parts of the boot sequence, e.g., /etc/rc*.
Differential revision: https://reviews.freebsd.org/D30464
Reviewed by: jilles, sjg
Obtained from: Stormshield
When there are many matches, find the longest common substring starting
from the beginning of each command and use that to replace input.
As an example: on my system, llv<tab> will be autocompleted to llvm-
and another <tab> will print all matching llvm commands.
Until this change, any bindings set in histedit() were lost on calls to
bindcmd().
Only bind -e and bind -v call libedit's keymacro_reset(). Currently you
cannot fool libedit/map.c:map_bind() by trying something like bind -le
as when p[0] == '-', it does a switch statement on p[1].
Allows capitalizing the background color character to enable an
underline instead of bold, capitalizing the foreground color char will
still do bold.
Differential Revision: https://reviews.freebsd.org/D30547
When multiple matches are found, we keep the provided string on the
input line and print unique matches as suggestions.
But the multiple matches might be the same command found in different
directories, so we should deduplicate the matches first and then decide
whether to autocomplete the command or not, based on the number of
unique matches.
in emacs mode ^W should delete the previous word by default
Note that upstreaming this change directly into libedit is in process.
Reported by: manu
Reviewed by: jills, pstef, manu
Differential Revision: https://reviews.freebsd.org/D29493
As 257886 reports, if ls(1) is built with WITHOUT_LS_COLORS="YES", it
issues a warning whenever COLORTERM is non-empty. The warning is not
useful, so I thought to remove it, but as Ed pointed out, we may want
to have a way to determine whether a particular copy of ls has been
compiled with color support or not.
Therefore move the warnx() call to the getopt loop in
a WITHOUT_LS_COLORS build to fire when the user asks for colored output.
PR: 257886
Reported by: Marko Turk
Reviewed by: kevans
Move code added in b315a7296d ("autocomplete commands") to
conditionally compiled part under #ifndef NO_HISTORY.
Reported by: bdrewery
Fixes: b315a7296d
This prevents ps from being built with address sanitizer instrumentation.
Reviewed By: trasz
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D31048
The early environment is typically cleared, so these new options
need the PRESERVE_EARLY_KENV kernel config(8) option. These environments
are reported as missing by kenv(1) if the option is not present in the
running kernel.
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D30835
The contents of the kenv will be dumped to stdout, while dump could have
also meant 'discard'. Call it 'list' instead.
Suggested by: imp
Reviewed by: imp, jhb
Differential Revision: https://reviews.freebsd.org/D30836
Implement persistent history storage:
the strategy is simple at start: loads the existing .sh_history file
at exit dump it.
The implementation respects the HISTFILE variable and its POSIX
definition: ~/.sh_history is used if HISTFILE is not set.
to avoid sh to create the history file, set HISTSIZE to 0 or HISTFILE to
en empty value
Co-authored-by: pstef
Reviewed by: jilles
Differential Revision: https://reviews.freebsd.org/D29493
In emacs mode, force ^R to backware search the history
This behaviour is the default in emacs mode for most of the other shells
Note: Note that this can still be overridden via $EDITRC, ~/.editrc or a
bind command after set -o emacs.
MFC after: 1 week
Approved by: jilles
Reviewed by: jilles, arichardson, pstef
Differential Revision: https://reviews.freebsd.org/D29494
Without this patch, sh can autocomplete file names but not commands from
$PATH. Use libedit's facility to execute custom function for autocomplete,
but yield to the library's standard autocomplete function when cursor is
not at position 0.
Reviewed by: bapt
Differential Revision: https://reviews.freebsd.org/D29361
The POSIX sh case statement does not allow for pattern matching using the
regex + qualifier so this case statement never matches. Instead just check
for a string starting with a digit followed by any character.
While touching these files also fix various shellcheck warnings.
`kyua -v parallelism=4 test` failed before, succeeds now.
Reviewed By: imp
Differential Revision: https://reviews.freebsd.org/D28480
The following situation can trigger the deadlock:
1) Long time ago a_service was started through rc.d
2) We want to restart a_service and issue service a_service restart
3) rc.subr reads current process PID (via file or process),
sends TERM signal and runs pwait with PID harvested
4) a_service process dies very quickly so it's PID becomes available.
It is possible that while original process was running,
PID counter overflowed and pwait got assigned a_service's PID.
This patch ignores pid(s) to wait that are equal to pwait PID.
Reported by: Dan McGregor, Boris Lytochkin
Submitted by: Boris Lytochkin <lytboris at gmail.com>
Reviewed By: 0mp
MFC after: 2 weeks
PR: 218598
Differential Revision: https://reviews.freebsd.org/D28240
We try to ensure building WITHOUT_CLEAN (or -DNO_CLEAN) works on an
ongoing basis. b7ab6832cd changed cat to build w/o -DWITH_CASPER
by default; add a cat.o dependency on the Makefile so that it gets
rebuilt.
There is no guarantee that after close(2)/free the errno will remain
persistent. The caller of the udom_open function depends on the errno
for reporting errors.
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D28185
With this change and D27598 make kernel-toolchain no longer emits any
warnings for me.
Reviewed By: emaste
Differential Revision: https://reviews.freebsd.org/D27599
PR252358 reported a serious performance problem when
copying a large non-sparse file on a UFS file system.
This problem seems to have been caused by a large
number of SEEK_HOLE operations, with one done
for each copy_file_range(2) call.
This patch modifies cp(1) to use a large (SSIZE_MAX)
len argument, reducing the number of system calls
and resolving the performance issue.
While here, convert the type of the "rcount" from "int"
to "ssize_t" so that it is consistent with that returned
by both read(2) and copy_file_range(2).
PR: 252358
Reviewed by: asomers
Differential Revision: https://reviews.freebsd.org/D27937
Support for fractional seconds has become much more widespread since
this text was originally written.
Reported by: Mark Eichin
Reviewed by: gbe, jilles
Differential Revision: https://reviews.freebsd.org/D26208
This change has been motivated by a mail from bde sent in 2015 in
which he mentioned inappropriate use of sscanf() in 3 programs in
/bin.
This change removes the potential mismatch of the types of the return
values and the variable width specified in the scan pattern. While
there was no issue with the patterns and types used, the new code is
simpler and more efficient.
The previous version normalized the width list (replaced empty fields
with "0") just to be able to use sscanf() on the string.
It is much simpler to just parse the string as-is.
The clearing of f_notabs is preserved for the case that less than 9
width values have been defined, but I do not understand the rationale
for this particular condition. E.g., LS_COLWIDTHS="::::::::" will be
counted as 9 defined fields (may clear f_notabs) but is no different
fron LS_COLWIDTHS="" with regard to the field width (and that does not
clear f_notabs, since there are less than 9 fields).
This is addressing cases such as fts_read(3) encountering an [EIO]
from fchdir(2) when FTS_NOCHDIR is not set. That would otherwise be
seen as a successful traversal in some of these cases while silently
discarding expected work.
As noted in r264201, fts_read() does not set errno to 0 on a successful
EOF so it needs to be set before calling it. Otherwise we might see
a random error from one of the iterations.
gzip is ignoring most errors and could be improved separately.
Reviewed by: vangyzen
Sponsored by: Dell EMC
Differential Revision: https://reviews.freebsd.org/D27184
The acl_from_stat function accepts a stat_t * argument, but only uses its
st_mode field. There is no reason to pass the whole struct, so make it accept
a mode_t and rename the function to acl_from_mode.
Linux has non-standard acl_from_mode function in its libacl, so naming the
function this way may help discovering it during porting efforts.
Reviewed by: tsoome, markj
Approved by: markj
Differential Revision: https://reviews.freebsd.org/D27292
This was marked deprecated in r329092, over two and a half years ago.
This functionality is also buggy per PR 237368.
PR: 237368
Reviewed by: brooks, cem, emaste, imp
Differential Revision: https://reviews.freebsd.org/D27197
Small EXAMPLES section.
Add reference to realpath(1) due to similarity.
Approved by: manpages (gbe@)
Differential Revision: https://reviews.freebsd.org/D26862
Add a dozen of examples to the EXAMPLES section for pgrep(1) and pkill(1).
Flags covered: -f, -F, -n, -j, -l, -S, -x
Approved by: mandoc (bcr@)
Differential Revision: pkill(1): Add EXAMPLES section to man page
* Add EXAMPLES section with four simple examples.
* Simplify -H flag description. This makes easy to see the difference between
this flag and -h
* While here, fix .Tn deprecated macro.
Approved by: manpages (bcr@)
Differential Revision: https://reviews.freebsd.org/D26662
* Add small EXAMPLES section to the man page showing the different
flags and exit codes.
* Complete description for -v flag.
Approved by: manpages (bcr@)
There are some tests available in the NetBSD test suite, but we don't
currently pass all of those; further investigation will go into that. For
now, just add a basic test as well as a test that copies from /dev/null to a
file.
The /dev/null test confirms that the file gets created if it's empty, then
that it truncates the file if it's non-empty. This matches some usage that
was previously employed in the build and was replaced in r366042 by a
simpler shell construct.
I will also plan on coming back to expand these in due time.
MFC after: 1 week
The current default is provided in various Makefile.inc in some top-level
directories and covers a good portion of the tree, but doesn't cover parts
of the build a little deeper (e.g. libcasper).
Provide a default in src.sys.mk and set WARNS to it in bsd.sys.mk if that
variable is defined. This lets us relatively cleanly provide a default WARNS
no matter where you're building in the src tree without breaking things
outside of the tree.
Crunchgen has been updated as a bootstrap tool to work on this change
because it needs r365605 at a minimum to succeed. The cleanup necessary to
successfully walk over this change on WITHOUT_CLEAN builds has been added.
There is a supplemental project to this to list all of the warnings that are
encountered when the environment has WARNS=6 NO_WERROR=yes:
https://warns.kevans.dev -- this project will hopefully eventually go away
in favor of CI doing a much better job than it.
Reviewed by: emaste, brooks, ngie (all earlier version)
Reviewed by: emaste, arichardson (depend-cleanup.sh change)
Differential Revision: https://reviews.freebsd.org/D26455
Even though copy_file_range has a file-system agnostic version, it still
fails on devfs (perhaps because the file descriptor is non-seekable?) In
that case, fallback to old-fashioned read/write. Fixes
"cp /dev/null /tmp/null"
PR: 249248
Reported by: Michael Butler
Reviewed by: mjg
MFC-With: 365549
Differential Revision: https://reviews.freebsd.org/D26395
This has three advantages over write(2)/read(2):
* Fewer context switches and data copies
* Mostly preserves a file's sparseness
* On some file systems (currently NFS 4.2) the file system will perform the
copy in an especially efficient way.
Reviewed by: rmacklem
MFC after: 2 weeks
Sponsored by: Axcient
Differential Revision: https://reviews.freebsd.org/D26377
POSIX is pretty clear that command -v, command -V and type shall write
absolute pathnames. Therefore, we need to prepend the current directory's
name to relative pathnames.
This can happen either when PATH contains a relative pathname or when the
operand contains a slash but is not an absolute pathname.
If job control is not enabled, a background job (... &) ignores SIGINT and
SIGQUIT, but this can be reverted using the trap builtin in the same shell
environment.
Using the set builtin to change options would also revert SIGINT and SIGQUIT
to their previous dispositions.
This broke due to r317298. Calling setsignal() reverts the effect of
ignoresig().
Reported by: bdrewery
MFC after: 1 week
When -H is specified, for kernel threads the command is formatted as
"<proc name>/<td name>" and truncated to MAXCOMLEN. But each of the
proc name and td name may be up to MAXCOMLEN bytes in length.
Also handle the ki_moretdname field to ensure that the full thread name
gets printed. This is already handled correctly when formatting for
"-o tdname".
Reported by: freqlabs
Reviewed by: freqlabs
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D25840
I've always found this a little bit confusing:
> sh
$ ^D> sh
$ ^D>
Reviewed by: 0mp, jilles
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D25813
realtime priorities
The current `ps -axO rtprio' show threads running at interrupt
priority such as the [intr] thread as '1:48' and threads running
at kernel priority such as [pagedaemon] as normal:4294967260.
This change shows [intr] as intr:48 and [pagedaemon] as kernel:4.
Reviewed by: kib
MFC after: 1 week (together with -r362369)
Differential Revision: https://reviews.freebsd.org/D25660
INTOFF postpones SIGINT processing and INTON enables it again. This is
important so an interactive shell can return to the top level prompt when
Ctrl+C is pressed.
Given that INTON is automatically done when a builtin completes, the part
where onsig() ignores suppressint when in_dotrap is true is both unnecessary
and unsafe. If the trap is for some other signal than SIGINT, arbitrary code
could have been interrupted.
Historically, INTOFF remained in effect for longer.
Reviewed by: bdrewery
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D25270
The manual page documents "\$" to expand to either "$" or "#" followed by
a single space. In reality, the single space character is not appended.
PR: 247791
Submitted by: kd-dev@pm.me
MFC after: 7 days
When job control is not enabled, the shell ignores SIGINT while waiting for
a foreground process unless that process exits on SIGINT. In this case, the
foreground process is sleep and it does not exit on SIGINT because the
signal is only sent to the shell. Depending on order of events, this could
cause the SIGINT to be unexpectedly ignored.
On lightly loaded bare metal, the chance of this happening tends to be less
than 0.01% but with higher loads and/or virtualization it becomes more
likely.
Starting the sleep in background and using the wait builtin ensures SIGINT
will not be ignored.
PR: 247559
Reported by: lwhsu
MFC after: 1 week
This flag will now show the processor number on which a process is running.
This change was inspired by PR129965. Initially I didn't think that the
patch attached to it was correct -- it sacrificed ki_estcpu use in "cpu"
for ki_lastcpu and I thought that the old functionality should be kept and
the new (cpu#) one added to it. But I've since discovered that ki_estcpu is
sched_4bsd-specific. What's worse, it represents the same thing as
ki_pctcpu, except ki_pctcpu is universal -- so "%cpu" has been using it
successfully. Therefore, I've decided to replace information based on
ki_estcpu with information based on ki_oncpu/ki_lastcpu.
Key parts of the code and manual changes were borrowed from top(1).
PR: 129965
Reported by: Nikola Knežević
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D25377
Fix the rtprio option that for some reason was progessively becoming an
option showing the priority class of threads. In particular:
- use the constants defined in sys/sys/rtprio.h instead of those defined in
sys/sys/priority.h: this helps making clearer that the code actually is
about realtime priorities and not standard scheduler priorities;
- remove the PRI_ITHD case that has nothing to do with realtime priorities;
- convert the priority levels to realtime priority levels using the same
formulas used for pri_to_rtp function in sys/kern/kern_resource.c.
- remove outdated note "101 = not a realtime process" in the man page and
replace it with a more useful reference to man 1 rtprio.
Approved by: src (mckusick), manpages (bcr), gerald (mentor)
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D25266
If job control is not enabled, background commands shall ignore SIGINT and
SIGQUIT, and it shall be possible to override that ignore in the same shell.
MFC after: 1 week
Use %hs (locale-based encoding) instead of %s (UTF-8) format for
strings that are expected to be in current locale encoding (date/time,
process names/argument list).
PR: 241491
Reviewed by: phil
Differential Revision: https://reviews.freebsd.org/D22160
FreeBSD DD utility has not had support for the O_DIRECT flag, which
is useful to bypass local caching, e.g. for unconditionally issuing
NFS IO requests during testing.
Reviewed by: rgrimes (mentor)
Approved by: rgrimes (mentor, blanket)
MFC after: 3 weeks
Sponsored by: NetApp, Inc.
Differential Revision: https://reviews.freebsd.org/D25066
Historically, we've allowed read() of a directory and some filesystems will
accommodate (e.g. ufs/ffs, msdosfs). From the history department staffed by
Warner: <<EOF
pdp-7 unix seemed to allow reading directories, but they were weird, special
things there so I'm unsure (my pdp-7 assembler sucks).
1st Edition's sources are lost, mostly. The kernel allows it. The
reconstructed sources from 2nd or 3rd edition read it though.
V6 to V7 changed the filesystem format, and should have been a warning, but
reading directories weren't materially changed.
4.1b BSD introduced readdir because of UFS. UFS broke all directory reading
programs in 1983. ls, du, find, etc all had to be rewritten. readdir() and
friends were introduced here.
SysVr3 picked up readdir() in 1987 for the AT&T fork of Unix. SysVr4 updated
all the directory reading programs in 1988 because different filesystem
types were introduced.
In the 90s, these interfaces became completely ubiquitous as PDP-11s running
V7 faded from view and all the folks that initially started on V7 upgraded
to SysV. Linux never supported this (though I've not done the software
archeology to check) because it has always had a pathological diversity of
filesystems.
EOF
Disallowing read(2) on a directory has the side-effect of masking
application bugs from relying on other implementation's behavior
(e.g. Linux) of rejecting these with EISDIR across the board, but allowing
it has been a vector for at least one stack disclosure bug in the past[0].
By POSIX, this is implementation-defined whether read() handles directories
or not. Popular implementations have chosen to reject them, and this seems
sensible: the data you're reading from a directory is not structured in some
unified way across filesystem implementations like with readdir(2), so it is
impossible for applications to portably rely on this.
With this patch, we will reject most read(2) of a dirfd with EISDIR. Users
that know what they're doing can conscientiously set
bsd.security.allow_read_dir=1 to allow read(2) of directories, as it has
proven useful for debugging or recovery. A future commit will further limit
the sysctl to allow only the system root to read(2) directories, to make it
at least relatively safe to leave on for longer periods of time.
While we're adding logic pertaining to directory vnodes to vn_io_fault, an
additional assertion has also been added to ensure that we're not reaching
vn_io_fault with any write request on a directory vnode. Such request would
be a logical error in the kernel, and must be debugged rather than allowing
it to potentially silently error out.
Commented out shell aliases have been placed in root's chsrc/shrc to promote
awareness that grep may become noisy after this change, depending on your
usage.
A tentative MFC plan has been put together to try and make it as trivial as
possible to identify issues and collect reports; note that this will be
strongly re-evaluated. Tentatively, I will MFC this knob with the default as
it is in HEAD to improve our odds of actually getting reports. The future
priv(9) to further restrict the sysctl WILL NOT BE MERGED BACK, so the knob
will be a faithful reversion on stable/12. We will go into the merge
acknowledging that the sysctl default may be flipped back to restore
historical behavior at *any* point if it's warranted.
[0] https://www.freebsd.org/security/advisories/FreeBSD-SA-19:10.ufs.asc
PR: 246412
Reviewed by: mckusick, kib, emaste, jilles, cy, phk, imp (all previous)
Reviewed by: rgrimes (latest version)
MFC after: 1 month (note the MFC plan mentioned above)
Relnotes: absolutely, but will amend previous RELNOTES entry
Differential Revision: https://reviews.freebsd.org/D24596
Austin Group bugs #1226 and #1250 changed the requirements for shell scripts
without #! (POSIX does not specify #!; this is about the shell execution
when execve(2) returns an [ENOEXEC] error).
POSIX says we shall allow execution if the initial part intended to be
parsed by the shell consists of characters and does not contain the NUL
character. This allows concatenating a shell script (ending with exec or
exit) and a binary payload.
In order to reject common binary files such as PNG images, check that there
is a lowercase letter or expansion before the last newline before the NUL
character, in addition to the check for the newline character suggested by
POSIX.
Highlights:
- CLICOLOR in the environment should imply --color=auto to maintain
compatibility with historical behavior
- -G should set CLICOLOR and imply --color=auto
The manpage has been updated to draw the connection between -G and --color;
the former is in-fact a sort of compromise between --color=always and
--color=auto, where we'll output color regardless of the environment lacking
CLICOLOR/COLORTERM assuming stdout is a tty.
X-MFC-With: r361318
The regression is in-fact that I flipped the default from never to auto. The
incorrect impression was based on an alias that I failed to notice,
installed by the Linux distribution that I used for testing compatibility
here. Users that want the old default should be doing so with a shell alias
as is done elsewhere, rather than making this decision in ls(1).
Many thanks to rgrimes for pointing out the alias that I clearly overlooked
that resulted in this; if you despised colors in your terminal from this,
consider buying him a beer at the next venue that you see him at.
MFC after: 1 week
Relnotes: yes
The shell maintains a count of the number of times SIGINT processing has
been disabled via INTOFF, so SIGINT processing resumes when all disables
have enabled again (INTON).
If an error occurs in a vfork() child, the processing of the error enables
SIGINT processing again, and the INTON in vforkexecshell() causes the count
to become negative.
As a result, a later INTOFF may not actually disable SIGINT processing. This
might cause memory corruption if a SIGINT arrives at an inopportune time. As
of r360452, it causes the shell to abort when it would unsafely allocate or
free memory in certain ways.
Note that various places such as errors in non-special builtins
unconditionally reset the count to 0, so the problem might still not always
be visible.
PR: 246497
Reported by: jbeich
MFC after: 2 weeks
Initially it seemed that there were multiple possible ways to do it.
Processing option -p could conditionally add selected processes and
their descendants to the list for further work, but it is not guaranteed
to know whether the -d option has been used or not, and it also doesn't
have access to the process list just yet.
There is also descendant_sort() which has access to all possibly needed
information, but serves the purely post-processing purpose of sorting
output.
Then there is the loop that uses invocation information and full process
list to create a list of processes for final display. It seems the most
natural place to implement this, but indeterminate state of the process
list and volatility of the final list that is being created obstruct
adding an elegant search for all elements of process descendancy trees.
So I opted for adding another loop, just before the one I mentioned
above. For all selected processes it conditionally adds direct
descendants to the end of this list of selected processes.
Possible usage:
* ps -auxd -p $$
* ps -auxd -p 1
* while x=$(pgrep svnlite); do clear; ps auxd -p $x; sleep 2; done
* ps -auxd -p `pgrep make`
Reviewed by: kevans, kaktus (earlier version)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D24380
As I noted in https://reviews.freebsd.org/D22756, INTOFF should be in effect
when calling ckmalloc/ckrealloc/ckfree to avoid memory leaks and double
frees. Therefore, change the functions to check if INTOFF is in effect
instead of applying it.
Reviewed by: bdrewery
Differential Revision: https://reviews.freebsd.org/D24599
r360139 made compiling with NO_HISTORY work. This #define does not remove
the fc and bind builtins completely but makes them always write an error
message.
However, there was also some code in builtins.def and mkbuiltins to remove
the fc builtin entirely (but not the bind builtin). The additional build
system complication to make this work seems not worth it, so remove that
code.
mips-gcc for mips32 was complaining that c was potentially used before
being set. Setting it to 0 before calling fdgetsc() looks like the right
thing to do in this instance; there's an explicit check for c == 0 later
on.
Tested: mips-gcc mips32 build, running /bin/sh on mips32
el is declared extern in myhistedit.h and defined in histedit.c. Remove the
duplicate definition in input.c to appease the -fno-common build.
-fno-common will become the default in GCC10/LLVM11.
MFC after: 3 days
Specifically, any system with a 32-bit size_t; -residue is calculated as a
32-bit *then* promoted to the 64-bit off_t and the result is ultimately
wrong. This resulted in what would appear to be truncated output, as only
the first line would be read.
Correct it by just making residue an off_t to begin with, since this is what
lseek will take anyways.
Reported by: antoine, dim
Triaged by: cem
Tested by: kevans
X-MFC-With: r358152
fd.
The read built-in command calls read(2) with a 1-byte buffer because
newline characters need to be detected even on a byte stream which
comes from a non-seekable file descriptor. Because of this, the
following script calls >6,000 read(2) to show a 6KiB file:
while read IN; do echo "$IN"; done < /COPYRIGHT
When the input byte stream is seekable, it is possible to read a data
block and then reposition the file pointer to where a newline
character found. This change adds a small buffer to do this and
reduces the number of read(2) calls.
Theoretically, multiple built-in commands reading the same seekable
byte stream in a single pipe chain can share the buffer. However,
this change just makes a single invocation of the read built-in
allocate a buffer and deallocate it every time for simplicity.
Although this causes read(2) to read the same regions multiple times,
the performance penalty should be small compared to the reduction of
read(2) calls.
Reviewed by: jilles
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D23747
`cat -l` is needed during the installworld phase and other system's cat
don't support that flag. To avoid portability issues when compiling on
Linux/macOS (such as the the direct access to &fp->_mbstate), we
disable the entire multibyte support when building as a boostrap tool.
Reviewed By: brooks, emaste
Differential Revision: https://reviews.freebsd.org/D13939
If getcwd() failed earlier on but later succeeded in the pwd builtin,
there was no INTOFF protection between calling savestr() and storing its
result.
It is quite rare for getcwd() to fail, and rarer for it to succeed later in
the same directory.
Found via code inspection for changing ckmalloc() and similar to assert
INTOFF protection instead of applying it directly (which protects against
corrupting malloc's internal state but allows memory leaks or double frees).
MFC after: 1 week
If executing a file fails with an [ENOEXEC] error, the shell executes the
file as a shell script, except that this execution may instead result in an
error message if the file is binary.
Per a recent Austin Group interpretation, we will need to change this to
allow a concatenation of a shell script and a binary payload. See
Austin Group bugs #1226 and #1250.
MFC after: 1 week
Update a bunch of Makefile.depend files as
a result of adding Makefile.depend.options files
Reviewed by: bdrewery
MFC after: 1 week
Sponsored by: Juniper Networks
Differential Revision: https://reviews.freebsd.org/D22494
Leaf directories that have dependencies impacted
by options need a Makefile.depend.options file
to avoid churn in Makefile.depend
DIRDEPS for cases such as OPENSSL, TCP_WRAPPERS etc
can be set in local.dirdeps-options.mk
which can add to those set in Makefile.depend.options
See share/mk/dirdeps-options.mk
Reviewed by: bdrewery
MFC after: 1 week
Sponsored by: Juniper Networks
Differential Revision: https://reviews.freebsd.org/D22469
Normally, count=n means read(2) will be called n times on the input to dd. If
the read() returns short, as may happen when reading from a pipe, fewer bytes
will be copied from the input. With conv=sync the buffer is padded with zeros
to fill the rest of the block.
iflag=fullblock causes dd to continue reading until the block is full, so that
count=n means n full blocks are copied. This flag is compatible with illumos
and GNU dd and is used in the ZFS test suite.
Submitted by: Ryan Moeller
Reviewed by: manpages, mmacy@
MFC after: 1 week
Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D21441
Sets the O_FSYNC flag on the output file. oflag=fsync and oflag=sync are
synonyms just as O_FSYNC and O_SYNC are synonyms. This functionality is
intended to improve portability of dd commands in the ZFS test suite.
Submitted by: Ryan Moeller
Reviewed by: manpages, mmacy@
MFC after: 1 week
Sponsored by: iXsytems, Inc.
Differential Revision: https://reviews.freebsd.org/D21422
close(2) can return errors from previous operations which should not be ignored.
PR: 229616
Submitted by: Thomas Hurst
Reported by: Thomas Hurst
Reviewed by: mmacy@
Obtained from: Ryan Moeller
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D21376
The fdatasync flag performs an fdatasync(2) on the output file before closing it.
This will be useful for the ZFS test suite.
Submitted by: Ryan Moeller
Reviewed by: manpages, mmacy@
MFC after: 1 week
Sponsored by: iXSystems, Inc.
Differential Revision: https://reviews.freebsd.org/D21373
The default package use to be FreeBSD-runtime but it should only contain
binaries and libs enough to boot to single user and repair the system, it
is also very handy to have a package that can be tranform to a small mfsroot.
So create a new package named FreeBSD-utilities and make it the default one.
Also move a few binaries and lib into this package when it make sense.
Reviewed by: bapt, gjb
Differential Revision: https://reviews.freebsd.org/D21506
The fsync flag performs an fsync(2) on the output file before closing it.
This will be useful for the ZFS test suite.
Submitted by: ryan@ixsystems.com
Reviewed by: jilles@, imp@
MFC after: 1 week
Sponsored by: iXsystems, Inc.
The RELEASE_CRUNCH check is redundant here. We don't need it for releases
anymore, and picobsd can control this more directly without making it a special
case.
asserted. Some development boards for example will reset on DTR,
and some radio interfaces will transmit on RTS.
This patch allows "stty -f /dev/ttyu9.init -rtsdtr" to prevent
RTS and DTR from being asserted on open(), allowing these devices
to be used without problems.
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D20031
newvers.sh has supported a variable setting only mode, use that in
preference to grep to future proof this script from changes there.
Differential Revision: https://reviews.freebsd.org/D19849
Since D19668 was done, new users of the -n flag have surfaced. Parse
and ignore it on the command line until they can be updated.
Suggested by: rgrimes (in D19668).
These were used to set dst flag and minutes west of UTC
respectively. These are obsolete and have been removed form the
kernel. These existed primarily to faithfully emulate early
Unix ABIs that have been removed from FreeBSD.
Reviewed by: jbh@, brooks@
Differential Revision: https://reviews.freebsd.org/D19550
