Until this change, any bindings set in histedit() were lost on calls to
bindcmd().
Only bind -e and bind -v call libedit's keymacro_reset(). Currently you
cannot fool libedit/map.c:map_bind() by trying something like bind -le
as when p[0] == '-', it does a switch statement on p[1].
When multiple matches are found, we keep the provided string on the
input line and print unique matches as suggestions.
But the multiple matches might be the same command found in different
directories, so we should deduplicate the matches first and then decide
whether to autocomplete the command or not, based on the number of
unique matches.
in emacs mode ^W should delete the previous word by default
Note that upstreaming this change directly into libedit is in process.
Reported by: manu
Reviewed by: jills, pstef, manu
Differential Revision: https://reviews.freebsd.org/D29493
Move code added in b315a7296d ("autocomplete commands") to
conditionally compiled part under #ifndef NO_HISTORY.
Reported by: bdrewery
Fixes: b315a7296d
Implement persistent history storage:
the strategy is simple at start: loads the existing .sh_history file
at exit dump it.
The implementation respects the HISTFILE variable and its POSIX
definition: ~/.sh_history is used if HISTFILE is not set.
to avoid sh to create the history file, set HISTSIZE to 0 or HISTFILE to
en empty value
Co-authored-by: pstef
Reviewed by: jilles
Differential Revision: https://reviews.freebsd.org/D29493
In emacs mode, force ^R to backware search the history
This behaviour is the default in emacs mode for most of the other shells
Note: Note that this can still be overridden via $EDITRC, ~/.editrc or a
bind command after set -o emacs.
MFC after: 1 week
Approved by: jilles
Reviewed by: jilles, arichardson, pstef
Differential Revision: https://reviews.freebsd.org/D29494
Without this patch, sh can autocomplete file names but not commands from
$PATH. Use libedit's facility to execute custom function for autocomplete,
but yield to the library's standard autocomplete function when cursor is
not at position 0.
Reviewed by: bapt
Differential Revision: https://reviews.freebsd.org/D29361
POSIX is pretty clear that command -v, command -V and type shall write
absolute pathnames. Therefore, we need to prepend the current directory's
name to relative pathnames.
This can happen either when PATH contains a relative pathname or when the
operand contains a slash but is not an absolute pathname.
If job control is not enabled, a background job (... &) ignores SIGINT and
SIGQUIT, but this can be reverted using the trap builtin in the same shell
environment.
Using the set builtin to change options would also revert SIGINT and SIGQUIT
to their previous dispositions.
This broke due to r317298. Calling setsignal() reverts the effect of
ignoresig().
Reported by: bdrewery
MFC after: 1 week
I've always found this a little bit confusing:
> sh
$ ^D> sh
$ ^D>
Reviewed by: 0mp, jilles
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D25813
INTOFF postpones SIGINT processing and INTON enables it again. This is
important so an interactive shell can return to the top level prompt when
Ctrl+C is pressed.
Given that INTON is automatically done when a builtin completes, the part
where onsig() ignores suppressint when in_dotrap is true is both unnecessary
and unsafe. If the trap is for some other signal than SIGINT, arbitrary code
could have been interrupted.
Historically, INTOFF remained in effect for longer.
Reviewed by: bdrewery
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D25270
The manual page documents "\$" to expand to either "$" or "#" followed by
a single space. In reality, the single space character is not appended.
PR: 247791
Submitted by: kd-dev@pm.me
MFC after: 7 days
When job control is not enabled, the shell ignores SIGINT while waiting for
a foreground process unless that process exits on SIGINT. In this case, the
foreground process is sleep and it does not exit on SIGINT because the
signal is only sent to the shell. Depending on order of events, this could
cause the SIGINT to be unexpectedly ignored.
On lightly loaded bare metal, the chance of this happening tends to be less
than 0.01% but with higher loads and/or virtualization it becomes more
likely.
Starting the sleep in background and using the wait builtin ensures SIGINT
will not be ignored.
PR: 247559
Reported by: lwhsu
MFC after: 1 week
If job control is not enabled, background commands shall ignore SIGINT and
SIGQUIT, and it shall be possible to override that ignore in the same shell.
MFC after: 1 week
Historically, we've allowed read() of a directory and some filesystems will
accommodate (e.g. ufs/ffs, msdosfs). From the history department staffed by
Warner: <<EOF
pdp-7 unix seemed to allow reading directories, but they were weird, special
things there so I'm unsure (my pdp-7 assembler sucks).
1st Edition's sources are lost, mostly. The kernel allows it. The
reconstructed sources from 2nd or 3rd edition read it though.
V6 to V7 changed the filesystem format, and should have been a warning, but
reading directories weren't materially changed.
4.1b BSD introduced readdir because of UFS. UFS broke all directory reading
programs in 1983. ls, du, find, etc all had to be rewritten. readdir() and
friends were introduced here.
SysVr3 picked up readdir() in 1987 for the AT&T fork of Unix. SysVr4 updated
all the directory reading programs in 1988 because different filesystem
types were introduced.
In the 90s, these interfaces became completely ubiquitous as PDP-11s running
V7 faded from view and all the folks that initially started on V7 upgraded
to SysV. Linux never supported this (though I've not done the software
archeology to check) because it has always had a pathological diversity of
filesystems.
EOF
Disallowing read(2) on a directory has the side-effect of masking
application bugs from relying on other implementation's behavior
(e.g. Linux) of rejecting these with EISDIR across the board, but allowing
it has been a vector for at least one stack disclosure bug in the past[0].
By POSIX, this is implementation-defined whether read() handles directories
or not. Popular implementations have chosen to reject them, and this seems
sensible: the data you're reading from a directory is not structured in some
unified way across filesystem implementations like with readdir(2), so it is
impossible for applications to portably rely on this.
With this patch, we will reject most read(2) of a dirfd with EISDIR. Users
that know what they're doing can conscientiously set
bsd.security.allow_read_dir=1 to allow read(2) of directories, as it has
proven useful for debugging or recovery. A future commit will further limit
the sysctl to allow only the system root to read(2) directories, to make it
at least relatively safe to leave on for longer periods of time.
While we're adding logic pertaining to directory vnodes to vn_io_fault, an
additional assertion has also been added to ensure that we're not reaching
vn_io_fault with any write request on a directory vnode. Such request would
be a logical error in the kernel, and must be debugged rather than allowing
it to potentially silently error out.
Commented out shell aliases have been placed in root's chsrc/shrc to promote
awareness that grep may become noisy after this change, depending on your
usage.
A tentative MFC plan has been put together to try and make it as trivial as
possible to identify issues and collect reports; note that this will be
strongly re-evaluated. Tentatively, I will MFC this knob with the default as
it is in HEAD to improve our odds of actually getting reports. The future
priv(9) to further restrict the sysctl WILL NOT BE MERGED BACK, so the knob
will be a faithful reversion on stable/12. We will go into the merge
acknowledging that the sysctl default may be flipped back to restore
historical behavior at *any* point if it's warranted.
[0] https://www.freebsd.org/security/advisories/FreeBSD-SA-19:10.ufs.asc
PR: 246412
Reviewed by: mckusick, kib, emaste, jilles, cy, phk, imp (all previous)
Reviewed by: rgrimes (latest version)
MFC after: 1 month (note the MFC plan mentioned above)
Relnotes: absolutely, but will amend previous RELNOTES entry
Differential Revision: https://reviews.freebsd.org/D24596
Austin Group bugs #1226 and #1250 changed the requirements for shell scripts
without #! (POSIX does not specify #!; this is about the shell execution
when execve(2) returns an [ENOEXEC] error).
POSIX says we shall allow execution if the initial part intended to be
parsed by the shell consists of characters and does not contain the NUL
character. This allows concatenating a shell script (ending with exec or
exit) and a binary payload.
In order to reject common binary files such as PNG images, check that there
is a lowercase letter or expansion before the last newline before the NUL
character, in addition to the check for the newline character suggested by
POSIX.
The shell maintains a count of the number of times SIGINT processing has
been disabled via INTOFF, so SIGINT processing resumes when all disables
have enabled again (INTON).
If an error occurs in a vfork() child, the processing of the error enables
SIGINT processing again, and the INTON in vforkexecshell() causes the count
to become negative.
As a result, a later INTOFF may not actually disable SIGINT processing. This
might cause memory corruption if a SIGINT arrives at an inopportune time. As
of r360452, it causes the shell to abort when it would unsafely allocate or
free memory in certain ways.
Note that various places such as errors in non-special builtins
unconditionally reset the count to 0, so the problem might still not always
be visible.
PR: 246497
Reported by: jbeich
MFC after: 2 weeks
As I noted in https://reviews.freebsd.org/D22756, INTOFF should be in effect
when calling ckmalloc/ckrealloc/ckfree to avoid memory leaks and double
frees. Therefore, change the functions to check if INTOFF is in effect
instead of applying it.
Reviewed by: bdrewery
Differential Revision: https://reviews.freebsd.org/D24599
r360139 made compiling with NO_HISTORY work. This #define does not remove
the fc and bind builtins completely but makes them always write an error
message.
However, there was also some code in builtins.def and mkbuiltins to remove
the fc builtin entirely (but not the bind builtin). The additional build
system complication to make this work seems not worth it, so remove that
code.
mips-gcc for mips32 was complaining that c was potentially used before
being set. Setting it to 0 before calling fdgetsc() looks like the right
thing to do in this instance; there's an explicit check for c == 0 later
on.
Tested: mips-gcc mips32 build, running /bin/sh on mips32
el is declared extern in myhistedit.h and defined in histedit.c. Remove the
duplicate definition in input.c to appease the -fno-common build.
-fno-common will become the default in GCC10/LLVM11.
MFC after: 3 days
Specifically, any system with a 32-bit size_t; -residue is calculated as a
32-bit *then* promoted to the 64-bit off_t and the result is ultimately
wrong. This resulted in what would appear to be truncated output, as only
the first line would be read.
Correct it by just making residue an off_t to begin with, since this is what
lseek will take anyways.
Reported by: antoine, dim
Triaged by: cem
Tested by: kevans
X-MFC-With: r358152
fd.
The read built-in command calls read(2) with a 1-byte buffer because
newline characters need to be detected even on a byte stream which
comes from a non-seekable file descriptor. Because of this, the
following script calls >6,000 read(2) to show a 6KiB file:
while read IN; do echo "$IN"; done < /COPYRIGHT
When the input byte stream is seekable, it is possible to read a data
block and then reposition the file pointer to where a newline
character found. This change adds a small buffer to do this and
reduces the number of read(2) calls.
Theoretically, multiple built-in commands reading the same seekable
byte stream in a single pipe chain can share the buffer. However,
this change just makes a single invocation of the read built-in
allocate a buffer and deallocate it every time for simplicity.
Although this causes read(2) to read the same regions multiple times,
the performance penalty should be small compared to the reduction of
read(2) calls.
Reviewed by: jilles
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D23747
If getcwd() failed earlier on but later succeeded in the pwd builtin,
there was no INTOFF protection between calling savestr() and storing its
result.
It is quite rare for getcwd() to fail, and rarer for it to succeed later in
the same directory.
Found via code inspection for changing ckmalloc() and similar to assert
INTOFF protection instead of applying it directly (which protects against
corrupting malloc's internal state but allows memory leaks or double frees).
MFC after: 1 week
If executing a file fails with an [ENOEXEC] error, the shell executes the
file as a shell script, except that this execution may instead result in an
error message if the file is binary.
Per a recent Austin Group interpretation, we will need to change this to
allow a concatenation of a shell script and a binary payload. See
Austin Group bugs #1226 and #1250.
MFC after: 1 week
The pipefail option allows checking the exit status of all commands in a
pipeline more easily, at a limited cost of complexity in sh itself. It works
similarly to the option in bash, ksh93 and mksh.
Like ksh93 and unlike bash and mksh, the state of the option is saved when a
pipeline is started. Therefore, even in the case of commands like
A | B &
a later change of the option does not change the exit status, the same way
(A | B) &
works.
Since SIGPIPE is not handled specially, more work in the script is required
for a proper exit status for pipelines containing commands such as head that
may terminate successfully without reading all input. This can be something
like
(
cmd1
r=$?
if [ "$r" -gt 128 ] && [ "$(kill -l "$r")" = PIPE ]; then
exit 0
else
exit "$r"
fi
) | head
PR: 224270
Relnotes: yes
SVN r342880 was designed to fix $((-9223372036854775808)) and things like
$((0x8000000000000000)) but also broke error detection for values of
variables without dollar sign ($((x))).
For compatibility, overflow in plain literals continues to be ignored and
the value is clamped to the boundary (except 9223372036854775808 which is
changed to -9223372036854775808).
Reviewed by: se (although he would like error checking to be removed)
MFC after: 2 weeks
X-MFC-with: r342880
Differential Revision: https://reviews.freebsd.org/D18926
The rest of this stuff is still to be discussed, but I think at this
point we have the agreement that the aliases should go.
MFC after: 2 weeks
Sponsored by: DARPA, AFRL
there's no need to even mention it in shell rc files. Not that it's wrong;
just pointless and somewhat misleading.
Reviewed by: jilles
MFC after: 2 weeks
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D18809
The libedit "fout" output must be sent to fd 2 since it contains prompts
that POSIX says must be sent to fd 2. However, the libedit "ferr" output
receives error messages such as from "bind" that make no sense to send to fd
1.
results between an expression that refers to a variable by name and the
same expression that includes the same variable by value.
Submitted by: se@
MFC after: 1 week
If word in ${param?word} is missing, the shell shall write a default error
message. So expanding ${param?} when param is not set should write an error
message like
sh: param: parameter not set
This was broken by r316417.
PR: 233585
The difference between EXERROR and EXEXEC was that EXEXEC passed along
exitstatus and EXERROR set exitstatus to 2 in the handling code.
By changing the places that raised EXERROR to set exitstatus to 2, the
handling of EXERROR and EXEXEC becomes the same.
The intention is to lower the value of the pointer, which according to ubsan
cannot be done by adding an unsigned quantity.
Reported by: kevans
Approved by: re (kib)
MFC after: 1 week
This fixes the build and I will redo these changes as part of a future review
that organizes them differently. The way I tried to do it here could be done
better. Sorry for the noise.
Approved by: will (mentor)
Differential Revision: https://reviews.freebsd.org/D16737
This moves the symlink creation to after where the files are installed.
This also inverts the shell change so that it only happens if MK_TCSH is on.
Approved by: will (mentor)
Differential Revision: https://reviews.freebsd.org/D16725
This simplifies pkgbase by migrating these to CONFS so they are properly
tagged as config files.
Approved by: will (mentor)
Differential Revision: https://reviews.freebsd.org/D16708
Using padvance() requires undoing its append of '/' and prevents adjusting
its '%' logic to allow most directories with '%' in PATH.
No functional change is intended.
These are not used to link the final tool anymore. At some point in the past
the suffix rules changed to not link these in. The original reason for this in
r19176 is unclear but seems to be related to mkdep. The .depend handling is
still broken here as it is for all build tool patterns like this.
Sponsored by: Dell EMC
POSIX requires accepting unquoted newlines in word in parameter expansions
like ${param+word}, ${param#word}, although the Bourne shell did not support
it, it is not commonly used and might make it harder to find a missing
closing brace.
It was also strange that something like
foo="${bar#
}"
was rejected.
Reported by: Martijn Dekker via Robert Elz
It won't work e.g. when crossbuilding from Ubuntu Linux as mktemp is in
/bin there.
Reviewed By: bdrewery
Approved By: jhb (mentor)
Differential Revision: https://reviews.freebsd.org/D13937
Mainly focus on files that use BSD 3-Clause license.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.
Non-tests/... changes:
- Add HAS_TESTS= to Makefiles with libraries and programs to enable iteration
and propagate the appropriate environment down to *.test.mk.
tests/... changes:
- Add appropriate support Makefile.inc's to set HAS_TESTS in a minimal manner,
since tests/... is a special subdirectory tree compared to the others.
MFC after: 2 months
MFC with: r322511
Reviewed by: arch (silence), testing (silence)
Differential Revision: D12014
directories to SUBDIR.${MK_TESTS} idiom
This is being done to pave the way for future work (and homogenity) in
^/projects/make-check-sandbox .
No functional change intended.
MFC after: 1 weeks
Apart from the fact that subtle syntactic changes make a poor compile-time
option, the NOHACK case has been obviously broken since it was added,
because it uses q uninitialized if (*p != '\0').
No functional change is intended.
If CDPATH is used non-trivially or the operand is "-", cd writes the
directory actually switched to. (We currently do this only in interactive
shells, but POSIX requires this in non-interactive shells as well.)
As mentioned in Austin group bug #1045, cd shall not return an error while
leaving the current directory changed. Therefore, ignore any write error.
Starting the fc -e editor can execute arbitrary script, and executing
arbitrary script with INTOFF in effect may cause unexpected results.
This change (together with other changes) serves mainly to allow asserting
that INTOFF is not in effect when starting the evaluation of a node.
Allocating and deallocating repeatedly the 1024-byte buffer for stdout from
builtins costs CPU time for little or no benefit.
A simple loop containing builtins that write to a file descriptor, such as
i=0; while [ "$i" -lt 1000000 ]; do printf .; i=$((i+1)); done >/dev/null
is over 10% faster in a simple benchmark on an amd64 virtual machine.
Similarly to how STPUTC was changed, change struct output to store the
pointer just past the end of the available space instead of the size of the
available space, so after writing a character it is only necessary to
increment a pointer and not to decrement a counter.
It does not make much sense to generate the '-' in a pattern bracket
expression using arithmetic expansion, but it does not make sense to forbid
it either.
Try to avoid reprocessing the string if it is unnecessary.
It does not make much sense to generate the '-' in a pattern bracket
expression using arithmetic expansion, but it does not make sense to forbid
it either.
This test already passes.
