This commit makes the Shell check for errors after a node is run(), and
prevents further execution by unwinding until the error is cleared.
Fixes#10649.
When parse_expression looks at '$((', there are two ways it can end up
in parse_expression again, three consumed characters later. All these
ways fail, so what happened was that the parser tried all possible
combinations, hence taking potentially an exponential amount of time.
1. parse_evaluate swallows the '$(', a new invocation of
parse_expression swallows the other '(', and through
parse_list_expression we're at another parse_expression.
2. parse_evaluate swallows the '$(', but returns a SyntaxError.
parse_expression used to not recognize the error, and treated it as a
regular AST node, calling into read_concat, then a new invocation of
parse_expression swallows the other '(', and through
parse_list_expression we're at another parse_expression.
Fixes#10561.
Found by OSS Fuzz, long-standing issue
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=28113
...while capturing its standard output.
As `$()` is an invalid construct, execute nodes are not supposed to
capture the output of no command being run; but it is possible to create
empty commands such as CastToCommand(Redirection(...)) or similar.
Make this a hard error instead of an unescapable select().
This was noticed in #10432, which should now error out like so:
```
Error: Cannot capture standard output when no command is being executed
0| $(<$file)
~~~~~^^^^^^^^^
1|
```
Our existing implementation did not check the element type of the other
pointer in the constructors and move assignment operators. This meant
that some operations that would require explicit casting on raw pointers
were done implicitly, such as:
- downcasting a base class to a derived class (e.g. `Kernel::Inode` =>
`Kernel::ProcFSDirectoryInode` in Kernel/ProcFS.cpp),
- casting to an unrelated type (e.g. `Promise<bool>` => `Promise<Empty>`
in LibIMAP/Client.cpp)
This, of course, allows gross violations of the type system, and makes
the need to type-check less obvious before downcasting. Luckily, while
adding the `static_ptr_cast`s, only two truly incorrect usages were
found; in the other instances, our casts just needed to be made
explicit.
And also try_create<T> => try_make_ref_counted<T>.
A global "create" was a bit much. The new name matches make<T> better,
which we've used for making single-owner objects since forever.
Only one place used this argument and it was to hold on to a strong ref
for the object. Since we already do that now, there's no need to keep
this argument around since this can be easily captured.
This commit contains no changes.
The new Statistics utility is now used when calling 'time -n' to get
some more information of the timings. For now only the standard
deviation is given in addition to the average.
This commit completely undos #9645 because everything that touched moved
into AK::Statistics.
This kinda sorta addresses the Shell side of #9655, however the fact
that `chdir` (and most other syscalls that take paths) are artifically
limited to a length of PATH_MAX remains.
You can now specify a number of iterations when timing a command.
The default value is 1 and behaves exactly as before.
If the iteration count is greater than 1, the command will be executed
that many times, and then you get a little timing report afterwards with
the average runtime per iteration, and also the average runtime
excluding the very first iteration. (Excluding the first iteration is
useful when it's slowed down by cold caches, etc.)
This is something I've been doing manually forever (running `time foo`
and then eyeballing the results to headmath an average) and this makes
that whole process so much nicer. :^)
This is primarily to be able to remove the GenericLexer include out of
Format.h as well. A subsequent commit will add AK::Result to
GenericLexer, which will cause naming conflicts with other structures
named Result. This can be avoided (for now) by preventing nearly every
file in the system from implicitly including GenericLexer.
Other changes in this commit are to add the GenericLexer include to
files where it is missing.
Heredocs have a different parse end condition than double-quoted
strings. parse_doublequoted_string_inner would assume that a string
would always end in a double quote, so let's generalize it to
parse_string_inner and have it take a StringEndCondition enum which
specifies how the string terminates.
That can happen because of anyone sending the process a SIGCONT.
Fixes an issue where continuing a process launched by the shell from
the System Monitor would cause the shell to spin on waitpid().
Since this is always set to true on the non-default constructor and
subsequently never modified, it is somewhat pointless. Furthermore,
there are arguably no invalid relative paths.
This will cause a problem when `NonnullRefPtr<T>::operator T*` will be
declared as RETURNS_NONNULL. Clang emits a warning for this pointless
null check, which breaks CI.
This commit converts naked `new`s to `AK::try_make` and `AK::try_create`
wherever possible. If the called constructor is private, this can not be
done, so we instead now use the standard-defined and compiler-agnostic
`new (nothrow)`.
This is so they can find their associated resources and it's
the same behavior as in Lagom.
This also required changing some tests so that they could
write their resources in a writable location.
Some of the code assumed that chars were always signed while that is
not the case on ARM hosts.
Also, some of the code tried to use EOF (-1) in a way similar to what
fgetc() does, however instead of storing the characters in an int
variable a char was used.
While this seemed to work it also meant that character 0xFF would be
incorrectly seen as an end-of-file.
Careful reading of fgetc() reveals that fgetc() stores character
data in an int where valid characters are in the range of 0-255 and
the EOF value is explicitly outside of that range (usually -1).
And use them to highlight javascript in HTML source.
This commit also changes how TextDocumentSpan::data is interpreted,
as it used to be an opaque pointer, but everyone stuffed an enum value
inside it, which made the values not unique to each highlighter;
that field is now a u64 serial id.
The syntax highlighters don't need to change their ways of stuffing
token types into that field, but a highlighter that calls another
nested highlighter needs to register the nested types for use with
token pairs.
This changes the Shell syntax highlighter to conform to the now-fixed
rendering of syntax highlighting spans in GUI::TextEditor.
This also adds some debug output to make debugging easier.
This replaces ctype.h with CharacterType.h everywhere I could find
issues with narrowing conversions. While using it will probably make
sense almost everywhere in the future, the most critical places should
have been addressed.
We had two functions for doing mostly the same thing. Combine both
of them into String::find() and use that everywhere.
Also add some tests to cover basic behavior.
Follow-on to #7337. Been seeing other CI test failures that point to
these temp directories, so let's just move all of them to /tmp. I'm sure
someone will write ext2fs stress tests later :^)
Example:
/usr/Tests/Shell/control-structure-as-command.sh
Core::Socket: Failed to connect() to /tmp/portal/inspectables: ...
+ rm -rf shell-test 2>/dev/null
+ mkdir shell-test
Error: The action has timed out.
Since applications using Core::EventLoop no longer need to create a
socket in /tmp/rpc/, and also don't need to listen for incoming
connections on this socket, we can remove a whole bunch of pledges!
This patch adds a new flag called history_dirty to Line::Editor that is
set when history is added to but written. Applications can leverage
this flag to write history only when it changes. This patch adds an
example usage of this functionality to Shell, which will now only save
the history when it is dirty.
Instead of the previous only-escape-with-backslashes, extend the
escaping to one of:
- No escape
- Escape with backslash
- Escape with "\xhh" if control character that isn't easily represented
as \X
- Escape with "\uhhhhhhhh" if unicode character that is too big to
represent as "\xhh".
Fixes#6986.
This commit makes the shell:
- highlight executables in the current directory as invalid, unless an
explicit `./' is given (so, `./foo` isn't red, but `foo` is)
- not suggest executables in the current directory unless explicitly
requested (by prepending `./`)
- not attempt to run an executable in the current directory that has
been invoked as a program name and failed execvp().
Note that `./foo` is still executed because it's not invoked as
a name, but rather as a path.
Fixes the other half of #6774.
This bit of code was kept unmodified since it was first implemented,
and I'm not entirely convinced that it ever actually worked :P
This commit updates the code to use "modern" classes and constructs,
and fixes an issue where the shebang would still contain the '#!'
when it was passed to execvp().
Fixes#6774.
Otherwise we would end up trying to parse the same heredoc entry, if it
contained a sequence terminated by a newline.
e.g. `<<-x\n$({` would attempt to read a heredoc entry after `x`, and
then after `{` while inside the first heredoc entry.
To make this work, we can simply empty the instance vector and keep the
state on the stack.
Issue found through oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=33852
Closes#4283.
Heredocs are implemented in a way that makes them feel more like a
string (and not a weird redirection, a la bash).
There are two tunables, whether the string is dedented (`<<-` vs `<<~`)
and whether it allows interpolation (quoted key vs not).
To the familiar people, this is how Ruby handles them, and I feel is the
most elegant heredoc syntax.
Unlike the oddjob that is bash, heredocs are treated exactly as normal
strings, and can be used _anywhere_ where a string can be used.
They are *required* to appear in the same order as used after a newline
is seen when parsing the sequence that the heredoc is used in.
For instance:
```sh
echo <<-doc1 <<-doc2 | blah blah
contents for doc1
doc1
contents for doc2
doc2
```
The typical nice errors are also implemented :^)
Some nodes (such as heredocs) cannot be validated immediately, so the
entire tree will need to be revalidated if we don't allow mutating
syntax errors.
We had some inconsistencies before:
- Sometimes "The", sometimes "the"
- Sometimes trailing ".", sometimes no trailing "."
I picked the most common one (lowecase "the", trailing ".") and applied
it to all copyright headers.
By using the exact same string everywhere we can ensure nothing gets
missed during a global search (and replace), and that these
inconsistencies are not spread any further (as copyright headers are
commonly copied to new files).
Previously this didn't work:
$ cd -- /usr
Invalid path '--'
This path fixes this issue and removes the unnecessary else
branch because we're already using realpath() later on to resolve
relative paths.
SPDX License Identifiers are a more compact / standardized
way of representing file license information.
See: https://spdx.dev/resources/use/#identifiers
This was done with the `ambr` search and replace tool.
ambr --no-parent-ignore --key-from-file --rep-from-file key.txt rep.txt *
We intentionally skimp out on checking isatty() before them to cut down
on syscalls, so we should also accept the errors and just let them be.
Closes#6471.
This flag warns on classes which have `virtual` functions but do not
have a `virtual` destructor.
This patch adds both the flag and missing destructors. The access level
of the destructors was determined by a two rules of thumb:
1. A destructor should have a similar or lower access level to that of a
constructor.
2. Having a `private` destructor implicitly deletes the default
constructor, which is probably undesirable for "interface" types
(classes with only virtual functions and no data).
In short, most of the added destructors are `protected`, unless the
compiler complained about access.
This makes commands like `foo 2>&1 | bar` behave as expected (which is
to pipe both stdout and stderr of `foo` to stdin of `bar`).
Previously, this would've piped stderr of `foo` into stdout, and the
stdout of `foo` into the stdin of `bar`.
This fixes `fg` and `bg` causing the shell to go into an infinite loop
of trying to `waitpid` until some current job changes state.
a.k.a. "Fix Shell backgrounding, yet again!" :P
Now a variable may have an optional slice (only _one_ slice), which can
also use negative indices to index from the end.
This works on both lists and strings.
The contents of the slice have the same semantics as brace expansions.
For example:
```sh
$ x=(1 2 3 4 5 6)
$ echo $x[1..3] # select indices 1, 2, 3
2 3 4
$ echo $x[3,4,1,0] # select indices 3, 4, 1, 0 (in that order)
4 5 2 1
$ x="Well Hello Friends!"
$ echo $x[5..9]
Hello
```
This commit adds a few basic variable substitution operations:
- length
Find the length of a string or a list
- length_across
Find the lengths of things inside a list
- remove_{suffix,prefix}
Remove a suffix or a prefix from all the passed values
- regex_replace
Replace all matches of a given regex with a given template
- split
Split the given string with the given delimiter (or to its
code points if the delimiter is empty)
- concat_lists
concatenates any given lists into one
Closes#4316 (the ancient version of this same feature)
Non-interactive shells (i.e. when running scripts) do not need this
functionality, so they are a boatload of wasted time.
This significantly reduces the script startup and shutdown times when
there are lots of executables in PATH or lots of entries in the history.
This is basically just for consistency, it's quite strange to see
multiple AK container types next to each other, some with and some
without the namespace prefix - we're 'using AK::Foo;' a lot and should
leverage that. :^)
(...and ASSERT_NOT_REACHED => VERIFY_NOT_REACHED)
Since all of these checks are done in release builds as well,
let's rename them to VERIFY to prevent confusion, as everyone is
used to assertions being compiled out in release.
We can introduce a new ASSERT macro that is specifically for debug
checks, but I'm doing this wholesale conversion first since we've
accumulated thousands of these already, and it's not immediately
obvious which ones are suitable for ASSERT.
Thanks to @trflynn89 for the neat implicit consteval ctor trick!
This allows us to basically slap `CheckedFormatString` on any
formatting function, and have its format argument checked at compiletime.
Note that there is a validator bug where it doesn't parse inner replaced
fields like `{:~>{}}` correctly (what should be 'left align with next
argument as size' is parsed as `{:~>{` following a literal closing
brace), so the compiletime checks are disabled on these temporarily by
forcing them to be StringViews.
This commit also removes the now unused `AK::StringLiteral` type (which
was introduced for use with NTTP strings).
This patch makes `Shell::block_on_job()` pump the event loop while the
job it's waiting for hasn't finished.
As this no longer pushes new event loops, it has the effect of
flattening the stack as well.
Fixes#4976.
There's no guarantee that the last executed command will have a zero
exit code, and so the shell exit code may or may not be zero, even if
all the tests pass.
Also changes the `test || echo fail && exit` to
`if not test { echo fail && exit }`, since that's nicer-looking.
This allows correct iteration over nested lists.
Also store values to variables without resolving them, to delay the
resolution step as much as possible (this helps with storing nested
lists in variables).
This API was a mostly gratuitous deviation from POSIX that gave up some
portability in exchange for avoiding the occasional strlen().
I don't think that was actually achieving anything valuable, so let's
just chill out and have the same open() API as everyone else. :^)
