378 lines
16 KiB
Markdown
378 lines
16 KiB
Markdown
---
|
||
obj: concept
|
||
arch-wiki: https://wiki.archlinux.org/title/Command-line_shell
|
||
wiki: https://en.wikipedia.org/wiki/Unix_shell
|
||
rev: 2024-09-16
|
||
---
|
||
|
||
# Shell
|
||
The shell is a command-line interpreter that provides a user interface to an operating system's services. It allows users to interact with the system through text-based commands and scripts. Shell scripting refers to writing a series of commands in a script file to automate tasks and perform complex operations.
|
||
|
||
The shell makes heavy use of [Environment Variables](../../linux/Environment%20Variables.md) for storing settings and configuration.
|
||
|
||
## Usage
|
||
You can enter commands to be executed in the shell.
|
||
```shell
|
||
command
|
||
```
|
||
|
||
Commands can be either a script, a binary or anything that can be executed. If you don't provide a full path to the file you want to run the shell will search in the locations defined in the `$PATH` environment variable.
|
||
|
||
### Keyboard Shortcuts
|
||
- `Ctrl + C`: Terminate the current running process or command.
|
||
- `Ctrl + Z`: Suspend the current process and push it to the background.
|
||
- `Ctrl + D`: Exit the terminal or send EOF (End Of File) if running a script or command.
|
||
- `Ctrl + L`: Clear the terminal screen (similar to the clear command).
|
||
- `Ctrl + A`: Move the cursor to the beginning of the line.
|
||
- `Ctrl + E`: Move the cursor to the end of the line.
|
||
- `Ctrl + U`: Delete everything from the cursor position to the beginning of the line.
|
||
- `Ctrl + K`: Delete everything from the cursor position to the end of the line.
|
||
- `Ctrl + W`: Delete the word before the cursor.
|
||
- `Ctrl + R`: Search command history backward interactively.
|
||
- `Ctrl + T`: Swap the current character with the previous one (transpose).
|
||
- `Ctrl + Y`: Paste (yank) the last killed text.
|
||
- `Ctrl + P`: Previous command in history (similar to the Up arrow key).
|
||
- `Ctrl + N`: Next command in history (similar to the Down arrow key).
|
||
- `Alt + F`: Move forward one word.
|
||
- `Alt + B`: Move backward one word.
|
||
- `Alt + D`: Delete the word after the cursor.
|
||
- `Shift + PageUp/PageDown`: Scroll the terminal output up or down.
|
||
- `Tab`: Auto-complete file or directory names.
|
||
|
||
### Process Management
|
||
- `Ctrl + Z`: Suspend a running process.
|
||
- `fg`: Resume the last suspended process in the foreground.
|
||
- `bg`: Resume the last suspended process in the background.
|
||
- `jobs`: List all jobs currently running or suspended.
|
||
|
||
### Arguments
|
||
Everything after the command will be provided to the command as arguments. Each argument is separated by a space character. To avoid that you could quote an argument or escape the space character.
|
||
|
||
```shell
|
||
command argument1 argument2 argument3
|
||
|
||
# These two have identical arguments
|
||
command "argument 1" "argument 2" "argument 3"
|
||
command argument\ 1 argument\ 2 argument\ 3
|
||
```
|
||
|
||
Arguments are typically structured into flags and options.
|
||
Flags (like `-v` or `--verbose`) toggle specific functionality while options (like `-o output.txt` or `--out output.txt`) allow you to specify a value for the command.
|
||
|
||
### Running in the background
|
||
Append `&` to a command to run it in the background.
|
||
```shell
|
||
long-running-command &
|
||
```
|
||
|
||
### Running Commands Sequentially
|
||
The `;` symbol marks the end of a command. This allows you to execute multiple commands in one line.
|
||
|
||
```shell
|
||
command1 ; command2
|
||
```
|
||
|
||
### Globs
|
||
The shell supports globs like `*` for matching.
|
||
For example to delete all text files in the current directory:
|
||
```shell
|
||
rm -v *.txt
|
||
```
|
||
|
||
## Redirections
|
||
### **Standard Input, Output, and Error**
|
||
In a shell environment, there are three standard streams:
|
||
- **Standard Input (stdin - fd 0):** Represents the input to a command.
|
||
- **Standard Output (stdout - fd 1):** Represents the output of a command.
|
||
- **Standard Error (stderr - fd 2):** Represents error messages generated by a command.
|
||
|
||
### **Output Redirection (`>` and `>>`)**
|
||
- `>` is used to redirect standard output to a file, overwriting the file's contents if it already exists.
|
||
```shell
|
||
echo "Hello, World!" > output.txt
|
||
```
|
||
- `>>` is used to redirect standard output to a file, appending to the file if it exists.
|
||
```shell
|
||
echo "More text" >> output.txt
|
||
```
|
||
|
||
### **Input Redirection (`<`)**
|
||
`<` is used to redirect standard input from a file.
|
||
```shell
|
||
while read line; do
|
||
echo "Line: $line"
|
||
done < input.txt
|
||
|
||
cat < input.txt
|
||
```
|
||
|
||
### **Error Redirection (`2>` and `2>>`)**
|
||
- `2>` is used to redirect standard error to a file, overwriting the file if it exists.
|
||
```shell
|
||
command_that_might_fail 2> error.log
|
||
```
|
||
- `2>>` is used to redirect standard error to a file, appending to the file if it exists.
|
||
```shell
|
||
command_that_might_fail 2>> error.log
|
||
```
|
||
|
||
### **Pipes (`|`)**
|
||
The pipe operator (`|`) allows the output of one command to be used as the input for another command. This enables the creation of powerful and concise command pipelines.
|
||
```shell
|
||
command1 | command2
|
||
```
|
||
|
||
Example: Counting Lines in a File
|
||
```shell
|
||
cat textfile.txt | wc -l
|
||
```
|
||
|
||
### **Combining Redirection and Pipes**
|
||
You can combine redirection and pipes to create more complex command sequences.
|
||
```shell
|
||
# Redirect stderr to a file, and then pipe the output to another command
|
||
command1 2> error.log | command2
|
||
```
|
||
|
||
### **Here Documents (`<<`)**
|
||
Here documents allow you to include multiple lines of input in a script or command. `EOF` stands for End Of File.
|
||
```shell
|
||
cat << EOF
|
||
This is a
|
||
multi-line
|
||
text block.
|
||
EOF
|
||
```
|
||
|
||
### **Command Substitution (`$()`)**
|
||
Command substitution allows the output of a command to replace the command itself.
|
||
```shell
|
||
result=$(ls -l)
|
||
echo "Listing: $result"
|
||
```
|
||
|
||
### **Named Pipes (FIFOs)**
|
||
Named pipes, or FIFOs (First In, First Out), are special files used for inter-process communication.
|
||
```shell
|
||
mkfifo mypipe
|
||
command1 > mypipe & # Background process writing to the pipe
|
||
command2 < mypipe # Reading from the pipe
|
||
```
|
||
|
||
### **tee Command**
|
||
The `tee` command reads from standard input and writes to standard output and files simultaneously.
|
||
```shell
|
||
echo "Hello, World!" | tee output.txt | wc -l
|
||
```
|
||
|
||
### **/dev/null**
|
||
`/dev/null` is a special file that discards all data written to it.
|
||
|
||
```shell
|
||
command > /dev/null # Redirects output to null
|
||
```
|
||
|
||
## Shell Scripting
|
||
Shell scripting involves writing a series of commands for the shell to execute, allowing automation and the creation of more complex programs. It's a powerful way to streamline tasks and manage system configurations.
|
||
|
||
### Shebang
|
||
The shebang is the first line of a script and it indicates which executable is used for the execution of the script. The syntax is `#!` followed by the absolute path to a executable.
|
||
|
||
For simple shell scripts add this to the file:
|
||
```shell
|
||
#!/bin/bash
|
||
```
|
||
|
||
Although if you are scripting in another language you could change it for example to `python`
|
||
|
||
```shell
|
||
#!/bin/python
|
||
```
|
||
|
||
### Comments
|
||
You can add comments to a script by using the `#` symbol.
|
||
```shell
|
||
# This is a comment
|
||
```
|
||
|
||
### Variables
|
||
You can store variables in a shell script by following a `key=value` syntax.
|
||
|
||
```shell
|
||
name="Neo" # string
|
||
age=25 # number
|
||
fruits=("apple" "banana" "orange") # array
|
||
|
||
# using arrays
|
||
echo "First elements ${fruits[0]}"
|
||
|
||
# using all elements of an array
|
||
${fruits[*]}
|
||
${fruits[@]}
|
||
```
|
||
|
||
Use variable or `default` if the variable does not exist:
|
||
```shell
|
||
VR1=Value
|
||
MYVAR=${VR1:-default} # $MYVAR is 'Value'
|
||
|
||
#VR2=Value
|
||
MYVAR=${VR2:-default} # $MYVAR is 'default'
|
||
```
|
||
|
||
### Conditionals
|
||
You can use conditional statements.
|
||
```shell
|
||
if [ "$age" -eq 18 ]; then
|
||
echo "You're 18!"
|
||
elif [ "$age" -gt 18 ]; then
|
||
echo "You're an adult!"
|
||
else
|
||
echo "You're a minor."
|
||
fi
|
||
|
||
# Case Statement (useful for checking a lot of conditions)
|
||
case $name in
|
||
"Neo")
|
||
echo "You are the one"
|
||
;;
|
||
"Morpheus")
|
||
echo "See you again"
|
||
;;
|
||
*)
|
||
# default if nothing matches
|
||
echo "Sorry, I don't understand"
|
||
;;
|
||
esac
|
||
```
|
||
|
||
#### Operators
|
||
##### Arithmetic Operators
|
||
Assume variable **a** holds 10 and variable **b** holds 20 then −
|
||
|
||
| Operator | Description | Example |
|
||
| ------------------ | --------------------------------------------------------------------- | --------------------------------------- |
|
||
| + (Addition) | Adds values on either side of the operator | `expr $a + $b` will give 30 |
|
||
| - (Subtraction) | Subtracts right hand operand from left hand operand | `expr $a - $b` will give -10 |
|
||
| * (Multiplication) | Multiplies values on either side of the operator | `expr $a \* $b` will give 200 |
|
||
| / (Division) | Divides left hand operand by right hand operand | `expr $b / $a` will give 2 |
|
||
| % (Modulus) | Divides left hand operand by right hand operand and returns remainder | `expr $b % $a` will give 0 |
|
||
| = (Assignment) | Assigns right operand in left operand | `a = $b` would assign value of b into a |
|
||
| == (Equality) | Compares two numbers, if both are same then returns true. | `[$a == $b ]` would return false. |
|
||
| != (Not Equality) | Compares two numbers, if both are different then returns true. | `[ $a != $b ]` would return true. |
|
||
|
||
##### Relational Operators
|
||
For example, following operators will work to check a relation between 10 and 20 as well as in between "10" and "20" but not in between "ten" and "twenty".
|
||
|
||
Assume variable **a** holds 10 and variable **b** holds 20 then −
|
||
|
||
| Operator | Description | Example |
|
||
| -------- | ------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------- |
|
||
| **-eq** | Checks if the value of two operands are equal or not; if yes, then the condition becomes true. | `[ $a -eq $b ]` is not true. |
|
||
| **-ne** | Checks if the value of two operands are equal or not; if values are not equal, then the condition becomes true. | `[ $a -ne $b ]` is true. |
|
||
| **-gt** | Checks if the value of left operand is greater than the value of right operand; if yes, then the condition becomes true. | `[ $a -gt $b ]` is not true. |
|
||
| **-lt** | Checks if the value of left operand is less than the value of right operand; if yes, then the condition becomes true. | `[ $a -lt $b ]` is true. |
|
||
| **-ge** | Checks if the value of left operand is greater than or equal to the value of right operand; if yes, then the condition becomes true. | `[ $a -ge $b ]` is not true. |
|
||
| **-le** | Checks if the value of left operand is less than or equal to the value of right operand; if yes, then the condition becomes true. | `[ $a -le $b ]` is true. |
|
||
|
||
##### Boolean Operators
|
||
Assume variable **a** holds 10 and variable **b** holds 20 then −
|
||
|
||
| Operator | Description | Example |
|
||
| -------- | -------------------------------------------------------------------------------------------------------- | --------------------------------------- |
|
||
| **!** | This is logical negation. This inverts a true condition into false and vice versa. | `[ ! false ]` is true. |
|
||
| **-o** | This is logical **OR**. If one of the operands is true, then the condition becomes true. | `[ $a -lt 20 -o $b -gt 100 ]` is true. |
|
||
| **-a** | This is logical **AND**. If both the operands are true, then the condition becomes true otherwise false. | `[ $a -lt 20 -a $b -gt 100 ]` is false. |
|
||
|
||
##### File Test Operators
|
||
We have a few operators that can be used to test various properties associated with a Unix file.
|
||
|
||
Assume a variable **file** holds an existing file name "test" the size of which is 100 bytes and has **read**, **write** and **execute** permission.
|
||
|
||
| Operator | Description | Example |
|
||
| ----------- | ---------------------------------------------------------------------------------------------------------------------- | --------------------------- |
|
||
| **-b file** | Checks if file is a block special file; if yes, then the condition becomes true. | `[ -b $file ]` is false. |
|
||
| **-c file** | Checks if file is a character special file; if yes, then the condition becomes true. | `[ -c $file ]` is false. |
|
||
| **-d file** | Checks if file is a directory; if yes, then the condition becomes true. | `[ -d $file ]` is not true. |
|
||
| **-f file** | Checks if file is an ordinary file as opposed to a directory or special file; if yes, then the condition becomes true. | `[ -f $file ]` is true. |
|
||
| **-g file** | Checks if file has its set group ID (SGID) bit set; if yes, then the condition becomes true. | `[ -g $file ]` is false. |
|
||
| **-k file** | Checks if file has its sticky bit set; if yes, then the condition becomes true. | `[ -k $file ]` is false. |
|
||
| **-p file** | Checks if file is a named pipe; if yes, then the condition becomes true. | `[ -p $file ]` is false. |
|
||
| **-t file** | Checks if file descriptor is open and associated with a terminal; if yes, then the condition becomes true. | `[ -t $file ]` is false. |
|
||
| **-u file** | Checks if file has its Set User ID (SUID) bit set; if yes, then the condition becomes true. | `[ -u $file ]` is false. |
|
||
| **-r file** | Checks if file is readable; if yes, then the condition becomes true. | `[ -r $file ]` is true. |
|
||
| **-w file** | Checks if file is writable; if yes, then the condition becomes true. | `[ -w $file ]` is true. |
|
||
| **-x file** | Checks if file is executable; if yes, then the condition becomes true. | `[ -x $file ]` is true. |
|
||
| **-s file** | Checks if file has size greater than 0; if yes, then condition becomes true. | `[ -s $file ]` is true. |
|
||
| **-e file** | Checks if file exists; is true even if file is a directory but exists. | `[ -e $file ]` is true. |
|
||
|
||
### Loops
|
||
|
||
You can also use loops.
|
||
```shell
|
||
# For Loop
|
||
for fruit in "${fruits[@]}"; do
|
||
echo "Fruit: $fruit"
|
||
done
|
||
|
||
# Example: Loop over files
|
||
for file in *.txt; do
|
||
echo "File: $file";
|
||
done
|
||
|
||
# While Loop
|
||
count=0
|
||
while [ $count -lt 5 ]; do
|
||
echo "Count: $count"
|
||
((count++))
|
||
done
|
||
```
|
||
|
||
You can use a few commands to control the loop.
|
||
- `break` breaks out of the loop, ending it prematurely
|
||
- `continue` skips to the next iteration of the loop, but skipping everything coming after it
|
||
|
||
If you want to loop over every single line even if one line might contain spaces (from a command output for example) you can use this trick:
|
||
```shell
|
||
command | while IFS= read -r varName; do
|
||
echo "Working on $varName"
|
||
done
|
||
```
|
||
|
||
### Functions
|
||
You can define your own functions. The arguments you give to the functions can be accessed via `$1`, `$2`, `$n`, etc. The same way the arguments passed to a shell script can be accessed.
|
||
To get all passed in arguments as is use `$@`.
|
||
|
||
```shell
|
||
greet() {
|
||
echo "Hello, $1!"
|
||
}
|
||
|
||
# Function Call
|
||
greet "Alice"
|
||
```
|
||
|
||
Your functions can also have return values.
|
||
```shell
|
||
add() {
|
||
local result=$(( $1 + $2 ))
|
||
echo $result
|
||
}
|
||
|
||
# Function Call
|
||
sum=$(add 5 3)
|
||
echo "Sum: $sum"
|
||
```
|
||
|
||
### Input Output
|
||
Read user input into a variable.
|
||
```shell
|
||
echo "Enter your name: "
|
||
read username
|
||
echo "Hello, $username!"
|
||
```
|
||
|
||
Output to Screen (`echo`, `printf`):
|
||
```shell
|
||
echo "Hello, World!"
|
||
```
|