--- obj: concept arch-wiki: https://wiki.archlinux.org/title/Command-line_shell wiki: https://en.wikipedia.org/wiki/Unix_shell rev: 2024-09-16 --- # Shell The shell is a command-line interpreter that provides a user interface to an operating system's services. It allows users to interact with the system through text-based commands and scripts. Shell scripting refers to writing a series of commands in a script file to automate tasks and perform complex operations. The shell makes heavy use of [Environment Variables](../../linux/Environment%20Variables.md) for storing settings and configuration. ## Usage You can enter commands to be executed in the shell. ```shell command ``` Commands can be either a script, a binary or anything that can be executed. If you don't provide a full path to the file you want to run the shell will search in the locations defined in the `$PATH` environment variable. ### Keyboard Shortcuts - `Ctrl + C`: Terminate the current running process or command. - `Ctrl + Z`: Suspend the current process and push it to the background. - `Ctrl + D`: Exit the terminal or send EOF (End Of File) if running a script or command. - `Ctrl + L`: Clear the terminal screen (similar to the clear command). - `Ctrl + A`: Move the cursor to the beginning of the line. - `Ctrl + E`: Move the cursor to the end of the line. - `Ctrl + U`: Delete everything from the cursor position to the beginning of the line. - `Ctrl + K`: Delete everything from the cursor position to the end of the line. - `Ctrl + W`: Delete the word before the cursor. - `Ctrl + R`: Search command history backward interactively. - `Ctrl + T`: Swap the current character with the previous one (transpose). - `Ctrl + Y`: Paste (yank) the last killed text. - `Ctrl + P`: Previous command in history (similar to the Up arrow key). - `Ctrl + N`: Next command in history (similar to the Down arrow key). - `Alt + F`: Move forward one word. - `Alt + B`: Move backward one word. - `Alt + D`: Delete the word after the cursor. - `Shift + PageUp/PageDown`: Scroll the terminal output up or down. - `Tab`: Auto-complete file or directory names. ### Process Management - `Ctrl + Z`: Suspend a running process. - `fg`: Resume the last suspended process in the foreground. - `bg`: Resume the last suspended process in the background. - `jobs`: List all jobs currently running or suspended. ### Arguments Everything after the command will be provided to the command as arguments. Each argument is separated by a space character. To avoid that you could quote an argument or escape the space character. ```shell command argument1 argument2 argument3 # These two have identical arguments command "argument 1" "argument 2" "argument 3" command argument\ 1 argument\ 2 argument\ 3 ``` Arguments are typically structured into flags and options. Flags (like `-v` or `--verbose`) toggle specific functionality while options (like `-o output.txt` or `--out output.txt`) allow you to specify a value for the command. ### Running in the background Append `&` to a command to run it in the background. ```shell long-running-command & ``` ### Running Commands Sequentially The `;` symbol marks the end of a command. This allows you to execute multiple commands in one line. ```shell command1 ; command2 ``` ### Globs The shell supports globs like `*` for matching. For example to delete all text files in the current directory: ```shell rm -v *.txt ``` ## Redirections ### **Standard Input, Output, and Error** In a shell environment, there are three standard streams: - **Standard Input (stdin - fd 0):** Represents the input to a command. - **Standard Output (stdout - fd 1):** Represents the output of a command. - **Standard Error (stderr - fd 2):** Represents error messages generated by a command. ### **Output Redirection (`>` and `>>`)** - `>` is used to redirect standard output to a file, overwriting the file's contents if it already exists. ```shell echo "Hello, World!" > output.txt ``` - `>>` is used to redirect standard output to a file, appending to the file if it exists. ```shell echo "More text" >> output.txt ``` ### **Input Redirection (`<`)** `<` is used to redirect standard input from a file. ```shell while read line; do echo "Line: $line" done < input.txt cat < input.txt ``` ### **Error Redirection (`2>` and `2>>`)** - `2>` is used to redirect standard error to a file, overwriting the file if it exists. ```shell command_that_might_fail 2> error.log ``` - `2>>` is used to redirect standard error to a file, appending to the file if it exists. ```shell command_that_might_fail 2>> error.log ``` ### **Pipes (`|`)** The pipe operator (`|`) allows the output of one command to be used as the input for another command. This enables the creation of powerful and concise command pipelines. ```shell command1 | command2 ``` Example: Counting Lines in a File ```shell cat textfile.txt | wc -l ``` ### **Combining Redirection and Pipes** You can combine redirection and pipes to create more complex command sequences. ```shell # Redirect stderr to a file, and then pipe the output to another command command1 2> error.log | command2 ``` ### **Here Documents (`<<`)** Here documents allow you to include multiple lines of input in a script or command. `EOF` stands for End Of File. ```shell cat << EOF This is a multi-line text block. EOF ``` ### **Command Substitution (`$()`)** Command substitution allows the output of a command to replace the command itself. ```shell result=$(ls -l) echo "Listing: $result" ``` ### **Named Pipes (FIFOs)** Named pipes, or FIFOs (First In, First Out), are special files used for inter-process communication. ```shell mkfifo mypipe command1 > mypipe & # Background process writing to the pipe command2 < mypipe # Reading from the pipe ``` ### **tee Command** The `tee` command reads from standard input and writes to standard output and files simultaneously. ```shell echo "Hello, World!" | tee output.txt | wc -l ``` ### **/dev/null** `/dev/null` is a special file that discards all data written to it. ```shell command > /dev/null # Redirects output to null ``` ## Shell Scripting Shell scripting involves writing a series of commands for the shell to execute, allowing automation and the creation of more complex programs. It's a powerful way to streamline tasks and manage system configurations. ### Shebang The shebang is the first line of a script and it indicates which executable is used for the execution of the script. The syntax is `#!` followed by the absolute path to a executable. For simple shell scripts add this to the file: ```shell #!/bin/bash ``` Although if you are scripting in another language you could change it for example to `python` ```shell #!/bin/python ``` ### Comments You can add comments to a script by using the `#` symbol. ```shell # This is a comment ``` ### Variables You can store variables in a shell script by following a `key=value` syntax. ```shell name="Neo" # string age=25 # number fruits=("apple" "banana" "orange") # array # using arrays echo "First elements ${fruits[0]}" # using all elements of an array ${fruits[*]} ${fruits[@]} ``` Use variable or `default` if the variable does not exist: ```shell VR1=Value MYVAR=${VR1:-default} # $MYVAR is 'Value' #VR2=Value MYVAR=${VR2:-default} # $MYVAR is 'default' ``` ### Conditionals You can use conditional statements. ```shell if [ "$age" -eq 18 ]; then echo "You're 18!" elif [ "$age" -gt 18 ]; then echo "You're an adult!" else echo "You're a minor." fi # Case Statement (useful for checking a lot of conditions) case $name in "Neo") echo "You are the one" ;; "Morpheus") echo "See you again" ;; *) # default if nothing matches echo "Sorry, I don't understand" ;; esac ``` #### Operators ##### Arithmetic Operators Assume variable **a** holds 10 and variable **b** holds 20 then − | Operator | Description | Example | | ------------------ | --------------------------------------------------------------------- | --------------------------------------- | | + (Addition) | Adds values on either side of the operator | `expr $a + $b` will give 30 | | - (Subtraction) | Subtracts right hand operand from left hand operand | `expr $a - $b` will give -10 | | * (Multiplication) | Multiplies values on either side of the operator | `expr $a \* $b` will give 200 | | / (Division) | Divides left hand operand by right hand operand | `expr $b / $a` will give 2 | | % (Modulus) | Divides left hand operand by right hand operand and returns remainder | `expr $b % $a` will give 0 | | = (Assignment) | Assigns right operand in left operand | `a = $b` would assign value of b into a | | == (Equality) | Compares two numbers, if both are same then returns true. | `[$a == $b ]` would return false. | | != (Not Equality) | Compares two numbers, if both are different then returns true. | `[ $a != $b ]` would return true. | ##### Relational Operators For example, following operators will work to check a relation between 10 and 20 as well as in between "10" and "20" but not in between "ten" and "twenty". Assume variable **a** holds 10 and variable **b** holds 20 then − | Operator | Description | Example | | -------- | ------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------- | | **-eq** | Checks if the value of two operands are equal or not; if yes, then the condition becomes true. | `[ $a -eq $b ]` is not true. | | **-ne** | Checks if the value of two operands are equal or not; if values are not equal, then the condition becomes true. | `[ $a -ne $b ]` is true. | | **-gt** | Checks if the value of left operand is greater than the value of right operand; if yes, then the condition becomes true. | `[ $a -gt $b ]` is not true. | | **-lt** | Checks if the value of left operand is less than the value of right operand; if yes, then the condition becomes true. | `[ $a -lt $b ]` is true. | | **-ge** | Checks if the value of left operand is greater than or equal to the value of right operand; if yes, then the condition becomes true. | `[ $a -ge $b ]` is not true. | | **-le** | Checks if the value of left operand is less than or equal to the value of right operand; if yes, then the condition becomes true. | `[ $a -le $b ]` is true. | ##### Boolean Operators Assume variable **a** holds 10 and variable **b** holds 20 then − | Operator | Description | Example | | -------- | -------------------------------------------------------------------------------------------------------- | --------------------------------------- | | **!** | This is logical negation. This inverts a true condition into false and vice versa. | `[ ! false ]` is true. | | **-o** | This is logical **OR**. If one of the operands is true, then the condition becomes true. | `[ $a -lt 20 -o $b -gt 100 ]` is true. | | **-a** | This is logical **AND**. If both the operands are true, then the condition becomes true otherwise false. | `[ $a -lt 20 -a $b -gt 100 ]` is false. | ##### File Test Operators We have a few operators that can be used to test various properties associated with a Unix file. Assume a variable **file** holds an existing file name "test" the size of which is 100 bytes and has **read**, **write** and **execute** permission. | Operator | Description | Example | | ----------- | ---------------------------------------------------------------------------------------------------------------------- | --------------------------- | | **-b file** | Checks if file is a block special file; if yes, then the condition becomes true. | `[ -b $file ]` is false. | | **-c file** | Checks if file is a character special file; if yes, then the condition becomes true. | `[ -c $file ]` is false. | | **-d file** | Checks if file is a directory; if yes, then the condition becomes true. | `[ -d $file ]` is not true. | | **-f file** | Checks if file is an ordinary file as opposed to a directory or special file; if yes, then the condition becomes true. | `[ -f $file ]` is true. | | **-g file** | Checks if file has its set group ID (SGID) bit set; if yes, then the condition becomes true. | `[ -g $file ]` is false. | | **-k file** | Checks if file has its sticky bit set; if yes, then the condition becomes true. | `[ -k $file ]` is false. | | **-p file** | Checks if file is a named pipe; if yes, then the condition becomes true. | `[ -p $file ]` is false. | | **-t file** | Checks if file descriptor is open and associated with a terminal; if yes, then the condition becomes true. | `[ -t $file ]` is false. | | **-u file** | Checks if file has its Set User ID (SUID) bit set; if yes, then the condition becomes true. | `[ -u $file ]` is false. | | **-r file** | Checks if file is readable; if yes, then the condition becomes true. | `[ -r $file ]` is true. | | **-w file** | Checks if file is writable; if yes, then the condition becomes true. | `[ -w $file ]` is true. | | **-x file** | Checks if file is executable; if yes, then the condition becomes true. | `[ -x $file ]` is true. | | **-s file** | Checks if file has size greater than 0; if yes, then condition becomes true. | `[ -s $file ]` is true. | | **-e file** | Checks if file exists; is true even if file is a directory but exists. | `[ -e $file ]` is true. | ### Loops You can also use loops. ```shell # For Loop for fruit in "${fruits[@]}"; do echo "Fruit: $fruit" done # Example: Loop over files for file in *.txt; do echo "File: $file"; done # While Loop count=0 while [ $count -lt 5 ]; do echo "Count: $count" ((count++)) done ``` You can use a few commands to control the loop. - `break` breaks out of the loop, ending it prematurely - `continue` skips to the next iteration of the loop, but skipping everything coming after it If you want to loop over every single line even if one line might contain spaces (from a command output for example) you can use this trick: ```shell command | while IFS= read -r varName; do echo "Working on $varName" done ``` ### Functions You can define your own functions. The arguments you give to the functions can be accessed via `$1`, `$2`, `$n`, etc. The same way the arguments passed to a shell script can be accessed. To get all passed in arguments as is use `$@`. ```shell greet() { echo "Hello, $1!" } # Function Call greet "Alice" ``` Your functions can also have return values. ```shell add() { local result=$(( $1 + $2 )) echo $result } # Function Call sum=$(add 5 3) echo "Sum: $sum" ``` ### Input Output Read user input into a variable. ```shell echo "Enter your name: " read username echo "Hello, $username!" ``` Output to Screen (`echo`, `printf`): ```shell echo "Hello, World!" ```