# Bash ## Resources - [BashGuide](https://mywiki.wooledge.org/BashGuide) - [bash cheat sheet](https://mywiki.wooledge.org/BashSheet) - [pure bash bible](https://github.com/dylanaraps/pure-bash-bible) - [Cronjobs](https://crontab.guru/) ## CDPATH - `CDPATH` is an environmental variable that makes it so that when using the `cd` command it will list items in your current directory [[( 2022-03-04 Secret Features in Your Unix Shell CDPATH]] <iframe width="560" height="315" src="https://www.youtube.com/embed/4-Nun5c3qeA" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe> ## Command Lists - `{ [command list]; }` - Execute the list of commands in the current shell as though they were one command. - Command grouping on its own isn't very useful. However, it comes into play wherever Bash syntax accepts only one command while you need to execute multiple. - For example, you may want to pass output of multiple commands via a pipe to another command's input: - ```bash { ls .; ls ..; } | grep file-name ``` - Or you may want to execute multiple commands after a || operator: ```bash rm file || { echo "Removal failed, aborting."; exit 1; } ``` - It is also used for [[#Functions In Bash]] bodies. Technically, this can also be used for loop bodies though this is undocumented, not portable and we normally prefer `do ...; done` for this): ```bash for digit in 1 9 7; { echo "$digit"; } # non-portable, undocumented, unsupported - for digit in 1 9 7; do echo "$digit"; done # preferred ``` - **\*Note**: You need a `;` before the closing `}` (or it must be on a new line). - Command Lists are similar but not identical to ## Command Substitution - `( [command list] )` - Execute the list of commands in a subshell. - This is exactly the same thing as the command grouping above, only, the commands are executed in a subshell. Any code that affects the environment such as variable assignments, cd, export, etc. do not affect the main script's environment but are scoped within the brackets. - **\*Note:** You do not need a `;` before the closing `)`. ## Documentation - [bashtips](https://drawings.jvns.ca/bashtips/) - [quoting variables SO](https://stackoverflow.com/questions/10067266/when-to-wrap-quotes-around-a-shell-variable#10067297) ## Exit Status In Bash - You can check the exit status with the exit status variable: ```shell echo "My script's exit status is '$?'" ``` - `$?` is only required if you need to retrieve the exact status of the previous command. If you only need to test for success or failure (any non-zero status), just test the command directly. e.g.: ```shell if cmd; then ... fi ``` ### Documentation - <https://mywiki.wooledge.org/BashPitfalls> ## fencing ### braces #### Curly Braces ```shell ${variable} # Parameter substitution ${!variable} # Indirect variable reference { command1; command2; . . . commandN; } # Block of code {string1,string2,string3,...} # Brace expansion {a..z} # Extended brace expansion {} # Text replacement, after find and xargs ``` ### parens #### Parentheses ```shell ( command1; command2 ) # Command group executed within a subshell Array=(element1 element2 element3) # Array initialization result=$(COMMAND) # Command substitution, new style >(COMMAND) # Process substitution <(COMMAND) # Process substitution ``` #### Double Parentheses ```shell (( var = 78 )) # Integer arithmetic var=$(( 20 + 5 )) # Integer arithmetic, with variable assignment (( var++ )) # C-style variable increment (( var-- )) # C-style variable decrement (( var0 = var1<98?9:21 )) # C-style ternary operation ``` ```shell ((a++)) ((meaning = 42)) for ((i=0; i<10; i++)) echo $((a + b + (14 * c))) ``` and they enable you to omit the dollar signs on integer and array variables and include spaces around operators for readability. ### square brackets #### Brackets <http://wiki.bash-hackers.org/scripting/obsolete> ```shell if [ CONDITION ] # Test construct if [[ CONDITION ]] # Extended test construct Array[1]=element1 # Array initialization [a-z] # Range of characters within a Regular Expression $[ expression ] # A non-standard & obsolete version of $(( expression )) ``` Single brackets are also used for array indices: ```shell array[4]="hello" element=${array[index]} ``` ## File Handling ### Documentation - [SO answer](https://unix.stackexchange.com/questions/181937/how-create-a-temporary-file-in-shell-script) ### Using Temp Files With Bash - Temp files are stored in the `/tmp/` directory and will need to be removed upon script completion. Why use a temp file or directory? Because it will make it easier to have a storing place of a file that holds some information you want to be read, or just act as an intermediary file/directory. ```bash # Documentation # All temp files need those X's for randomization # check out `man mktemp` for more details tmpfile=$(mktemp /tmp/abc-script.XXXXXX) # <CODE> rm "$tmpfile" ``` ### Avoid conflicts with filenames that start with a dash ```bash # -- indicates the end of the options section # -myFile.txt uses a dash after the options section to avoid conflicts cat -- -myFile.txt ``` ## flow ### Bash Test Conditionals - Testing conditions is down to using square bracket syntax: ```bash if [ 1 = 3 ] || [ 2 = 2 ]; then echo "yes!" fi ``` - To note, double brackets are a BASH-ism and not POSIX compliant. To test multiple conditions for the same evaluation do not use the internal `-a` for and or the `-o` for or like: ```bash if [ 1 = 3 -o 2 = 2 ]; then echo "yes!" fi ``` - Instead separate the tests into separate commands with the `&&` operator. This way it will run each test as a separate command, AND it will only continue forward if the condition is true: - `<this has to be true> && "AND" <this has to be true, to continue>`. - Double quote variables use in the test condition unless you explicitly know and understand why they should be unquoted. ### loops #### Loops In Bash - For looping over all positional arguments / words sent to the command you can use the following: ```bash for arg in "$@" # Or simply: for arg ``` - Since looping over the positional parameters is such a common thing to do in scripts, for arg defaults to for arg in `"$@"`. The double-quoted `"$@"` is special magic that causes each parameter to be used as a single word (or a single loop iteration). It's what you should be using at least 99% of the time. ```bash # Correct version for x in "$@"; do echo "parameter: '$x'" done # or better: for x do echo "parameter: '$x'" done $ ./myscript 'arg 1' arg2 arg3 #> parameter: 'arg 1' #> parameter: 'arg2' #> parameter: 'arg3' ``` ```bash do [command list]; done ``` - This constitutes the actual loop that is used by the next few commands. The list of commands between the do and done are the commands that will be executed in every iteration of the loop. ```bash for [name] in [words] ``` - The next loop will iterate over each WORD after the in keyword. The loop's commands will be executed with the value of the variable denoted by name set to the word. ```bash for (( [arithmetic expression]; [arithmetic expression]; [arithmetic expression] )) ``` - The next loop will run as long as the second arithmetic expression remains true. The first arithmetic expression will be run before the loop starts. The third arithmetic expression will be run after the last command in each iteration has been executed. ```bash while [command list] ``` - The next loop will be repeated for as long as the last command ran in the command list exits successfully. ```bash until [command list] ``` - The next loop will be repeated for as long as the last command ran in the command list exits unsuccessfully ("fails"). ```bash select [name] in [words] ``` - The next loop will repeat forever, letting the user choose between the given words. - The iteration's commands are executed with the variable denoted by name's value set to the word chosen by the user. Naturally, you can use break to end this loop. #### for ```bash for filename in *; do echo "put ${filename}"; done for file in *; do if [ -f "$file" ]; then echo "$file" fi ``` ### case #### Bash Case Statements - Case statement syntax in bash is a little strange but also really easy and they are very friendly and nice. - case will look for a value in a list of options, each option is appended with a paren then the commands that this option will run. these options can also expand with wild cards. To have a final catch all statement for anything that didnt meet a prior condition use `*` as the final case option. ```bash case "$RESPONSE" in n) exit ;; N) exit ;; q) exit ;; Q) exit ;; y) mkdir $DIR && touch $TEMPLATE_DECK && echo "category1:question1:answer1" >> $TEMPLATE_DECK && echo "category2:question2:answer2" >> $TEMPLATE_DECK && echo "category3:question3:answer3" >> $TEMPLATE_DECK && echo "$DIR_MADE_MSG" ;; Y) mkdir $DIR && touch $TEMPLATE_DECK && echo "category1:question1:answer1" >> $TEMPLATE_DECK && echo "category2:question2:answer2" >> $TEMPLATE_DECK && echo "category3:question3:answer3" >> $TEMPLATE_DECK && echo "$DIR_MADE_MSG" ;; *) echo "invalid choice, please select either 'y' or 'n'" ;; esac ``` ### if-else #### Bash IF Statements - Many beginners have an incorrect intuition about `if` statements brought about by seeing the very common pattern of an if keyword followed immediately by a `[` or `[[`. This convinces people that the `[` is somehow part of the `if` statement's syntax, just like parentheses used in C's if statement. - This is not the case! `if` takes a command. `[` is a command, not a syntax marker for the if statement. It's equivalent to the test command, except that the final argument must be a `]`. For example: ```bash # POSIX if [ false ]; then echo "HELP"; fi if test false; then echo "HELP"; fi ``` - Are equivalent -- both checking that the argument "false" is non-empty. In both cases HELP will always be printed, to the surprise of programmers from other languages guessing about shell syntax. - The syntax of an if statement is: ```bash if COMMANDS then <COMMANDS> elif <COMMANDS> # optional then <COMMANDS> else <COMMANDS> # optional fi # required ``` ### ternary test #### Ternary Tests - [pure bash bible](https://github.com/dylanaraps/pure-bash-bible) ```bash # Ternary Tests ## Set the value of var to var2 if var2 is greater than var. ## var: variable to set. ## var2>var: Condition to test. ## ?var2: If the test succeeds. ## :var: If the test fails. ((var=var2>var?var2:var)) ``` ## Functions In Bash - [[#Variables]] assigned outside of functions can be overwritten in functions and display aberrant results if local isn't used. ```bash foo="bar" echo "top level: $foo" main(){ foo="func bar" echo "func $foo" } main echo "end $foo" # OUTPUT: #> top level: bar #> func func bar #> end func bar #===#===#===#===#===#===# foo="bar" echo "top level: $foo" main(){ local foo="func bar" echo "func $foo" } main echo "end $foo" # OUTPUT: #> top level: bar #> func func bar #> end bar ``` - Do not use the function keyword, it reduces compatibility with older versions of bash. ```bash # Right. do_something() { # ... } # Wrong. function do_something() { # ... } ``` ```bash # Current function. "${FUNCNAME[0]}" # Parent function. "${FUNCNAME[1]}" # So on and so forth. "${FUNCNAME[2]}" "${FUNCNAME[3]}" # All functions including parents. "${FUNCNAME[@]}" ``` ## Get Opts In Bash how to get flag options in a bash script ```bash # Documentation a_flag='' b_flag='' files='' verbose='false' print_usage() { printf "Usage: ..." } while getopts 'abf:v' flag; do case "${flag}" in a) a_flag='true' ;; b) b_flag='true' ;; f) files="${OPTARG}" ;; v) verbose='true' ;; *) print_usage exit 1 ;; esac done ``` ###### Documentation - [SO Article](https://stackoverflow.com/questions/7069682/how-to-get-arguments-with-flags-in-bash#21128172) ## Internal Field Separator - IFS `IFS` = Internal Field Separator. Unbelievable as it may seem, POSIX requires the treatment of IFS as a field terminator, rather than a field separator. What this means in our example is that if there's an empty field at the end of the input line, it will be discarded: ```shell IFS=, read -ra fields <<< "a,b," declare -p fields #> declare -a fields='([0]="a" [1]="b")' ``` Where did the empty field go? It was eaten for historical reasons ("because it's always been that way"). This behavior is not unique to bash; all conforming shells do it. A non-empty field is properly scanned: ```shell IFS=, read -ra fields <<< "a,b,c" declare -p fields #> declare -a fields='([0]="a" [1]="b" [2]="c")' ``` So, how do we work around this nonsense? As it turns out, appending an `IFS` character to the end of the input string will force the scanning to work. If there was a trailing empty field, the extra `IFS` character "terminates" it so that it gets scanned. If there was a trailing non-empty field, the `IFS` &lt;character creates a new, empty field that gets dropped. ```shell input="a,b," IFS=, read -ra fields <<< "$input," declare -p fields #> declare -a fields='([0]="a" [1]="b" [2]="")' ``` <https://mywiki.wooledge.org/BashPitfalls> <https://mywiki.wooledge.org/IFS> ## Interpolation In Bash - [Variable Interpolation](https://en.wikipedia.org/wiki/String_interpolation) In computer programming, string interpolation is the process of evaluating a string literal containing one or more placeholders, yielding a result in which the placeholders are replaced with their corresponding values. It is a form of simple template processing or, in formal terms, a form of quasi-quotation. ```shell # Documentation var="theres no place like '$HOME'" echo "$var" #> theres no place like /Users/bryanjenks ``` ## Arithmetic Expansion - `(( [arithmetic expression] ))` - Evaluates the given expression in an arithmetic context. - That means, strings are considered names of integer variables, all operators are considered arithmetic operators (such as ++, \\=\\=, >, &lt;=, etc..) ==You should always use this for performing tests on numbers!== - `$(( [arithmetic expression] ))` - Expands the result of the given expression in an arithmetic context. - This syntax is similar to the previous, but expands into the result of the expansion. We use it inside other commands when we want the result of the arithmetic expression to become part of another command. ## Projects ### question extractor **Question Extractor Written In Bash** ```shell if [[ -e $1 ]] then filename="Questions $1" count=$(grep -Fc '**Q**' "$1") report="$count Questions extracted from $filename" printf "# $report \n \n" > "./$filename" grep -Fn '**Q**' "$1" | sed -e 's/\*\*Q:*\*\*:*//g' | sed -e 's/^/- /' >> "$filename" echo $report else echo Provide a file as an argument fi ``` This script will take a file as an input argument and then any lines it finds the text `**Q**` on it will extract into a list of questions in a new file called `questions_<input file name>` and also report in the CLI how many questions were extracted ## Bash Redirections When Bash starts, normally, 3 file descriptors are opened, 0, 1 and 2 also known as standard input (`stdin`), standard output (`stdout`) and standard error (`stderr`). | num | val | | --- | -------- | | 0 | `stdin` | | 1 | `stdout` | | 2 | `stderr` | ```shell 2>/dev/null ``` ### Documentation - <https://wiki.bash-hackers.org/howto/redirection_tutorial> ## Text String Manipulation ### case modify #### Documentation - See: <https://gist.github.com/cfraizer/8f17c375837f6d904bcafd3adaa8466d> for the code. #### Text Case Modification In Bash - I realize that this is beside your point, but…don't shell out to `tr` like you did. It's really slow and Bash has built-in facilities for manipulating strings—especially case. Equivalent to your code `input=$(echo "$value" | tr '[:upper:]' '[:lower:]')` would be something like: - input="$\*" - input="${input,,}" - We are assigning a new value to the `bash` variable `input`. The right-hand-side of the `=` is the new value. If we used `${input}`, that would just be the value already in variable `input` The magic is in those two commas `,,`. A `,` operator after the variable name downcases the first letter of the variable and leaves the rest of the value unchanged. The double-comma `,,` operator after the variable downcases every character in the value. - You can use `^` and `==` for uppercasing. - You could have done the same in a single line with `input="${*,,}"` - I ran the `tr` version as written above 1000 times and a "pure" Bash equivalent The `tr` version took 4.1 sec versus "pure" Bash's 0.04 sec (100 X faster). ### param #### Parameter What does it do? ```shell ${VAR^} # Uppercase first character. ${VAR==} # Uppercase all characters. ${VAR,} # Lowercase first character. ${VAR,,} # Lowercase all characters. ${VAR~} # Reverse case of first character. ${VAR~~} # Reverse case of all characters. ``` ```shell #!/usr/bin/env bash foo() { local value="The Quick Brown FOX Jumped over The Lazy Dog." local -i loopCount=1000 local -i i=0 for (( i = 0; i < loopCount; ++i )); do local newVal="" newVal="${value,}" printf "%s\n" "$newVal" done } bar() { local value="The Quick Brown FOX Jumped over The Lazy Dog." local -i loopCount=1000 local -i i=0 for (( i = 0; i < loopCount; ++i )); do # shellcheck disable=SC2155 local newVal=$(echo "$value" | tr '[:upper:]' '[:lower:]') printf "%s\n" "$newVal" done } baz() { local value="$*" printf "%s\n" "${value,,}" } foo bar baz # Lower Case Conversion lower() { # Usage: lower "string" printf '%s\n' "${1,,}" } # Upper Case Conversion upper() { # Usage: upper "string" printf '%s\n' "${1==}" } # Reverse Case Conversion reverse_case() { # Usage: reverse_case "string" printf '%s\n' "${1~~}" } ``` ### text blocks #### Documentation - <https://mywiki.wooledge.org/BashPitfalls> #### Text Blocks In Bash - You can have blocks of formatted text in a script for use such as a help menu If you want some of the text to be dynamic you will need to wrap it in double quote. Also echo CAN work for this but its just better to use `cat` or `printf` ```shell cat <<EOF Hello world How's it going? EOF # Or use printf (also efficient, printf is built-in): printf %s "\ Hello world How's it going? " ``` ### text repl #### Documentation - [pure bash bible](https://github.com/dylanaraps/pure-bash-bible) #### Text Replacement In Bash - Bash tip: instead of spawning an instance of `sed`, you can do text replacement in "pure" Bash like `status="${status//,/}"`. Breaking that down: - `status=` # assign a new value to the Bash variable `status`. - `status="${ … }"` # I just always use double quotes when doing parameter (and command) substitution. - `status="${status … }"` # the new value I am assigning to `status` is the expansion of (value of) `status`, BUT first we alter that value… - `status="${status//PATTERN/STRING}"` # the `//` means "globally replace" (a single `/` would replace the first occurrence of PATTERN) ```shell status="${status//,/}" status="${status//PATTERN/STRING}" firstString="I love Suzi and Marry, but Suzi Most" secondString="Sara" echo "${firstString/Suzi/$secondString}" # prints 'I love Sara and Marry' ``` ### substring removal #### Documentation - [SO Answer](https://stackoverflow.com/questions/16623835/remove-a-fixed-prefix-suffix-from-a-string-in-bash#16623897) - [puse bash bible](https://github.com/dylanaraps/pure-bash-bible) <iframe width="560" height="315" src="https://www.youtube.com/embed/QXineadwG4E" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe> #### Sub-String Removal In Bash Removing substrings without `awk`/`cut` function calls ```shell string="hello-world" prefix="hell" suffix="ld" foo=${string#"$prefix"} foo=${foo%"$suffix"} echo "${foo}" #===#===#===#===#===#===#===# string="hello-world" prefix="hell" suffix="ld" #remove "hell" from "hello-world" if "hell" is found at the beginning. prefix_removed_string=${string/#$prefix} #remove "ld" from "o-world" if "ld" is found at the end. suffix_removed_String=${prefix_removed_string/%$suffix} echo $suffix_removed_String #> o-wor # NOTES: # `#$prefix` : adding `#` makes sure that substring "hell" is removed only if # it is found in beginning. # `%$suffix` : adding `%` makes sure that substring "ld" is removed only if it # is found in end. # Without these, the substrings "hell" and "ld" will get removed everywhere, # even it is found in the middle. var="apple orange" # this command will print 'apple' echo "${var%% *}" # This command will print 'orange' echo "${var##* }" ``` ## tips & tricks ### Resources [[( 2022-03-16 Five Powerful Tips for Bash Scripting]] #### color output ##### Colorized Output In Bash - Using ANSI escape codes you can make your terminal display colored output | Color | Code | Color | Code | | :----------: | :------------: | :----------: | :------------: | | Black | '\\033\[0;30m' | Dark Gray | '\\033\[1;30m' | | Red | '\\033\[0;31m' | Light Red | '\\033\[1;31m' | | Green | '\\033\[0;32m' | Light Green | '\\033\[1;32m' | | Brown/Orange | '\\033\[0;33m' | Yellow | '\\033\[1;33m' | | Blue | '\\033\[0;34m' | Light Blue | '\\033\[1;34m' | | Purple | '\\033\[0;35m' | Light Purple | '\\033\[1;35m' | | Cyan | '\\033\[0;36m' | Light Cyan | '\\033\[1;36m' | | Light Gray | '\\033\[0;37m' | White | '\\033\[1;37m' | - `RED='\033[0;31m'` - 30-37 sets foreground color - 40-47 sets background color ```bash RED='\033[0;31m' NC='\033[0m' echo -e "${LRED}Hard${NC} [1] ${RED}Difficult${NC} [2] ${YELLOW}Normal${NC} [3] ${GREEN}Mild${NC} [4] ${LGREEN}Easy${NC} [5]" ``` ###### Documentation - [SO answer](https://stackoverflow.com/questions/5947742/how-to-change-the-output-color-of-echo-in-linux#5947802) - [more documentation](https://misc.flogisoft.com/bash/tip_colors_and_formatting) #### create a directory and change into it at the same time ```bash mkd() { mkdir -p "$@" && cd "$@"; } ``` #### create default variable values [[( 2022-03-16 Five Powerful Tips for Bash Scripting]] #### create loading animations ```bash #!/bin/bash while true; do # Frame #1 printf "\r< Loading..." sleep 0.5 # Frame #2 printf "\r> Loading..." sleep 0.5 done ``` or ```bash #!/bin/bash sleep 5 & pid=$! frames="/ | \\ -" while kill -0 $pid 2&>1 > /dev/null; do for frame in $frames; do printf "\r$frame Loading..." sleep 0.5 done done printf "\n" ``` #### delete all files in a folder that don t match a certain file extension ```bash rm !(*.foo|*.bar|*.baz) ``` #### displaying native gui notifications from bash ##### Linux ```bash #!/bin/bash sleep 10 notify-send "notify.sh" "Task #1 was completed successfully" ``` ##### OSX ```bash #!/bin/bash sleep 10 osascript -e "display notification \"Task #1 was completed successfully\" with title \"notify.sh\"" ``` #### dont rely on positional args [[( 2022-03-16 Five Powerful Tips for Bash Scripting]] #### error proof your variables #### globbing vs ls ##### Use globbing instead of ls Instead of using `ls -l <pattern>` to return a wildcard glob, use the wildcard as it is and avoid `ls` ```bash # Documentation pattern="ex*" printf '%s\n' $pattern # not ``ls -1 $pattern'' # > file1.txt # > file_other.txt for file in $pattern; do # definitely, definitely not ``for file in $(ls $pattern)'' printf 'Found file: %s\n' "$file" done # > Found file: file1.txt # > Found file: file_other.txt ``` ###### Documentation - [parsing file globs with ls](https://mywiki.wooledge.org/ParsingLs) - <https://superuser.com/questions/31464/looping-through-ls-results-in-bash-shell-script#31466> #### ignore first n lines ##### Ignore the first N lines > By default, the tail command will show the last n rows, but if you specified the option -n with a number that starts with the + symbol, like +5 , the first 5 lines are going to be skipped. ```bash # In this example, the tail command is going to skip the first 10 lines and print the rest of the file content. tail -n +10 dataset.csv ``` #### multiprocessing in bash scripts ```bash #!/bin/bash function task1() { echo "Running task1..." sleep 5 } function task2() { echo "Running task2..." sleep 5 } task1 & task2 & wait echo "All done!" ``` the `wait` builtin makes sure that all background processes have completed before carrying on #### output as file arg ##### Use the output of another command as a file argument ```bash wc file1 <(echo “hello world”) file2 ``` > When you wrap a command with `<(...)` bash generate a temporal file in a path like `/dev/fd/64`, then execute your wrapped command, put the output in this temporal file, and finally replace `<(...)` with the filename of the temporal file, in this case, `/dev/fd/64` #### pipe stdout and stderr to separate commands ```bash some_command > >(/bin/cmd_for_stdout) 2> >(/bin/cmd_for_stderr) ``` #### quickly truncate a file ```bash >filename ``` #### re run cmds ##### Re-run commands ```bash # of course there's sudo !! # but you can also do !-N # where N is the Nth command (Relative) # or !N # for the N command in your history (Absolute) ``` #### re use cmd args ##### Re-use command arguments ```bash mkdir very-large-directory-name cd very-large-directory-name # Instead of duplicating the argument of the mkdir command, you can use !$ for retrieve the last argument of the last command, the result is: mkdir very-large-directory-name cd !$ # == cd very-large-directory-name ``` #### redirect stout and stderr each to separate files and print both to the screen ```bash (some_command 2>&1 1>&3 | tee errorlog ) 3>&1 1>&2 | tee stdoutlog ``` #### shorten if statements [[( 2022-03-16 Five Powerful Tips for Bash Scripting]] #### stdin as arg ##### Stdin as a file argument ```bash # Command expects a file: wc file1 file2 # Instead of making a temp file to read in a little text use this to pass in # text as a temp file to STDIN wc file1 - file2 # waits for you to type input and you complete this process by using `CTRL+D` which inserts the EOF character ``` #### track content of file ##### Track the content of a log file > See the contents of a file in real time ```bash watch cat log.txt ``` > Although this command does the job, it is not the best option. You can use the tail command with the -f option to track only the new lines that are appended to the file, ## Variables Variables in bash are assigned with a single `=` No spacing between the variable name, the `=` and the assigned value You can specify the variables scope with either `export` or `local` or an environmental variable with no explicit scope. Once declared in your script or environment etc, you can reference your variables by matching the exact casing of the variable name and pre-pending a ` so my path variable for binaries to execute would be `$PATH`. When referencing your variables always quote them because of: "General rule: quote it if it can either be empty or contain spaces" "`$?` doesn't need quotes since it's a numeric value." ```shell ## Variables ### Local vars local var=2 ### Global Vars var=2 ### Environment export var=2 echo "$var" ``` "In short, quote everything where you do not require the shell to perform token splitting and wild card expansion." ```shell ## Token Splitting words="foo bar baz" for word in $words; do echo "$word" done #> foo #> bar #> baz ``` Double quotes are suitable when variable interpolation is required. With suitable adaptations, it is also a good workaround when you need single quotes in the string. (There is no straightforward way to escape a single quote between single quotes, because there is no escape mechanism inside single quotes -- if there was, they would not quote completely verbatim.) No quotes are suitable when you specifically require the shell to perform token splitting and/or wild card expansion. ```shell ## Wildcard Expansion ### Literal Strings pattern='file*.txt' ls $pattern # > file1.txt file_other.txt ### Double Quotes ls "$pattern" #> ls: cannot access file*.txt: No such file or directory # (There is no file named literally file*.txt.) ls '$pattern' #> ls: cannot access $pattern: No such file or directory # (There is no file named $pattern, either!) ``` In more concrete terms, anything containing a filename should usually be quoted (because filenames can contain whitespace and other shell meta characters). Anything containing a URL should usually be quoted (because many URL's contain shell meta characters like `?` and `&`). Anything containing a regex should usually be quoted (ditto ditto). Anything containing significant whitespace other than single spaces between non-whitespace characters needs to be quoted (because otherwise, the shell will munge the whitespace into, effectively, single spaces, and trim any leading or trailing whitespace). When you know that a variable can only contain a value which contains no shell meta characters, quoting is optional. Thus, an unquoted `$?` is basically fine, because this variable can only ever contain a single number. However, `"$?"` is also correct, and recommended for general consistency and correctness (though this is my personal recommendation, not a widely recognized policy). Values which are not variables basically follow the same rules, though you could then also escape any meta characters instead of quoting them. For a common example, a URL with a & in it will be parsed by the shell as a background command unless the meta character is escaped or quoted. ### Meta characters with variables The braces, in addition to delimiting a variable name are used for parameter expansion so you can do things like: Truncate the contents of a variable ```shell var="abcde"; echo ${var%d*} #> abc ``` Make substitutions similar to sed ```shell var="abcde"; echo ${var/de/12} #> abc12 ``` Use a default value ```shell default="hello"; unset var; echo ${var:-$default} #> hello ``` and several more Also, brace expansions create lists of strings which are typically iterated over in loops: ```shell echo f{oo,ee,a}d #> food feed fad mv error.log{,.OLD} # (error.log is renamed to error.log.OLD because the brace expression # expands to "mv error.log error.log.OLD") for num in {000..2}; do echo "$num"; done #> 000 #> 001 #> 002 echo {00..8..2} #> 00 02 04 06 08 echo {D..T..4} #> D H L P T ``` ### Exporting variables Export variables for other programs to use in your shell environment with ```shell export var=myvar ``` variable with ```shell unset myvar ``` Export copies variables to the environment, `declare -x` also does the same as export? Export functions with ```shell export -f myfunc ``` Just printing export will list all current environment variables Functions don’t get a copy of the variables in the environment, they share them and therefor can mutate them To see built-ins use ```shell enable ``` To see keywords use ```shell compgen -k ```