UNIX Shell Script

How to Write Shell Script

INDEX Saranya Wipro Technologies

Page No: 1

How to Write Shell Script

Introduction to Shell

3

Comments

3

Variables

4

Expressions in variables

4

Other forms of input to shell variables or commands in a script Shell commands and control structures

7 8

Quoting

15

Functions

17

Command line arguments

18

Command substitution

19

Script Examples

20

How do you execute (run) a shell script?

23

Debugging

24

Shell Tips & Tricks

25

Reference

32

Introduction Saranya Wipro Technologies

2

How to Write Shell Script

Shell Programming Even though there are various graphical interfaces available for Linux the shell still is a very neat tool. The shell is not just a collection of commands but a really good programming language. You can automate a lot of tasks with it, the shell is very good for system administration tasks, you can very quickly try out if your ideas work which makes it very useful for simple prototyping and it is very useful for small utilities that perform some relatively simple tasks where efficiency is less important than ease of configuration, maintenance and portability. So let's see now how it works: Creating a script There are a lot of different shells available for Linux but usually the bash (bourne again shell) is used for shell programming as it is available for free and is easy to use. So all the scripts we will write in this article use the bash (but will most of the time also run with its older sister, the bourne shell). For writing our shell programs we use any kind of text editor, e.g. nedit, kedit, emacs, vi...as with other programming languages. The program must start with the following line (it must be the first line in the file): #!/bin/sh

The #! characters tell the system that the first argument that follows on the line is the program to be used to execute this file. In this case /bin/sh is shell we use. When you have written your script and saved it you have to make it executable to be able to use it. To make a script executable type chmod +x filename Then you can start your script by typing: ./filename

Comments Comments in shell programming start with # and go until the end of the line. We really recommend you to use comments. If you have comments and you don't use a certain script for some time you will still know immediately what it is doing and how it works.

Saranya Wipro Technologies

3

How to Write Shell Script

Variables As in other programming languages you can't live without variables. In shell programming all variables have the datatype string and you do not need to declare them. To assign a value to a variable you write: varname=value

To get the value back you just put a dollar sign in front of the variable: #!/bin/sh # assign a value: a="hello world" # now print the content of "a": echo "A is:" echo $a

Type this lines into your text editor and save it e.g. as first. Then make the script executable by typing chmod +x first in the shell and then start it by typing ./first The script will just print: A is: hello world

Sometimes it is possible to confuse variable names with the rest of the text: num=2 echo "this is the $numnd"

This will not print "this is the 2nd" but "this is the " because the shell searches for a variable called numnd which has no value. To tell the shell that we mean the variable num we have to use curly braces: num=2 echo "this is the ${num}nd"

This prints what you want: this is the 2nd There are a number of variables that are always automatically set. We will discuss them further down when we use them the first time. If you need to handle mathematical expressions then you need to use programs such as expr (see table below). Besides the normal shell variables that are only valid within the shell program there are also environment variables. A variable preceeded by the keyword export is an environment variable. We will not talk about them here any further since they are normally only used in login scripts.

Expressions involving variables Expressions can be used in assigning values to new variables; as substitutions in command lines in the script; and in flow-of-control statements: if, foreach, while, and switch.

Saranya Wipro Technologies

4

How to Write Shell Script Expressions are always enclosed in a pair of parentheses ( and ). Note that you should put blanks (white space) around the parentheses and the operators within them to insure proper parsing. •

Arithmetic expressions

Standard arithmetic operators (+ - * /) can be used to operate on integer variable values. To save the results in another variable, however, you should use the @ command rather than the set command for the new variable. For example, you can increment a sum with the value of an argument like this: @ sum = 0 # initialize @ sum = ( $sum + $1 ) Note that there must be a space between the @ command and the name of the variable that it is setting. The C-shell cannot do floating point (decimal) arithmetic, for example, 1.1 * 2.3. However, you can invoke the bc calculator program from within a shell script to perform decimal arithmetic. Probably the simplest way to do that is to define an alias, called for example, "MATH", that performs decimal arithmetic via the bc program on the variables or constants passed to the alias. For example, you can define this alias in your script # Set MATH alias - takes an arithmetic assignment statement # as argument, e.g., newvar = var1 + var2 # Separate all items and operators in the expression with blanks alias MATH 'set \!:1 = `echo "\!:3-$" | bc -l`' This says that the word "MATH" will be replaced by a set command which will set the variable given as the first word following the alias equal to the result of a math calculation specified by the remainder of the words following the alias, starting with the third word (skips the = sign), which is piped to the bc program to perform. For example, you can use this MATH alias in a shell script to multiply two floating point values like this: set width = 20.8 set height = 5.4 MATH area = $width * $height echo $area Note to experts: normally, you would quote the asterisk character in a command line like the one shown above, so the shell does not try to interpret it as the filename wildcard matching character. But the alias expansion is done first, before filename matching, and as a result, the asterisk becomes protected by the quotes in the alias. •

Logical expressions

Saranya Wipro Technologies

5

How to Write Shell Script These are usually used in if statements for conditional execution. Logical expressions always evaluate to either the value 1 if true, or value 0 if false. The result of a logical expression can also be assigned to another variable with the @ operator. These expressions test whether a variable equals another, or a constant value, or is greater than or less than another value. There are some special advanced tests, but the basic ones are indicated by the following operators, with a variable or constant on either side. To get a variable's value, you must, of course, use the $ operator. Character constants should be enclosed in a pair of quote characters. Use either single (') or double (") quotes, as long as you use the same type on both ends of the constant. If the variable being tested may contain blanks or other special shell characters as part of its value, then it must also be enclosed in quotes, but only double quotes can be used for variables, e.g., "$a". ( $a < $b ) Tests if the numeric value of variable a is less than the numeric value of variable b. You will get an error message if either variable does not contain a numeric string. ( $a $b ) "Greater than" works the same as "less than". There is also a "greater than or equal" test, using the operator >= Again, you will get an error message if either variable does not contain a numeric string. ( $a == 5 ) Tests if the numeric value of variable a is exactly equal to numeric value 5. Again, you will get an error message if the variable a does not contain a numeric string. ( "$c" == "yes" ) Tests if the string value of variable c is exactly equal to the character string constant yes. Logical expressions can be modified by the "NOT" (!) operator or combined with the "OR" (||) or "AND" (&&) operators. Examples: ( ! "$c" == "yes" ) The logical NOT operator -- the exclamation point -- can be put at the front of any expression (inside the parentheses) to reverse the result of the expression. That is, whenever the expression would normally give a "true" result, it now gives a "false" result, and vice-versa. In this example, the overall expression is true whenever the value of the variable c is not equal to the constant string "yes". ( $a > 1 && $a < 5 ) Tests if the value of variable a is greater than numeric value 1 and also less than numeric value 5. Both cases have to be true for the expression to evaluate "true". ( $c == "yes" || $c == "YES" ) Tests if value of variable c is either the character string constant yes or the string constant YES (remember that upper and lower case are not the same). If it is equal to either one, then the expression evaluates "true". Saranya Wipro Technologies

6

How to Write Shell Script •

Built-in logical tests for file existence

The C-shell has built in tests for existence or characteristics of a file. These tests are made with a special form of a logical expression in which a variable substitution or constant string is preceded by a one letter operator introduced with the minus sign. For example, the expression ( -f "junk" ) evaluates to "true" if there is an existing plain file (not a directory or special file) in the current directory with the name junk. Similarly, if the first shell script argument was the string subdir, then the expression ( -d $1 ) evaluates to "true" if subdir is the name of a existing directory (not a plain file) in the current directory. The complete list of these operators, and what they test for, is: -e

tests if a file exists (it can be any type of file)

-f

tests for an existing plain file

-d

tests for an existing directory

-z

tests for an existing plain file of zero size (that is, an empty file)

-r

tests if you have read access to a file

-w

tests if you have write access to a file

-x

tests if you have execute access to a file

-o

tests if you own a file

•

Pattern matching expressions

A more advanced feature allows you to test if a variable value matches a pattern instead of just whether it is an exact match to a character string. Read about this in the C-shell reference manual.

Other forms of input to shell variables or commands in a script You can execute a command from the shell script and have it read the next few lines in the script as the command's standard input. You do this by redirecting the standard input with the special symbol newfile newtext.file This replaces the first occurance of the string linuxfocus in each line with LinuxFocus. If there are lines where linuxfocus appears several times and you want to replace all use: cat text.file | sed 's/linuxfocus/LinuxFocus/g' > newtext.file Most of the time awk is used to extract fields from a text line. 9

How to Write Shell Script The default field separator is space. To specify a different one use the option -F. cat file.txt | awk -F, '{print $1 "," $3 }'

Here we use the comma (,) as field separator and print the first and third ($1 $3) columns. If file.txt has lines like: Adam Bor, 34, India Kerry Miller, 22, USA

then this will produce: Adam Bor, India Kerry Miller, USA

There is much more you can do with awk but this is a very common use.

2) Concepts: Pipes, redirection and backtick: They are not really commands but they are very important concepts. Pipes : (|) send the output (stdout) of one program to the input (stdin) of another

program. grep "hello" file.txt | wc -l

Finds the lines with the string hello in file.txt and then counts the lines. The output of the grep command is used as input for the wc command. You can concatinate as many commands as you like in that way (within reasonable limits). Redirection : writes the output of a command to a file or appends data to a file

> writes output to a file and overwrites the old file in case it exists >> appends data to a file (or creates a new one if it doesn't exist already but it never overwrites anything). Backtick :

The output of a command can be used as command line arguments (not stdin as above, command line arguments are any strings that you specify behind the command such as file names and options) for another command. You can as well use it to assign the output of a command to a variable. The command find . -mtime -1 -type f -print

finds all files that have been modified within the last 24 hours (-mtime -2 would be 48 hours). If you want to pack all these files into a tar archive (file.tar) the syntax for tar would be: tar xvf file.tar infile1 infile2 ...

Instead of typing it all in you can combine the two commands (find and tar) using backticks. Tar will then pack all the files that find has printed: #!/bin/sh # The ticks are backticks (`) not normal quotes ('): tar -zcvf lastmod.tar.gz `find . -mtime -1 -type f -print`

Saranya Wipro Technologies

10

How to Write Shell Script

3) Control structures :

The "if" statement tests if the condition is true (exit status is 0, success). If it is the "then" part gets executed: if ....; then .... elif ....; then .... else .... fi

Most of the time a very special command called test is used inside if-statements. It can be used to compare strings or test if a file exists, is readable etc... The "test" command is written as square brackets " [ ] ". Note that space is significant here: Make sure that you always have space around the brackets. Examples: [ [ [ [

-f "somefile" ] -x "/bin/ls" ] -n "$var" ] "$a" = "$b" ]

: : : :

Test Test Test Test

if if if if

somefile is a file. /bin/ls exists and is executable. the variable $var contains something the variables "$a" and "$b" are equal

Run the command "man test" and you get a long list of all kinds of test operators for comparisons and files. Using this in a shell script is straight forward: #!/bin/sh if [ "$SHELL" = "/bin/bash" ]; then echo "your login shell is the bash (bourne again shell)" else echo "your login shell is not bash but $SHELL" fi

The variable $SHELL contains the name of the login shell and this is what we are testing here by comparing it against the string "/bin/bash" 4)Shortcut operators :

People familiar with C will welcome the following expression: [ -f "/etc/shadow" ] && echo "This computer uses shadow passwors"

The && can be used as a short if-statement. The right side gets executed if the left is true. You can read this as AND. Thus the example is: "The file /etc/shadow exists AND the command echo is executed". The OR operator (||) is available as well. Here is an example: #!/bin/sh mailfolder=/var/spool/mail/james [ -r "$mailfolder" ] || { echo "Can not read $mailfolder" ; exit 1; } echo "$mailfolder has mail from:" grep "^From " $mailfolder

The script tests first if it can read a given mailfolder. If yes then it prints the "From" lines in the folder. If it cannot read the file $mailfolder then the OR operator takes effect. In plain English you read this code as "Mailfolder readable or exit program". The problem here is that you must have exactly one command behind the OR but we need two: -print an error message Saranya Wipro Technologies

11

How to Write Shell Script -exit the program To handle them as one command we can group them together in an anonymous function using curly braces. Functions in general are explained further down. You can do everything without the ANDs and ORs using just if-statements but sometimes the shortcuts AND and OR are just more convenient. The case statement can be used to match (using shell wildcards such as * and ?) a given string against a number of possibilities. case ... in ...) do something here;; esac

Let's look at an example. The command file can test what kind of file type a given file is: file lf.gz returns: lf.gz: gzip compressed data, deflated, original filename, last modified: Mon Aug 27 23:09:18 2001, os: Unix

We use this now to write a script called smartzip that can uncompress bzip2, gzip and zip compressed files automatically : #!/bin/sh ftype=`file "$1"` case "$ftype" in "$1: Zip archive"*) unzip "$1" ;; "$1: gzip compressed"*) gunzip "$1" ;; "$1: bzip2 compressed"*) bunzip2 "$1" ;; *) error "File $1 can not be uncompressed with smartzip";; esac

Here you notice that we use a new special variable called $1. This variable contains the first argument given to a program. Say we run smartzip articles.zip then $1 will contain the string articles.zip The select statement is a bash specific extension and is very good for interactive use. The user can select a choice from a list of different values: select var in ... ; do break done .... now $var can be used ....

Here is an example: #!/bin/sh echo "What is your favourite OS?" select var in "Linux" "Gnu Hurd" "Free BSD" "Other"; do break done echo "You have selected $var"

Here is what the script does: What is your favourite OS?

Saranya Wipro Technologies

12

How to Write Shell Script 1) Linux 2) Gnu Hurd 3) Free BSD 4) Other #? 1 You have selected Linux

In the shell you have the following loop statements available: while ...; do .... done

The while-loop will run while the expression that we test for is true. The keyword "break" can be used to leave the loop at any point in time. With the keyword "continue" the loop continues with the next iteration and skips the rest of the loop body. The for-loop takes a list of strings (strings separated by space) and assigns them to a variable: for var in ....; do .... done

The following will e.g. print the letters A to C on the screen: #!/bin/sh for var in A B C ; do echo "var is $var" done

A more useful example script, called showrpm, prints a summary of the content of a number of RPM-packages: #!/bin/sh # list a content summary of a number of RPM packages # USAGE: showrpm rpmfile1 rpmfile2 ... # EXAMPLE: showrpm /cdrom/RedHat/RPMS/*.rpm for rpmpackage in $*; do if [ -r "$rpmpackage" ];then echo "=============== $rpmpackage ==============" rpm -qi -p $rpmpackage else echo "ERROR: cannot read file $rpmpackage" fi done

Above you can see the next special variable, $* which contains all the command line arguments. If you run showrpm openssh.rpm w3m.rpm webgrep.rpm then $* contains the 3 strings openssh.rpm, w3m.rpm and webgrep.rpm. The GNU bash knows until-loops as well but generally while and for loops are sufficient. 5)Flow-of-control statements

if and foreach are the basic flow-of-control statements. There are more advanced ones named switch and while which are similar to the statements of the same name in the C language. Saranya Wipro Technologies

13

How to Write Shell Script if The if command allows you to conditionally execute a command or set of commands depending upon whether some expression is true. There are two forms. •

if ( logical_expression ) command ...

This form will execute command (which can have a long list of arguments) if the logical__expression is true. This expression can be one of the logical or file testing expressions described above. For example, you can test if a file whose name is stored in the shell's built-in variable $1 (first argument to the shell script) exists as a plain file, and if so, make a backup copy of it with: if ( -f $1 ) cp $1 $1.bak This simple cp command will not work if given a directory to copy, which is why there is the test for a "plain" file. •

if ( logical_expression ) then block of commands - any number of lines to be executed if logical_expression is "true" (or has non-zero value).

else another block of commands - any number of lines to be executed if logical_expression is "false" (or has value 0). endif This form allows you to execute more than one command if the expression is true. The then keyword must follow the logical_expression test on the same line, and the endif keyword must be on a line by itself to end the entire if command. The else statement is optional. If you use this, the else keyword must be on a line by itself. The following lines up to the endif are executed if the expression was false. The "blocks of commands" may in turn contain additional nested if commands. Just be sure that each if has a matching endif statement enclosed in the same block. •

foreach

This statement allows you to execute a loop, like a do statement in Fortran or a for statement in C. The syntax is: Saranya Wipro Technologies

14

How to Write Shell Script foreach name (wordlist) block of statements to be executed end foreach and end are keywords. The end statement must be on a line by itself. name is the name of a variable that you create. This is the "loop index". (wordlist) is a list of "words", meaning any character strings separated by blanks. The parentheses are required. The "word list" can be a list of actual constant values, or the results of variable substitution, arithmetic expressions, or command substitution. When the foreach statement is encountered in a shell script, the wordlist is evaluated (necessary variable and command substitutions and expressions are done) and stored. Then the new variable name is set equal to the first word in the list and the block of statements is executed. When the end statement is reached, the script goes back to the foreach line and sets the variable name equal to the next word in the list and executes the block of statements again. This is done over and over until all words in the wordlist have been used up. This type of looping statement is good for repetitively executing the same commands for a set of arguments. For example, you could check all the arguments given to the shell to see if they are plain files, and if so, make backup copies, using these statements in a shell script: foreach file ( $* ) if (-f $file) cp $file $file.bak end The foreach command can also be used interactively from the terminal. In this form, you will get question mark prompts (?) to enter your commands. When done, type end after one of these prompts. For example, you could type the following commands interactively at your terminal to make backup copies of all Fortran programs in your current directory: foreach file ( *.f ) ? if (-f $file) cp $file $file.bak ? end

Quoting : Before passing any arguments to a program the shell tries to expand wildcards and variables. To expand means that the wildcard (e.g. *) is replaced by the appropriate file names or that a variable is replaced by its value. To change this behaviour you can use quotes: Let's say we have a number of files in the current directory. Two of them are jpgfiles, mail.jpg and tux.jpg. #!/bin/sh echo *.jpg

This will print "mail.jpg tux.jpg". Quotes (single and double) will prevent this wildcard expansion: Saranya Wipro Technologies

15

How to Write Shell Script #!/bin/sh echo "*.jpg" echo '*.jpg'

This will print "*.jpg" twice. Single quotes are most strict. They prevent even variable expansion. Double quotes prevent wildcard expansion but allow variable expansion: #!/bin/sh echo $SHELL echo "$SHELL" echo '$SHELL'

This will print: /bin/bash /bin/bash $SHELL

Finally there is the possibility to take the special meaning of any single character away by preceeding it with a backslash: echo \*.jpg echo \$SHELL

This will print: *.jpg $SHELL

Here documents Here documents are a nice way to send several lines of text to a command. It is quite useful to write a help text in a script without having to put echo in front of each line. A "Here document" starts with