Bourne Shell Tutorial by Steve Parker ( www.steve-parker.org ) - reformatted.
                 1.INTRO
                 2.PHILOSOPHY
                 3.A FIRST SCRIPT
                 4.VARIABLES - Part I
                 5.WILDCARDS
                 6.ESCAPE CHARACTERS
                 7.LOOPS
                 8.TEST
                 9.CASE
                10.VARIABLES - Part II
                11.EXTERNAL PROGRAMS
                12.FUNCTIONS
                13.HINTS AND TIPS
                14.CGI SCRIPTING

 

INTRO

home - top of the page - email

The most recent version of this tutorial is available from: http://www.steve-parker.org/sh/sh.shtml.
The Bourne shell was written by Steve Bourne - and it appeared in the Seventh Edition Bell Labs Research version of Unix.

Let's get on unix prompt and create a simple 2-line script - store it in a file  my-script.sh - and run it. Here is what we do (and see) on unix prompt:

$ echo "#!/bin/sh" > my-script.sh $ echo "echo Hello World" >> my-script.sh $ chmod 755 my-script.sh $ ./my-script.sh Hello World $

Here is how the script may look with comments added:

#!/bin/sh # This is a comment! echo Hello World        # This is a comment, too!

 

Philosophy

home - top of the page - email

Philosophy

  There are a number of factors which can go into good, clean, quick, shell scripts.
  The most important criteria must be a clear, readable layout.
  Second is avoiding unnecessary commands.
  A clear layout makes the difference between a shell script appearing as "black magic" and one which is easily maintained and understood.
  You may be forgiven for thinking that with a simple script, this is not too significant a problem, but two things here are worth bearing in mind.
  First, a simple script will, more often than anticipated, grow into a large, complex one.
  Secondly, if nobody else can understand how it works, you will be lumbered with maintaining it yourself for the rest of your life!
  One of the major weaknesses in many shell scripts is lines such as:
            cat /tmp/myfile | grep "mystring"
  which would run much faster as:
            grep "mystring" /tmp/myfile
  Not much, you may consider; the OS has to load up the /bin/grep executable, which is a reasonably small 75600 bytes on my system, open a pipe in memory for the transfer, load and run the /bin/cat executable, which is an even smaller 9528 bytes on my system, attach it to the input of the pipe, and let it run.
  Of course, this kind of thing is what the OS is there for, and it's normally pretty efficient at doing it. But if this command were in a loop being run many times over, the saving of not locating and loading the cat executable, setting up and releasing the pipe, can make some difference, especially in, say, a CGI environment where there are enough other factors to slow things down without the script itself being too much of a hurdle.
  As a result of this, you may hear mention of The Award For The Most Gratuitous Use Of The Word Cat In A Serious Shell Script being bandied about on the comp.os.unix.shell newsgroup from time to time. This is purely a way of peers keeping each other in check, and making sure that things are done right.
  Speaking of which, I would like to reccommend the comp.os.unix.shell newsgroup to you, although its signal to noise ratio seems to have increased in recent years. There are still some real gurus who hang out there with good advice for those of us who need to know more (and that's all of us!). Sharing experiences is the key to all of this - the reason behind this tutorial itself, and we can all learn from and contribute to open discussions about such issues.
  Which leads me nicely on to something else: Don't ever feel too close to your own shell scripts; by their nature, the source cannot be closed. If you supply a customer with a shell script, s/he can inspect it quite easily. So you might as well accept that it will be inspected by anyone you pass it to; use this to your advantage with the GPL - encourage people to give you feedback and bugfixes for free!

 

A FIRST SCRIPT

home - top of the page - email

A First Script
For our first shell script, we'll just write a script which says "Hello World". We will then try to get more out of a Hello World program than any other tutorial you've ever read :-)
Create a file (first.sh) as follows:

The second line begins with a special symbol: #. This marks the line as a comment, and it is ignored completely by the shell.
The only exception is when the very first line of the file starts with #! - as ours does. This is a special directive which Unix treats specially. It means that even if you are using csh, ksh, or anything else as your interactive shell, that what follows should be interpreted by the Bourne shell.

The third line runs a command: echo, with two parameters, or arguments - the first is "Hello"; the second is "World".
Note that echo will automatically put a single space between its parameters.
The # symbol still marks a comment; the # and anything following it is ignored by the shell.

now run chmod 755 first.sh to make the text file executable, and run ./first.sh.
Your screen should then look like this:

$ chmod 755 first.sh $ ./first.sh Hello World $

You will probably have expected that! You could even just run:

$ echo Hello World Hello World $

Now let's make a few changes.
First, note that echo puts ONE space between its parameters. Put a few spaces between "Hello" and "World". What do you expect the output to be? What about putting a TAB character between them?
As always with shell programming, try it and see.
The output is exactly the same! We are calling the echo program with two arguments; it doesn't care any more than cp does about the gaps in between them.
Now modify the code again:

#!/bin/sh # This is a comment! echo "Hello      World"       # This is a comment, too!

This time it works. You probably expected that, too, if you have experience of other programming languages. But the key to understanding what is going on with more complex command and shell script, is to understand and be able to explain: WHY?
echo has now been called with just ONE argument - the string "Hello    World". It prints this out exactly.
The point to understand here is that the shell parses the arguments BEFORE passing them on to the program being called. In this case, it strips the quotes but passes the string as one argument.
As a final example, type in the following script. Try to predict the outcome before you run it:

#!/bin/sh # This is a comment! echo "Hello      World"       # This is a comment, too! echo "Hello World" echo "Hello * World" echo Hello * World echo Hello      World echo "Hello" World echo Hello "     " World echo "Hello \"*\" World" echo `hello` world echo 'hello' world

Is everything as you expected? If not, don't worry! These are just some of the things we will be covering in this tutorial ... and yes, we will be using more powerful commands than echo!

 

VARIABLES - Part I

home - top of the page - email

Variables - Part I
Just about every programming language in existance has the concept of variables - a symbolic name for a chunk of memory to which we can assign values, read and manipulate its contents. The bourne shell is no exception, and this section introduces idea. This is taken further in Variables - Part II which looks into variables which are set for us by the environment.
Let's look back at our first Hello World example. This could be done using variables (though it's such a simple example that it doesn't really warrant it!)
Enter the following code into var1.sh:

#!/bin/sh MY_MESSAGE="Hello World" echo $MY_MESSAGE

This assigns the string "Hello World" to the variable MY_MESSAGE then echoes out the value of the variable.
Note that we need the quotes around the string Hello World. Whereas we could get away with echo Hello World because echo will take any number of parameters, a variable can only hold one value, so a string with spaces must be quoted to that the shell knows to treat it all as one.

The shell does not care about types of variables; they may store strings, integers, real numbers - anything you like. In truth, these are all stored as strings, but routines which expect a number can treat them as such. Of course, if you assign a string to a variable then try to add   1   to it, the results may be surprising! But there is no syntactic difference between:
MY_MESSAGE="Hello World"
MY_NUMBER=1
MY_OTHER_NUMBER=3.142
MY_OTHER_NUMBER="3.142"
MY_SOMETHING=123abc
Note though that special characters must be properly escaped to avoid interpretation by the shell. This is discussed further in Escape Characters.

We can interactively set variable names using the read command; the following script asks you for your name then greets you personally:

#!/bin/sh read USER_NAME echo Hello there, $USER_NAME - how are you today?

This is using the shell-builtin command read which reads a line from standard input into the variable supplied.

Note that even if you give it your full name and don't use double quotes around the echo command, it still outputs correctly. How is this done? With the MY_MESSAGE variable earlier we had to put double quotes around it to set it.
What happens, is that the read command automatically places quotes around its input, so that spaces are treated correctly.

Scope of Variables: - Variables in the bourne shell do not have to be declared, as they do in languages like C. But if you try to read an undeclared varable, the result is the empty string.

There is a command called export which has a fundamental effect on the scope of variables. In order to really know what's going on with your variables, you will need to understand something about how this is used.

Create a small shell script, myvar.sh:

#!/bin/sh echo "MYVAR is: $MYVAR" MYVAR="hi there" echo "MYVAR is: $MYVAR"

Now run the script:
$ ./myvar.sh
MYVAR is:
MYVAR is: hi there

MYVAR hasn't been set to any value, so it's blank.
Now run:
$ MYVAR=hello
$ ./myvar.sh
MYVAR is:
MYVAR is: hi there

It's still not been set! What's going on?!

When you call myvar.sh from your interactive shell, a new shell is spawned to run the script. We need to export the variable for it to be inherited by the shell script itself. Type:
$ export MYVAR
$ ./myvar.sh
MYVAR is: hello
MYVAR is: hi there
Now look at line 3 of the script: this is changing the value of MYVAR. But there is no way that this will be passed back to your interactive shell. Try reading the value of MYVAR:
$ echo $MYVAR
hello
$
Once the shell script exits, its environent is destroyed. But MYVAR keeps its value of hello within your interactive shell.
In order to receive environment changes back from the script, we must source the script - this effectively runs the script within our own interactive shell, instead of spawning another shell to run it.
We can source a script via the "." command:
$ MYVAR=hello
$ export MYVAR
$ echo MYVAR
hello
$ . ./myvar.sh
MYVAR is: hello
MYVAR is: hi there
$ echo $MYVAR
hi there

The change has now made it out into our shell again! This is how your .profile file works, for example.

One other thing worth mentioning at this point about variables, is to consider the following shell script:

#!/bin/sh echo "What is your name?" read USER_NAME echo "Hello $USER_NAME" echo "I will create you a file called $USER_NAME_file" touch $USER_NAME_file

This will cause an error unless there is a variable called USER_NAME_file. The shell does not know where the variable begins and the rest starts. How can we define this?
The answer is, that we enclose the variable itself in curly brackets:

#!/bin/sh echo "What is your name?" read USER_NAME echo "Hello $USER_NAME" echo "I will create you a file called ${USER_NAME}_file" touch ${USER_NAME}_file

The shell now knows that we are referring to the variable USER_NAME and that we want it suffixed with _file. This can be the downfall of many a new shell script programmer, as the source of the problem can be difficult to track down.

 

WILDCARDS

home - top of the page - email

Wildcards

Wildcards are really nothing new if you have used Unix at all before. It is not necessarily obvious how they are useful in shell scripts though. This section is really just to get the old grey cells thinking how things look when you're in a shell script - predicting what the effect of using different syntaxes are. This will be used later on, particularly in the Loops section.

Think first how you would copy all the files from /tmp/a  into  /tmp/b. All the .txt files? All the .html files?
Hopefully you will have come up with:
$ cp /tmp/a/* /tmp/b/
$ cp /tmp/a/*.txt /tmp/b/
$ cp /tmp/a/*.html /tmp/b/

Now how would you list the files in /tmp/a/ without using ls /tmp/a/?
How about echo /tmp/a/*? What are the two key differences between this and the ls output? How can this be useful? Or a hinderance?
How could you rename all .txt files to .bak? Note that
$ mv *.txt *.bak
will not have the desired effect; think about how this gets expanded by the shell before it is passed to mv.
We will look into this further later on, as it uses a few concepts not yet covered.

 

ESCAPE CHARACTERS

home - top of the page - email

Escape Characters

Certain characters are significant to the shell; we have seen, for example, that the use of double quotes (") characters affect how spaces and TAB characters are treated, for example:
$ echo Hello       World
Hello World
$ echo "Hello       World"
Hello     World

So how do we display: Hello    "World" ?
$ echo "Hello   \"World\""

The first and last " characters wrap the whole lot into one parameter passed to echo so that the spacing between the two words is kept as is. But the code:
$ echo "Hello   "World""
would be interpreted as three parameters:
"Hello   "
World
""
So the output would be
Hello    World

Note that we lose the quotes entirely. The first and second quotes mark off the Hello and following spaces; the second argument is an unquoted "World" and the third argument is the empty string; "".

Most characters (*, ', etc) are not interpreted (ie, they are taken literally) by means of placing them in double quotes ("). They are taken as is and passed on the the command being called.
However, ", $, `, and \ are still interpreted by the shell, even when they're in double quotes.
The backslash (\) character is used to mark these special characters so that they are not interpreted by the shell, but passed on to the command being run (for example, echo).
So to output the string: "A quote is ", backslash is \, backtick is `, a few spaces are     and dollar is $", we would have to write:
$ echo "A quote is \", backslash is \\, backtick is \`, a few spaces are    and dollar is \$"
A quote is ", backslash is \, backtick is `, a few spaces are    and dollar is $

We have seen why the " is special for preserving spacing. Dollar is special because it marks a variable, so $MY_VAR is replaced by the shell with the contents of the variable MY_VAR. Backslash is special because it is itself used to mark other characters off; we need the following options for a complete shell:
$ echo "This is \\ a backslash"
This is \ a backslash
$ echo "This is \" a quote and this is \\ a backslash"
This is " a quote and this is \ a backslash

So backslash itself must be escaped to show that it is to be taken literally. The other special character, the backtick, is discussed later in External Programs.

 

LOOPS

home - top of the page - email

Loops

Most languages have the concept of loops: If we want to repeat a task twenty times, we don't want to have to type in the code twenty times, with maybe a slight change each time.

As a result, we have FOR and WHILE loops in the Bourne shell.

for Loops - for loops iterate through a set of values until the list is exhausted:

#!/bin/sh for i in 1 2 3 4 5 do   echo "Looping ... number $i" done

Try this code and see what it does.
Note that the values can be anything at all:

Another code is well worth trying:

#!/bin/sh for i in hello 1 * 2 goodbye do   echo "Looping ... i is set to $i" done

Make sure that you understand what is happening here. Try it without the * and grasp the idea, then re-read the Wildcards section and try it again with the * in place. Try it also with the * surrounded by double quotes, and try it preceded by a backslash (\*)

While Loops - while loops can be much more fun! (depending on your idea of fun...)

#!/bin/sh INPUT_STRING=hello while [ "$INPUT_STRING" != "bye" ] do   echo "Please type something in (bye to quit)"   read INPUT_STRING   echo "You typed: $INPUT_STRING" done

What happens here, is that the echo and read statements will run indefinitely until you type "bye" when prompted.

The colon ( : ) always evaluates to true; whilst using this can be necessary sometimes, it is preferrable to use a real exit condition. Compare quitting the above loop with the one below; see which is the more elegant:

#!/bin/sh while : do   echo "Please type something in (^C to quit)"   read INPUT_STRING   echo "You typed: $INPUT_STRING" done

Another useful trick is the while  read   f   loop.

#!/bin/sh while read f do   case $f in         hello)          echo English    ;;         howdy)          echo American   ;;         gday)           echo Australian ;;         bonjour)        echo French     ;;         "guten tag")    echo German     ;;         *)              echo Unknown Language: $i               ;;    esac done < myfile.txt

Review Variables - Part I to see why we set INPUT_STRING=hello before testing it. This makes it a repeat loop, not a traditional while loop.
We will use while loops further in the Test and Case sections.

 

TEST

home - top of the page - email

Test

Test is used by virtually every shell script written. It is a simple but powerful comparison utility. For full details, run man test on your system, but here are some usages and typical examples.

Test is most often invoked indirectly via the  if  and  while  statements. It is also the reason you will come into difficulties if you create a program called test and try to run it, as this shell builtin will be called instead of your program!

The syntax for if...then...else... is:

if [ ... ] then   # if-code else   # else-code fi

Note that fi is if backwards! This is used again later with case and esac.

Taking the following code snippet:

if [ "$X" -lt "0" ] then   echo "X is less than zero" fi if [ "$X" -gt "0" ]; then   echo "X is more than than zero" fi [ "$X" -le "0" ] && echo "X is less than or equal to  zero" [ "$X" -ge "0" ] && echo "X is more than or equal to zero" [ "$X" = "0" ] && echo "X is the string or number \"0\"" [ "$X" = "hello" ] && echo "X is \"hello\"" [ "$X" != "hello" ] && echo "X is not the string \"hello\"" [ -n "$X" ] && echo "X is of nonzero length" [ -f "$X" ] && echo "X is the path of a real file" || echo "Error: No such file: $X" [ -x "$X" ] && echo "X is the path of an executable file" [ "$X" -nt "$Y" ] && echo "X is a file which is newer than Y"

Note that we can use the semicolon ( ; ) to join two lines together. This is often done to save a bit of space in simple if statements.

As we see from these examples, test can perform many tests on numbers, strings, and filenames.

There is a simpler way of writing if statements: The && and || commands give code to run if the result is true.

#!/bin/sh [ $X -ne 0 ] && echo "X isn't zero" || echo "X is zero" [ -f $X ] && echo "X is a file" || echo "X is not a file" [ -n $X ] && echo "X is of non-zero length" || echo "X is of zero length"

This syntax is possible because there is a file (or shell-builtin) called   [   which is linked to test. Be careful using this construct, though, as overuse can lead to very hard-to-read code. The if...then...else... structure is much more readable. Use of the [...] construct is recommended for while loops and trivial sanity checks with which you do not want to overly distract the reader.

We can use test in while loops and the like as follows:

#!/bin/sh X=0 while [ -n $X ] do   echo "Enter some text (RETURN to quit)"   read X   echo "You said: $X" done

This code will keep asking for input until you hit RETURN (X is zero length).

 

CASE

home - top of the page - email

Case

The case statement saves going through a whole set of if .. then .. else statements. Its syntax is really quite simple:

#!/bin/sh echo "Please talk to me ..." while : do   read INPUT_STRING   case $INPUT_STRING in         hello)                 echo "Hello yourself!"                 ;;         bye)                 echo "See you again!"                 break                 ;;         *)                 echo "Sorry, I don't understand"                 ;;   esac done echo echo "That's all folks!"

Okay, so it's not the best converstionalist in the world; it's only an example!

Try running it and check how it works...
$ ./case.sh
Please talk to me ...
hello
Hello yourself!
what do you think of politics?
Sorry, I don't understand
bye
See you again!

That's all folks!
$

The syntax is quite simple:
The case line itself is always of the same format, and it means that we are testing the value of the variable INPUT_STRING.

The options we understand are then listed and followed by a right bracket, as hello) and bye).
This means that if INPUT_STRING matches hello then that section of code is executed, up to the double semicolon.
If INPUT_STRING matches bye then the goodbye message is printed and the loop exits. Note that if we wanted to exit the script completely then we would use the command exit instead of break.
The third option here, the *), is the default catch-all condition; it is not required, but is often useful for debugging purposes even if we think we know what values the test variable will have.

The whole case statement is ended with esac (case backwards!) then we end the while loop with a <done>.

That's about as complicated as case conditions get, but they can be a very useful and powerful tool. They are often used to parse the parameters passed to a shell script, amongst other uses.

 

VARIABLES - Part II

home - top of the page - email

Variables - Part II
There are a set of variables which are set for you already, and most of these cannot have values assigned to them.
These can contain useful information, which can be used by the script to know about the environment in which it is running.

The first set of variables we will look at are $0 .. $9 and $#.
The variable $0 is the basename of the program as it was called.
$1 .. $9 are the first 9 additional parameters the script was called with.
The variable $@ is all parameters $1 .. whatever.
$# is the number of parameters the script was called with.
Let's take an example script:

#!/bin/sh echo "I was called with $# parameters" echo "My name is $0" echo "My first parameter is $1" echo "My second parameter is $2" echo "All parameters are $@"

Let's look at running this code and see the output:
$ /home/steve/var2.sh
I was called with 0 parameters
My name is /home/steve/var2.sh
My first parameter is
My second parameter is
All parameters are
$
$ ./var2.sh hello world earth
I was called with 3 parameters
My name is ./var2.sh
My first parameter is hello
My second parameter is world
All parameters are hello world earth

Note that the value of $0 changes depending on how the script was called. $# and $1 .. $2 are set automatically by the shell.

We can take more than 9 parameters by using the shift command; look at the script below:

#!/bin/sh while [ "$#" -gt "0" ] do   echo "\$1 is $1"   shift done

This script keeps on using shift until $# is down to zero, at which point the list is empty.

Another special variable is $?. This contains the exit value of the last run command. So the code:

#!/bin/sh /usr/local/bin/my-command if [ "$?" -ne "0" ]; then   echo "Sorry, we had a problem there!" fi

will attempt to run /usr/local/bin/my-command which should exit with a value of zero if all went well, or a nonzero value on failure. We can then handle this by checking the value of $? after calling the command. This helps make scripts robust and more intelligent.

The other two main variables set for you by the environment are $$ and $!. These are both process numbers.
$$ variable is the PID (Process IDentifier) of the currently running shell. This can be useful for creating temporary files, such as /tmp/my-script.$$  which is useful if many instances of the script could be run at the same time, and they all need their own temporary files.
The $! variable is the PID of the last run background process. This is useful to keep track of the process as it gets on with its job.

Another interesting variable is IFS. This is the Internal Field Seperator. The default value is SPACE TAB NEWLINE, but if you are changing it, it's easier to take a copy, as shown:

#!/bin/sh old_IFS="$IFS" IFS=: echo "Please input some data seperated by colons ..." read x y z IFS=$old_IFS echo "x is $x y is $y z is $z"

This script runs like this:
$ ./ifs.sh
Please input some data seperated by colons ...
hello:how are you:today
x is hello y is how are you z is today

 

EXTERNAL PROGRAMS

home - top of the page - email

External Programs

External programs are often used within shell scripts; there are a few builtin commands (echo, which, and test are commonly builtin), but many useful commands are actually Unix utilities, such as tr, grep, expr and cut.

The backtick (`) is also often associated with external commands. Because of this, we will discuss the backtick first.
The backtick is used to indicate that the enclosed text is to be executed as a command. This is quite simple to understand. First, use an interactive shell to read your full name from /etc/passwd:
$ grep "^${USER}:" /etc/passwd | cut -d: -f5
Steve Parker

Now we will grab this output into a variable which we can manipulate more easily:
$ MYNAME=`grep "^${USER}:" /etc/passwd | cut -d: -f5`
$ echo $MYNAME
Steve Parker

So we see that the backtick simply catches the standard output from any command or set of commands we choose to run. It can also improve performance if you want to run a slow command or set of commands and parse various bits of its output:

#!/bin/sh find / -name "*.html" -print | grep "/index.html$" find / -name "*.html" -print | grep "/contents.html$"

This code could take a long time to run, and we are doing it twice!
A better solution is:

#!/bin/sh HTML_FILES=`find / -name "*.html" -print` echo $HTML_FILES | grep "/index.html$" echo $HTML_FILES | grep "/contents.html$"

This way, we are only running the slow find once, roughly halving the execution time of the script.
We discuss specific examples further in the Hints and Tips section of this tutorial.

 

FUNCTIONS

home - top of the page - email

Functions

One often-overlooked feature of Bourne shell script programming is that you can easily write functions for use within your script. This is generally done in one of two ways; with a simple script, the function is simply declared in the same file as it is called.
However, when writing a suite of scripts, it is often easier to write a "library" of useful functions, and source that file at the start of the other scripts which use the functions. This will be shown later.
The method is the same however it is done; we will primarily be using the first way here.

There could be some confusion about whether to call shell functions procedures or functions; the definition of a function is traditionally that is returns a single value, and does not output anything. A procedure, on the other hand, does not return a value, but may produce output. A shell function may do neither, either or both. It is generally accepted that in shell scripts they are called functions.

A simple script using a function would look like this:

#!/bin/sh # A simple script with a function... add_a_user() {   USER=$1   PASSWORD=$2   shift; shift;   COMMENTS=$@   echo "Adding user $USER ..."   echo useradd -c "$COMMENTS" $USER   echo passwd $USER $PASSWORD   echo "Added user $USER ($COMMENTS) with password $PASSWORD" } ### # Main body of script starts here ### echo "Start of script..." add_a_user bob letmein Bob Holness the presenter add_a_user fred badpassword Fred Durst the singer add_a_user bilko worsepassword Sgt. Bilko the role model echo "End of script..."

Line 4 identifies itself as a function declaration by ending in (). This is followed by {, and everything following to the matching } is taken to be the code of that function.

Note that for this example the useradd and passwd commands have been prefixed with echo - this is a useful debugging technique to check that the right commands would be executed. It also means that you can run the script without being root or adding dodgy user accounts to your system!

So looking through the code, we have been used to the idea that a shell script is executed sequentially. This is not so with functions.
In this case, the function add_a_user is read in and checked for syntax, but not executed until it is explicitly called.
Execution starts with the echo statement "Start of script...". The next line, add_a_user bob letmein Bob Holness is recognised as a function call so the add_a_user is entered and starts executing with certain additions to the environment:
$1=bob
$2=letmein
$3=Bob
$4=Holness
$5=the
$6=presenter
So whilst within that function, $1 is set to bob, regardless of what $1 may be set to outside of the function.
We use the shift command again to get the $3 and onwards parameters into $@.
The function then adds the user and sets their password. It echoes a comment to that effect, and returns control to the next line of the main code.

Note that functions can be recusive - here's a simple example of a factorial function:

As promised, we will now briefly discuss using libraries between shell scripts. These can also be used to define common variables, as we shall see.

# common.lib # Note no #!/bin/sh as this should not spawn an extra shell. # It's not the end of the world to have one, but clearer not to. # STD_MSG="About to rename some files..." rename() {   # expects to be called as: rename .txt .bak   FROM=$1   TO=$2   for i in *$FROM   do     j=`basename $i $FROM`     mv $i ${j}$TO   done }

Hints and Tips

Unix is full of text manipulating utilities, some of the more powerful of which we will now discuss in this section of this tutorial. The significance of this, is that virtually everything under Unix is text. Virtually any file you can think of is controlled by either a text file, or by a command-line-interface (CLI). The only thing you can't automate using a shell script is a GUI-only utility or feature. And under Unix, there aren't too many of them!

We have already shown above a use of the simple but effective cut command. We shall discuss a few examples here some of the more common external programs to be used.

grep is an extremely useful utility for the shell script programmer.
An example of grep would be:

#!/bin/sh steves=`grep -i steve /etc/passwd | cut -d: -f1` echo "All users with the word \"steve\" somewhere in their passwd" echo "entry are: $steves"

This script looks fine if there's only one match. However, if there are two lines in /etc/passwd with the word "steve" in them, then the interactive shell will display:
$> grep -i steve /etc/passwd
steve:x:5062:509:Steve Parker:/home/steve:/bin/bash
fred:x:5068:512:Fred Stevens:/home/fred:/bin/bash
$> grep -i steve /etc/passwd |cut -d: -f1
steve
fred

But the script will display:
steve fred

By putting the result into a variable we have changed the NEWLINEs into spaces; the sh manpage tells us that the first character in $IFS will be used for this purpose. IFS is <space><tab><cr> by default. Maybe though we wanted to keep the NEWLINEs: It could look better if we made the spaces into NEWLINEs.... This is a job for tr:

#!/bin/sh steves=`grep -i steve /etc/passwd | cut -d: -f1` echo "All users with the word \"steve\" somewhere in their passwd echo "entry are: " echo "$steves" | tr ' ' '\012'

Note that tr translated the spaces into octal character 012 (NEWLINE).

Another common use of tr is its use of range... it can convert text to upper or lower case, for example:

#!/bin/sh steves=`grep -i steve /etc/passwd | cut -d: -f1` echo "All users with the word \"steve\" somewhere in their passwd echo "entry are: " echo "$steves" | tr ' ' '\012' | tr '[a-z]' '[A-Z]'

Here we have added a translation of [a-z] to [A-Z]. Note that there are exactly the same number of values in the range a-z as A-Z. This can then translate any character falling into the ASCII range a-z into A-Z ... in other words, converting lowercase letters into uppercase. tr is actually cleverer than this: tr [:lower:] [:upper:] would do the job just as well, and possibly more readably. It's also not as portable; not every tr can do this.

 

CGI Scripting

home - top of the page - email

For CGI programming, there are a couple of extra variables to be aware of, and a few tips I've picked up along the way.
Although the shell may not seem the obvious choice for CGI programming, it is quick to write and simple to debug. As such, it makes for an ideal prototyping language for CGI scripts, and fine for simple or little-used CGI scripts permanently.
fortune.cgi

#!/bin/sh echo "Content-type: text/html" echo echo "<html> <head> <title>Fortune Cookie</title> </head>"   oIFS=$IFS IFS="&" echo $QUERY_STRING|sed s/"%2F"/"\/"/g |sed s/"\%23"/"#"/g> /tmp/cookie.$$ . /tmp/cookie.$$ rm /tmp/cookie.$$ IFS=$oIFS cat - << EOFHTML <body text="$textcolour" bgcolor="$bgcolor"> <center> <h1>Fortune Cookie</H1> </center>  <BR> <PRE> EOFHTML   PARM=" " echo -n "Reading " if [ "$cookiels" = "long" ] then PARM="-l"  echo -n "long " ckls=">" fi if [ "$cookiels" = "short" ] then PARM="-s"  echo -n "short " ckls="<" fi if [ "$cookiels" = "both" ] then echo -n "all " fi if [ "$cookiels" != "both" ] then PARM="-n ${cookielength} ${PARM}" echo -n "(${ckls} ${cookielength} character) " fi if [ -n "${usegivenfile}" ] then PARM="${givenfile} ${PARM}" echo -n "cookies from ${givenfile} ..." else PARM="${cookiedirectory} ${PARM}" #echo -n "all cookies in ${cookiedirectory} ..." echo -n "cookies..." fi echo echo "$ /usr/games/fortune $PARM" echo "<HR> <BR> <BR> <BR>" /usr/games/fortune ${PARM} echo "</pre>" cat -- << EOFHTML <BR> <BR> <BR> <HR> <a href="/cgi-bin/cookie.cgi">Choose new settings for cookie</a> </body> </html> EOFHTML

cookie.cgi

#!/bin/sh dir=/usr/share/games/fortunes echo "Content-type: text/html" echo cat - << EOFHTML <html> <head> <title>Fortune Cookie Loader</title> </head> <body text=white bgcolor=black> <center> <h1>Fortune Cookie Loader</H1> </center>  <BR> <BR> <BR> <form action=./fortune.cgi method=get> From Directory: <input type=text name=cookiedirectory value=$dir size=30c> <HR> Subject: <select name=givenfile> EOFHTML   for i in `ls -1 ${dir} | grep -v "\.dat$"` do echo "<option value=${i}>${i}</option>" done cat - << EOFHTML </select> <BR> Use only this file:  <input type=checkbox name=usegivenfile> <HR> Only Long Cookies: <input type=radio name=cookiels value=long> <BR> Only Short Cookies:  <input type=radio name=cookiels value=short> <BR> A Short cookie is less than <input type=text size=4c name=cookielength value=160> characters long <BR> Both Long and Short Cookies:  <input type=radio name=cookiels value=both checked> <HR> Text Colour:  <select name=textcolour> EOFHTML for i in lightgreen green black white red yellow do echo "<option value=$i>$i</option>" done echo "</select> <BR>Background Colour: " echo "<select name=bgcolor>" for i in black green white red yellow do echo "<option value=$i>$i</option>" done echo "</select>" cat - << EOFHTML <HR> <input type=submit value="Show me the Cookies!"> ... or ... <input type=reset value="Reset to Defaults"> </form> </body> </html> EOFHTML

Okay, I'll work on this ... Even as a text file, it's reading the Content-type stuff from the script! For now, if you don't see code, hit SHIFT-RELOAD under Netscape.