levselector.com New York
home > Unix Shells Email

 
Shell Scripting Other pages
intro
setting environment
  (  - ksh .profile, - tcsh  .cshrc   .login  .aliases, - bash )
wrappers
  ( - wrapper1, - db1, db2, db3, db4)
ftp telnet
bash_event_designators
unix
tcsh
Bourne sh_short
Bourne sh_tutorial


 
intro home - top of the page - email

A shell is a command interpreter. You type commands on the prompt. Shell interprets them and passes further for the execution. Over the years unix developers created many shells. Here are manpages for 5 most important ones:
www.engin.umich.edu/htbin/mangate?manpage=sh - sh (bourne shell) - first, simpliest.
www.engin.umich.edu/htbin/mangate?manpage=csh - csh - from Berkley. Use tcsh instead of it.
www.engin.umich.edu/htbin/mangate?manpage=ksh - ksh (korn shell) - common, many programming features
www.engin.umich.edu/htbin/mangate?manpage=tcsh - tcsh - very good convenient shell (enchanced csh)
www.engin.umich.edu/htbin/mangate?manpage=bash - bash (born again shell) - main shell in the Linux world

www.whereisit.com/shellscript/index2.html - shellscript.com - some good links

If you have any experience at all with Unix - you worked with one of those. The first one (Bourne shell) is very common - you can use it to write scripts. The C-shell (csh) is common, but should be avoided. You better use tcsh for command line editing, and bourne shell for installation scripts (read Tom Christiansen's article why csh (and tcsh) should not be used for writing shell scripts).

The last 3 scripts (ksh,tcsh, bash) are much more convenient to work with and you can use one of them as your command-line shell.

To find which shell you are running login to the prompt and use ps command with no options.
Often $SHELL variable will show the current shell too.
Also in the /etc/passwd file you can see what is your default shell.
Here is what you can see on the prompt (your login name is "mylogin"):
> ps
  PID  TT  STAT      TIME COMMAND
 6111  p0  Ss     0:00.06 -bash (bash)
 6124  p0  R+     0:00.01 ps
> echo $SHELL
/bin/bash
> which bash
/usr/local/bin/bash
> which tcsh
/bin/tcsh
> which ksh
/bin/ksh
> which sh
/bin/sh
> which csh
/bin/csh
> which zsh
/bin/zsh
> whoami
mylogin
> cat /etc/passwd | grep mylogin
mylogin:*:1487:102:,,,,:/home/mylogin:/bin/bash
>

www.opennc.org/onlinepubs/7908799/xcu/sh.html - using sh (includes vi-mode command line editing).
http://codedata.box.sk/ghostrider/bash.tutor - bash tutorial
http://uracil.cmc.uab.edu/MolModLab/tutor/irix/basecmd.shtml - intro into unix, includes shell scripting

2 ways to run shel scripts:
  1.start a new shell (from command line or from a "shabang" line (first line of your script)).
  2.source the file (using source or '.' (dot command)) - this way you can change the enviromental variables in the currently running shell.

To force shell to echo commands:
  use -v option (works with ksh, csh, tcsh & bash), for example:
    tcsh -v myscript.sh

Using shell scripts:
I mostly use Perl instead of shell scripting. But you still need to know at least a little bit of shell scripting for:
  - writing very short scripts or installations scripts (see examples below).
  - setting environment (in .profile or .cshrc file)
  - writing thin wrappers (see examples below)

Again, let me repeat main recommendations:
1. use tcsh, ksh or bash for your command-line editing.
2. use perl for scripting
3. If you need to write a shell script (for installation, for example) - use bourne shell (sh) and NOT csh.

 
setting environment home - top of the page - email

ksh:   .profile

.profile
# ------------------------------------------
#  .profile for ksh on Solaris
# ------------------------------------------

if [ -f /somedir/setEnv.sh ]; then
    . /somedir/setEnv.sh
fi

set -o vi
EDITOR=vi; export EDITOR
VISUAL=vi; export VISUAL

ulimit -c 0     # LIMIT CORE DUMP SIZE

rev=`uname -r`
case $rev in
5.*)  echo Solaris Profile
 PATH=$PATH:/usr/bin:/usr/sbin:/opt/bin:/opt/systools:/opt/sybase/bin:/usr/ucb:/usr/ccs/bin:/usr/java/bin:.
 export PATH
 LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib:/usr/java/lib:.:/opt/sybase/openclient/11.11/lib:/opt/sybase/lib
 export LD_LIBRARY_PATH
 CLASSPATH=$CLASSPATH:/opt/jconnect-5.2/classes/jconn2.jar:/usr/java/lib:.:/opt/devshare/htdocs/classes:/somedir/selector.jar
 export CLASSPATH
 ;;
4.*) echo SunOS Profile
 PATH=$PATH:/bin:/usr/bin:/opt/bin:/etc:/usr/ucb:/usr/bin/X11:/usr/lib:/usr/etc:/usr/man:$HOME:/opt/systools:/usr/openwin/bin
 export PATH
 ;;
esac

# ------------------------------------------
#         set up ksh
# ------------------------------------------
TERM=xterm;  export TERM
set -o emacs
umask 002

PS1=`/usr/ucb/whoami`:'$PWD:'" "
PS2=">"

HISTSIZE=1024
HISTFILE=$HOME/.sh_history

stty erase ^H

# ------------------------------------------
#         set up MAN path
# ------------------------------------------
MANPATH=$MANPATH:/usr/local/man:/usr/dt/man:/usr/openwin/man
MANPATH=/usr/share/man:$MANPATH:/opt/perl/perl5.003/man
export MANPATH
##### Check printer ###
if [ -f $HOME/.printer ] ; then
 PRINTER=`cat $HOME/.printer` ; export PRINTER
fi

# ------------------------------------------
if [ -f $HOME/.profile.local ]; then
 . $HOME/.profile.local
fi

# ------------------------------------------
LD_LIBRARY_PATH=/opt/sybase/lib:/usr/lib:/usr/local/lib:/usr/lib/X11:/usr/local/openwin/lib:/wrk/staging/FrameWork/rogue/lib
export LD_LIBRARY_PATH

export SYBASE=/opt/sybase

PATH=$PATH:$HOME/bin:$HOME/util
export PATH

CVSEDITOR=vi; export CVSEDITOR
CVSROOT=/home/proj/cvs_repository; export CVSROOT

alias l='ls -alF'
alias ii='isql -S PRODSERV -U mylogname -P mypasswd'
alias ss='sqsh -S PRODSERV -U mylogname -P mypasswd' 
alias cgi="cd ~/proj/www/cgi-bin"
alias myupdate="cvs update -d -P"

Settings for your xterm:
/usr/local/bin/dtterm -d 10.3.3.101:0.0 -fg black -bg white &
xterm -d daddypc:0.0 -fg green -bg black &
xemacs -d daddypc:0.0 &
alias term='xterm -d daddypc:0.0 -fn -*-Fixed-Medium-R-*-*-*-120-75-75-*-*-ISO8859-1 -bg DarkCyan -fg White -geom 90x30 -cr red &'
alias term0='xterm -d daddypc:0.0 -fg black -bg white &'
alias term1='xterm -d daddypc:0.0 -fn -*-Fixed-Medium-R-*-*-*-120-75-75-*-*-ISO8859-1 -bg black -fg LemonChiffon -geom 90x30 -cr red'
alias term2='xterm -d daddypc:0.0 -fn -*-Fixed-Medium-R-*-*-*-140-75-75-*-*-ISO8859-1 -bg black -fg LemonChiffon -geom 100x40 -cr red'
# export DISPLAY=daddypc:0.0
/usr/openwin/bin/xfontsel -d daddypc:0.0 &
/usr/openwin/bin/xlsfonts
CTRL-(right-mouse-button)

tcsh:   .cshrc  .login   .aliases

.cshrc
# ------------------------------------------
# .cshrc file for tcsh
# ------------------------------------------
if ( $?prompt ) then
    echo "Running cshrc..."
endif

if ($?DEBUGCSHRC) set echo

umask 2

# figure out what os we're in
set os=`/usr/bin/uname -r -s | /usr/bin/sed -e 's/ \([0-9]\).*$/\1/'`
switch ($os)
 case "SunOS4":
  setenv VERSION sunos
  breaksw
 case "SunOS5":
  setenv VERSION sparc
  breaksw
 default:
endsw

if ( $?prompt ) then
    test -f ~hallro/.aliases && source ~/.aliases
    set notify
    set filec
    set nobeep
    set history=100
endif

if ($?HAVEPATH == 0) then
   if ($?OPENWINHOME == 0) setenv OPENWINHOME /usr/local/openwin
   set path=(/home/hallro/usr/bin \
              /home/oedev/usr/bin \
              /home/slsdev/bin \
              /opt/perl-5.004/bin \
              /opt/workshop/SUNWspro/bin \
              $OPENWINHOME/bin \
              /bin \
              /usr/bin \
              /usr/ucb \
              /usr/sbin \
              /opt/bin \
              /opt/systools \
              /opt/sybase/bin \
              /usr/ucb \
              /usr/ccs/bin \
              /usr/java/bin \
             /opt/java/jdk1.2.2/bin \
              . \
   )

   if (-d /home/nysid10/sqllib/) then
        source /home/nysid10/sqllib/db2cshrc
   endif

  setenv HOSTNAME `/bin/hostname`
  setenv HAVEPATH
endif

if ($?SYBASE == 0) then
    if (-d /opt/sybase) then
       if ($?LD_LIBRARY_PATH == 0) then
           setenv LD_LIBRARY_PATH ""
       endif
       setenv LD_LIBRARY_PATH $LD_LIBRARY_PATH\/opt/sybase/lib:
       setenv LD_LIBRARY_PATH $LD_LIBRARY_PATH\/opt/sybase/openclient/11.11/lib

       setenv SYBASE /opt/sybase
       setenv PATH $PATH\/opt/systools:/opt/sybase/bin:
    else if (-d /usr/local/sybase) then
       setenv SYBASE /usr/local/sybase
    endif
endif

if ($?OPENWINHOME) then
    if (-d /usr/local/openwin) then
        setenv OPENWINHOME /usr/local/openwin
        setenv LD_LIBRARY_PATH $LD_LIBRARY_PATH$OPENWINHOME/lib
        setenv PATH $PATH\/usr/openwin/bin:
    endif
endif

.login
# ------------------------------------------
# .login file for tcsh
# ------------------------------------------

if ( $?prompt ) then
    echo "Running login..."
endif

if ($?DISPLAY == 0) then
    if ($?REMOTEHOST != 0) then
        setenv DISPLAY $REMOTEHOST":0.0"
    endif
endif

if ($?MANPATH == 0) then
    setenv MANPATH ""
endif
setenv MANPATH $MANPATH\/usr/local/man:/usr/openwin/man:/usr/man:/opt/man:
setenv MANPATH $MANPATH\/home/hallro/usr/man:

if ($?PRINTER) then
    setenv PRINTER pusg25a
endif

setenv EDITOR vi
setenv CVSROOT /home/mydev/cvs_repository
setenv CVSEDITOR vi

.aliases
# ------------------------------------------
# .aliases file for tcsh
# ------------------------------------------
if ( $?prompt ) then
  echo "Running aliases..."

  alias ls      "\ls -F"
  alias l       "\ls -alF"
  alias cls     "\clear"
  alias cl      "\clear; ls"
  alias h       history
  alias DISPLAY "setenv DISPLAY fcfrnb419.jany.gs.com:0.0"
  alias hw      "\hostname; \whoami"

  # ---- Find Magic :-)

  alias fpcgi 'find . -name "*.cgi" -print -exec grep "\!^" {} \;'
  alias fcgi  'find . -name "*.cgi" -exec grep "\!^" {} \;'
  alias fppl  'find . -name "*.pl" -print -exec grep "\!^" {} \;'
  alias fpl   'find . -name "*.pl" -exec grep "\!^" {} \;'
  alias fppm  'find . -name "*.pm" -print -exec grep "\!^" {} \;'
  alias fpm   'find . -name "*.pm" -exec grep "\!^" {} \;'

  # ---- Web development...

  alias GET    "setenv REQUEST_METHOD GET"
  alias POST   "setenv REQUEST_METHOD POST"

  # ---- Project Specific

  alias pf "cd ~someuser/somedir; pwd"

  alias DEVSERV "echo 'DEVSERV as mylogname'; \
                   isql -S DEVSERV -U mylogname"
  alias PRODSERV "echo 'PRODSERV as mylogname'; \
                    isql -S PRODSERV -U mylogname"

  alias sybdev "echo 'DEVSERV as mylogname'; \
                isql -S DEVSERV -U mylogname"
  alias sybprd "echo 'PRODSERV as mylogname'; \
                isql -S PRODSERV -U mylogname"

  alias iifiam="isql -U mylogname -P mylogname -S SERV2"

  alias cdcom "cd ~/dir/common; pwd"
  alias cdpr "cd ~/dir/project/www/cgi-bin; pwd"

  alias london "rlogin -l mylogin  serv.domain.com"

  # ---- Host specific

  if (`hostname` == "host1.dom1.com") then 
      alias  wl  "tail  -f  somepath/logs/errors"
  endif

  alias   psgrep  'ps -auxgww | grep $1'
  alias   psapi   'ps -auxgww | grep api'
  alias   ldir    'ls -al | grep "^d"'
  alias   tm     '(set d = `date` ; echo $d[4])'
  alias   since   'set t = `who | grep $USER` ; echo $t[5]; unset t'
  alias   fixt    'stty erase' 
  alias   sett    'setenv TERM vt100'
  alias   l       ls -alF
  alias   em  'emacs -fg white -bg black -fn 6x13'
endif

# setting nice prompt (see www.engin.umich.edu/caen/faqs/setprompt.html )

   if( ${?prompt} ) then
      alias setprompt 'set prompt = `whoami`:"%/-> "'
#      alias setprompt 'set prompt = `whoami`:"%~-> "'
      alias cwdcmd 'setprompt'
      setprompt
   endif

# --------------------
# and some more
# --------------------
set addsuffix
set ampm
set autolist=ambiguous
set autoexpand
#set autologout=600
unset autologout
set cdpath=(~/user ~/user/procmon ~/user/src ~)
set complete = enhance
set symlinks=chase                        # $cwd has real path (follows links)
#set correct=all
set correct=cmd
set echo_style=both                       # an hpux thing to make -n opt work
set fignore = (\- .o CVS)                 # suffixes of files for tab to ignore
set filec
set history = 500 
set hislit                                #tcsh-show history as typed
set host = "`hostname`"; set host=$host:r; set host=$host:r
#set hostfull = (`nslookup $host | sed -n -e "/Name:/s/\(Name: \)\(.*\)/\2/p" `)
set listjobs = long                       #tcsh-list all jobs when suspending
#set mail=(10 /usr/mail/lane)
set nobeep
set nonomatch
set nostat = (~/xstuff)                   #tcsh-dirs not to stat for completion
set notify
set printexitvalue                        #tcsh-exit val for interactive prog
set prompt2
set prompt3="CORRECT> %R (y|n|e)?"
set rmstar
set savehist = 500


bash    .bashrc     .bash_profile

.bashrc
# --------------------
# .bashrc
# --------------------
alias rm='rm -i'
alias cp='cp -i'
alias mv='mv -i'

# ---- User specific aliases and functions

# ---- Source global definitions
if [ -f /etc/bashrc ]; then
        . /etc/bashrc
fi

.bash_profile
# --------------------
# .bash_profile
# --------------------

# ---- Get the aliases and functions
if [ -f ~/.bashrc ]; then
        . ~/.bashrc
fi

# ---- User specific environment and startup programs
umask 022

ORACLE_BASE=/usr/local/oracle/8i/u01/app/oracle
ORACLE_HOME=$ORACLE_BASE/product/8.1.6
ORACLE_SID=starter

BASH_ENV=$HOME/.bashrc
USERNAME=""
JAVA_HOME=/usr/local/java
JSDK_HOME=/usr/local/jsdk
CLASSPATH=.:$JAVA_HOME/lib/dt.jar
CLASSPATH=$CLASSPATH:$JSDK_HOME/servlet-2.0.jar
CLASSPATH=$CLASSPATH:/usr/local/java/lib/xalan.jar
CLASSPATH=$CLASSPATH:/usr/local/java/lib/xerces.jar
CLASSPATH=$CLASSPATH:/usr/local/oracle/8i/u01/app/oracle/product/8.1.6/jdbc/lib/classes111.zip
CLASSPATH=$CLASSPATH:/usr/local/oracle/8i/u01/app/oracle/product/8.1.6/jdbc/lib/nls_charset11.zip

PATH=.:$HOME/util:$PATH:$HOME/bin:$ORACLE_HOME/bin:$JAVA_HOME/bin
export USERNAME BASH_ENV PATH JAVA_HOME JSDK_HOME CLASSPATH ORACLE_BASE ORACLE_HOME ORACLE_SID


 
writing thin wrappers home - top of the page - email

Example: a thin wrapper running a perl script (note how syntax is different in 2 shells):
C-shell Bourne shell
#! /bin/csh
set mailTo='me@somedomain.com'
set bindir='/home/some_directory'
set log1file='/tmp/myfile.log'
rm -f $log1file
touch $log1file
echo "Start time:" >> $log1file
date >> $log1file

$bindir/some_script.pl >>& $log1file
if ( "$status" != "0" ) then
  echo "Error at:" >> $log1file
  date >> $log1file
  /opt/fax/bin/sendmime.pl --to=$mailTo  \
  --subject='myjob failed' --msg_file=$log1file
  exit(1)
endif

echo "Successful end time:" >> $log1file
date >> $log1file
/opt/fax/bin/sendmime.pl --to=$mailTo \
--subject='myjob Successful' --msg_file=$log1file
exit(0)

#!/bin/sh
mailTo='selector@pipeline.com'
bindir='/home/selecl/tests'
log1file="/tmp/myfile.log"
rm -f $log1file
touch $log1file
echo "Start time:" >> $log1file
date >> $log1file

$bindir/some_script.pl  1>>$log1file 2>>$log1file
if [ $? -ne 0 ]; then
   echo "Error at:" >> $log1file
   date >> $log1file
   /opt/fax/bin/sendmime.pl --to=$mailTo \
   --subject='myjob failed' --msg_file=$log1file
   exit 1
fi

echo "Successful end time:" >> $log1file
date >> $log1file
/opt/fax/bin/sendmime.pl --to=$mailTo \
--subject='myjob Successful' --msg_file=$log1file
exit 0

Example: a simple shell script I use to install tables and stored procedures into Sybase database.
#! /bin/csh
set  dbServer="-S MY_DB_SERVER -U someuser -P somepasswd"
isql $dbServer -ifile1.sp
isql $dbServer -ifile2.sp
isql $dbServer -ifile3.sp
isql $dbServer -ifile4.sp

Two small examples of including text into the script as "here doc" - or reading from a file  (csh syntax):

$SYBASE/bin/isql -U$DB_USER -P$DB_PASSWD -S$SERVER -w300 <<EOF > $resultFile 
use $DATABASE 
go 
exec some_stored_procedure
go 
EOF 
 

$SYBASE/bin/isql -U$DB_USER -P$DB_PASSWD -S$SERVER -w300 <<! 
:r /home/somefile.sql
go 

 

More sophisticated version - requires one parameter - a name of a text file with SQL to run
#!/usr/bin/sh
SERVER="MY_DB_SERVER"
DATABASE="mydatabase"
DB_USER=someuser
DB_PASSWD=somepasswd

if [ $# != 1 ]
then 
    echo "Usage: $0 <file>"
    exit 1
fi

file=$1

if [ ! -f $file ]
then
    echo "File \"$file\" Does Not Exist"
    exit 1
fi

PATH="/bin:/usr/bin:/usr/local/bin:/usr/local/sybase/bin:someotherdirectories"
SYBASE="/usr/local/sybase"
export PATH
export SYBASE

$SYBASE/bin/isql -U$DB_USER -P$DB_PASSWD -S$SERVER -w250 <<EOF
print "Installing $file in Database: $DATABASE Server: $SERVER"
go
use $DATABASE
go
`cat $file`
EOF
exit 0

# db.sh - wrapper to prevent password from showing in ps:
#!/bin/sh
MYSQL="sql.txt"
MYPASS="somepaswd"

isql -S SOMESERVER -U someuser <<EOF
$MYPASS
`cat $MYSQL`
EOF
exit(0)

Here "sql.txt" is a file with SQL commands, for example:
use somedatabase
go
select * from sometable
go

 
writing thin wrappers home - top of the page - email

Automating FTP:
#!/bin/sh

ftp -n <<EOF
open www.levselector.com
user myname mypass
get sleep.pl
quit
EOF

Note: "-n" option prevents ftp from asking the login and passwd on open command.
Thus you can supply them both using the user command.
Here is another way of doing the same plus some minimal error checking
#!/bin/sh 

echo "open webprod
      user login passwd 
      verbose
      cd dir1/dir2
      bin
      get file1.cgi
      close
      quit" | ftp -n > myftplog 

# Check FTP log to see if the file was retrieved okay 

if grep "^226" myftplog 
then 
  echo "Successfully got the file from remote server" >> myftplog
else 
  echo "Failed to get the file" >> myftplog 
fi

By the way, you can always use Perl:
Example using  Net::FTP
use Net::FTP;
$ftp = Net::FTP->new("some.host.name");
$ftp->login("anonymous","me@here.there");
$ftp->cwd("/pub");
$ftp->binary;
$ftp->get("that.file");
$ftp->quit;

Example using LWP::Simple
use LWP::Simple;
my $status = getstore('ftp://user:password@host/foo/file', 'localfile');

 
Automating telnet home - top of the page - email

Sending telnet from a script - I don't know how to do this from a shell script, because it doesn't have options for that. I think.
Usually people use something like remote shell (rsh)
    http://www.engin.umich.edu/htbin/mangate?manpage=rsh
or kermit
    http://www.columbia.edu/kermit/ckermit.html
    http://www.columbia.edu/kermit/telnetd.html
    http://www.columbia.edu/kermit/ck80.html

I would prefer to use a Perl script using NET::Telnet module from CPAN:
    http://www.perldoc.com/cpan/Net/Telnet.html

Here are examples right from documentation of NET::Telnet  :
 
use Net::Telnet ();
$t = new Net::Telnet (Timeout => 10,  Prompt => '/bash\$ $/');
$t->open("sparky");
$t->login($username, $passwd);
@lines = $t->cmd("/usr/bin/who");   # give command
print @lines;      # collect response
   # etc.

 
my ($forecast, $t);
use Net::Telnet ();
$t = new Net::Telnet;
$t->open("rainmaker.wunderground.com");
## Wait for first prompt and "hit return".
$t->waitfor('/continue:.*$/');
$t->print("");
## Wait for second prompt and respond with city code.
$t->waitfor('/city code.*$/');
$t->print("BRD");
## Read and print the first page of forecast.
($forecast) = $t->waitfor('/[ \t]+press return to continue/i');
print $forecast;
exit;

 
# -----------------------------------------------------------
# example checking mail on a pop server
# -----------------------------------------------------------
my ($hostname, $line, $passwd, $pop, $username);
$hostname = "your_destination_host_here";
$username = "your_username_here";
$passwd = "your_password_here";
use Net::Telnet ();
$pop = new Net::Telnet (Telnetmode => 0);
$pop->open(Host => $hostname, Port => 110);
## Read connection message.
$line = $pop->getline;
die $line unless $line =~ /^\+OK/;
## Send user name.
$pop->print("user $username");
$line = $pop->getline;
die $line unless $line =~ /^\+OK/;
## Send password.
$pop->print("pass $passwd");
$line = $pop->getline;
die $line unless $line =~ /^\+OK/;
## Request status of messages.
$pop->print("list");
$line = $pop->getline;
print $line;
exit;



 

BASH Event Designators home - top of the page - email


- www.gnu.org/software/bash/manual/bashref.html - Bash Reference

An event designator is a reference to a command line entry in the history list.
! Start a history substitution, except when followed by a space, tab, the end of the line, `=' or `('
!n Refer to command line n
!-n Refer to the command n lines back
!! Refer to the previous command. This is a synonym for `!-1'
!string Refer to the most recent command starting with string
!?string[?] Refer to the most recent command containing string. The trailing `?' may be omitted if the string is followed immediately by a newline
^string1^string2^ Quick Substitution. Repeat the last command, replacing string1 with string2. Equivalent to !!:s/string1/string2/
!# The entire command line typed so far