Linux Shell Scripting Tutorial (LSST) v1.05r3
Chapter 7: awk Revisited

Real life example in awk

Before learning more features of awk its time to see some real life example in awk.

Our first Example

I would like to read name of all files from the file and copy them to given destination directory. For e.g. The file filelist.conf; looks something as follows:
/home/vivek/awks/temp/file1     /home/vivek/final
/home/vivek/awks/temp/file2     /home/vivek/final
/home/vivek/awks/temp/file3     /home/vivek/final
/home/vivek/awks/temp/file4     /home/vivek/final

In above file first field ($1) is the name of file that I would like to copy to the given destination directory ($2 - second field) i.e. copy /home/vivek/awks/temp/file1 file to /home/vivek/final directory. For this purpose write the awk program as follows:

$ cat > temp2final.awk
#Linux Shell Scripting Tutorial v1.05, March 2001
#Author: Vivek G Gite


# main logic is here
    sfile = $1
    dfile = $2
    cpcmd = "cp " $1 " " $2
    printf "Coping %s to %s\n",sfile,dfile

# End action, if any, e.g. clean ups

Run it as follows:
$ awk -f temp2final.awk filelist.conf

Above awk Program can be explained as follows:

sfile = $1Set source file path i.e. first field ($1) from the file filelist.conf
dfile = $2Set source file path i.e. second field ($2) from the file filelist.conf
cpcmd = "cp " $1 " " $2Use your normal cp command for copy file from source to destination. Here cpcmd, variable is used to construct cp command.
printf "Coping %s to %s\n",sfile,dfileNow print the message
system(cpcmd)Issue the actual cp command using system(), function.

system() function execute given system command. For e.g. if you want to remove file using rm command of Linux, you can write system as follows
system("rm foo")
dcmd = "rm " $1

The output of command is not available to program; but system() returns the exit code (error code) using which you can determine whether command is successful or not. For e.g. We want to see whether rm command is successful or not, you can write code as follows:

$ cat > tryrmsys
  dcmd = "rm " $1
   if ( system(dcmd) != 0 )
         printf "rm command not successful\n"
        printf "rm command is successful and %s file is removed \n", $1

Run it as (assume that file foo exist and bar does not exist)
$ awk -f tryrmsys
rm command is successful and foo file is removed
rm command not successful

(Press CTRL + D to terminate)

Our Second Example:

As I write visual installation guide, I use to capture lot of images for my work, while capturing images I saved all images (i.e. file names) in UPPER CASE for e.g.


Now I would like to rename all files to lowercase then I tried with following two scripts:

up2low and rename.awk

up2low can be explained as follows:

AWK_SCRIPT="rename.awk"Name of awk scripts that renames file


Where our awk script is installed usguall it shoude installed under your-home-directory/bin (something like /home/vivek/bin)
ls -1 > /tmp/file1.$$List all files in current working directory line by line and send output to /tmp/file1.$$ file.
tr "[A-Z]" "[a-z]" < /tmp/file1.$$ > /tmp/file2.$$Now convert all Uppercase filename to lowercase and store them to /tmp/file2.$$ file.
paste /tmp/file1.$$ /tmp/file2.$$ > /tmp/tmpdb.$$Now paste both Uppercase filename and lowercase filename to third file called /tmp/tmpdb.$$ file
rm -f /tmp/file1.$$
rm -f /tmp/file2.$$
Remove both file1.$$ and file2.$$ files
if [ -f $awkspath ]; then
  awk -f $awkspath /tmp/tmpdb.$$
  echo -e "\n$0: Fatal error - $awkspath not found"
  echo -e "\nMake sure \$awkspath is set correctly in $0 script\n"
See if rename.awk script installed, if not installed give error message on screen. If installed call the rename.awk script and give it /tep/tepdb.$$ path to read all filenames from this file.
rm -f /tmp/tmpdb.$$Remove the temporary file.

rename.awk can be explained as follows:

isdir1 = "[ -d " $1 " ] "This expression is quite tricky. Its something as follows:
isdir1 = [ -d $1 ]
Which means see if directory exists using [ expr ]. As you know [ expr ] is used to test whether expr is true or not. So we are testing whether directory exist or not.
What does $1 mean? If you remember, in awk $1 is the first field.
isdir2 = "[ -d " $2 " ] "As above except it test for second field as
isdir2 = [ -d $2 ]
i.e. Whether second field is directory or not.
scriptname = "up2low"
awkscriptname = "rename.awk"
Our shell script name (up2low) and awk script name (rename.awk).
sfile = $1Source file
dfile = $2Destination file
if ( sfile == scriptname || sfile == awkscriptname )
Make sure we don't accidentally rename our own scripts, if scripts are in current working directory
else if( ( system(isdir1) ) == 0 || system((isdir2)) == 0 )
  printf "%s or %s is directory can't rename it to lower case\n",sfile,dfile
  next # continue with next recored
Make sure source or destination are files and not the directory. We check this using [ expr ] command of bash. From the awk script you can called or invoke (as official we called it) the [ expr ] if directory do exists it will return true (indicated by zero) and if not it will return nonzero value.
else if ( sfile == dfile )
  printf "Skiping, \"%s\" is alrady in lowercase\n",sfile
If both source and destination file are same, it mean file already in lower case no need to rename it to lower case.
else # everythink is okay rename it to lowercase
  mvcmd = "mv " $1 " " $2
  printf "Renaming %s to %s\n",sfile,dfile

Now if source and destination files are not

  • Directories
  • Name of our scripts
  • And File is in UPPER CASE

Then rename it to lowercase by issuing command mv command.


Note that if you don't have files name in UPPER case for testing purpose you can create files name as follows:

$ for j in 1 2 3 4 5 6 7 8 9 10; do touch TEMP$j.TXT; done

Above sample command creates files as TEMP1.TXT,TEMP2.TXT,....TEMP10.TXT files.

Run it as follows:
$ up2low
Letters or letters is directory can't rename it to lower case
RH6_FILES or rh6_files is directory can't rename it to lower case
Renaming RH7x01.JPG to rh7x01.jpg
Renaming RH7x02.JPG to rh7x02.jpg
Renaming RH7x03.JPG to rh7x03.jpg
Renaming RH7x04.JPG to rh7x04.jpg
Renaming RH7x05.JPG to rh7x05.jpg
Renaming RH7x06.JPG to rh7x06.jpg
Renaming RH7x138.JPG to rh7x138.jpg

On my workstation above output is shown.

Loops in awk
awk miscellaneous