Writing some bash scripts that parse command lines, I wrote this handy template with getopt. It is easy to apply even for simplest scripts.

OPTION_SPEC="help,flag1,flag2_params:"
PARSED_OPTIONS=$(getopt -n "$0" -a -o h --long $OPTION_SPEC -- "$@")
OPTIONS_RET=$?
eval set -- "$PARSED_OPTIONS"


Parsing error or no flags

if [ $OPTIONS_RET -ne 0 ] || [ $# -le 0 ]; then
  usage;
  die;
fi

while [ $# -ge 1 ]; do
  case $1 in
    --help | -h )  usage; die;;
    --flag1 )  FLAG1=1;;
    --flag2_params )  shift; FLAG2_PARAMS="$1";;
    -- ) shift;;
    * ) echo "ERROR: unknown flag $1"; usage; die;;
  esac
  shift
done

No more unannotated $ns in my scripts!

I recently had to hack a small shell script that would read files in a directory structure generated based on the date, something like 2007/08/16. The trick was that the script would look at yesterday’s file or files generated a few days ago.

A quick search on info and here’s the magic command

FILE="...$(date -d 'yesterday' +%Y/%m/%d)"

Interestingly, you can also use things like 3 days ago, next Monday, 2 months etc. Cool!

I recently made a really stupid mistake. I wanted to do a simple variable assignment and run a command. What I did was the following:

VAR=value command $VAR

Obviously, this doesn’t work! As this is a single command, the shell does variable expansion before the assignment takes place, so the variable is null (or whatever it was before). Obviously, the variable is set correctly within the command’s execution environment, but it’s too late then

What I should have done in this case was:

VAR=value; command $VAR

Sadly, the command did not complain about my mistake and I only realized it after a lost night of computation. Phew!

I don’t like programming in bash but it does make some things very simple. Unfortunately, it is not obvious how to do some simple things like iterating through an array

Assuming that we have an array ITEMS=( a b c d ), we can use a ${ITEMS[@]} construct to iterate through all the elements in a for loop:

ITEMS=( a b c d )
for ITEM in ${ITEMS[@]}; do
    echo $ITEM
done

BTW: I found this pattern here.

I used it some time ago, but have already forgotten how I did it and had to reinvent the wheel. Here it goes.

You sometimes need to start a background process in shell, which can die right away, in which case you want to know about it. Situations in which it is useful include init.d scripts, running some background processes… Here are two useful shell functions:

function pidactive () {
    #sends a signal which checks if the process is active (doesn't kill anything)
    kill -0 $1 2> /dev/null
    return
}

function pidkill () {
    echo "killing pid"
    kill $1 || return
    #adjust depending how long it takes to die gracefully
    sleep 1
    if pidactive $1; then
        #escalating
        kill -9 $1
    fi  
}

What I do in the script is the following:

<command> &
PID=$!
#wait for the process to startup or die...
sleep 5
if ! pidactive $PID; then
    wait $PID
    die "Command failed with $?"
fi

...
#kill the process before exiting
pidkill $PID

It’s stupid, but it took me a good hour to figure this out, so maybe I’m not the only one…

I’ve recently had a problem with command-line arguments in my Java program. The problem was that command line arguments containing spaces were parsed incorrectly, i.e. chopped into individual arguments. My initial suspect was gnu.Getopt package I use for parsing arguments, but as it turned out I was wrong.

The real culprit was a shell wrapper script I used to wrap my java code. The code was the following:

java <some parameters> <programm.class> $@

See the problem? I didn’t. You need quotes around "$@" in which case the parameter gets expanded to: "$1" "$2" "$3"... With no quotes the shell expands it to $1 $2 $3, hence all parameters containing spaces get chopped (also globbing takes place in this case).

BTW: There’s also "$*" which is used to combine all parameters into a single one, i.e, "$*" expands to "$1c$2c$3c..., where c is $IFS (or space). Here it’s also important to have it enclosed in quotes.

Trying to automate something I had to run a program with a configuration file modified for each run. The easy way to do this is to create a template file and generate the correct configuration file with variable substitution for each run.

This task could be done in a number of ways (sed, perl, shell), to name a few, but as Diego pointed out there is already a tool for this m4. It’s a bit archaic and has odd syntax (think strange quotes or strangely named built-in macros), but it’s ideal for my simple task.

At the end I run it like this: m4 -Dparam1=value1 -Dparam2=value2 <infile> > <outfile>

As simple as that. Thanks Diego! BTW: some pointers to M4: manual and linux journal article.

Recently learned about .ssh/config file, in which you can customize parameters used for connecting to different hosts. The complete syntax is described in “man ssh_config”, here is just a few highlights:

Host <short hostname> Hostname <full hostname> Port port User user LocalForward 54320 localhost:5432 Dynamic Forward 9050

As simple as that: cat id_*.pub | ssh servername “cat >> ~/.ssh/authorized_keys”