Tadek’s Blog

Python (at least up to version 2.4) doesn’t have a much needed ?: operator. Here’s how you can hack it yourself (I’m not sure if I would use it, except for very special situations, though).

# if-then-else function taking functions as arguments.
def ite(condition, true, false):
    if condition:
        return true()
    else:
        return false()

# Call it like this.
ite(condition, lambda:arg_true, lambda:arg_false);


# Just to check if it really works
def fun(arg):
    print "Evaluated "+str(arg)
    return arg

print ite(0, lambda:fun(1), lambda:fun(0));
print ite(1, lambda:fun(1), lambda:fun(0));

I don’t like programming in bash but it does make some things very simple. Unfortunately, it is not obvious how to do some simple things like iterating through an array

Assuming that we have an array ITEMS=( a b c d ), we can use a ${ITEMS[@]} construct to iterate through all the elements in a for loop:

ITEMS=( a b c d )
for ITEM in ${ITEMS[@]}; do
    echo $ITEM
done

BTW: I found this pattern here.

Sometimes I have a list of something in a file, one line per item and want to convert it to a comma(collon,…)-separated line (with no trailing separator of course) that can be used as a command-line parameter to some other tool. A perl oneliner comes in handy:

`perl -e '@_=<STDIN>; chomp(@_); print join(";",@_);' < data_file`

1. Plotting a matrix of small charts on one page:

   #setup a (close to) square matrix for plotting
   matrix_par <- function(numplots, ...) {
      par(mfrow = c(ceiling(numplots / floor(sqrt(numplots))), floor(sqrt(numplots)) ),...);
   }
   

2. Barplots with confidence intervals

   library(gplots);
   #plot a barplot with confidence intervals
   barplot_ci <- function (y, y_ci, ...) {
       barplot2(y, ci.l=y-y_ci, ci.u=y+y_ci, plot.ci=TRUE, ...);
   }
   

3. Filtering certain columns in a data frame:

   restrict = c("val1", "val2", "val3");
   x = read.table(...);
   if (is.vector(restrict)) 
       x = x[ x$V1 %in% restrict, ];
   

4. Cummulative series with sapply:

   sum_x = sapply(seq(x), function(i) { sum(x[1:i]); });
   

5. Processing multiple files in a directory and generating output files

   #execute for all input files
   input_files = list.files(".","\.ssv");
   for(input_file in input_files) {
      #convert the filename according to this regexp
      output_file = sub("\.ssv","\.new_extension",input_file);
   }
   

While playing with tcpdump and bro I noticed that all outgoing packets have garbled TCP checksums. The diagnosis was simple: TCP checksum offloading. The real question was how to disable it

After a big of googling I found the magic command:

sysctl -w net.link.ether.inet.apple_hwcksum_tx=0
sysctl -w net.link.ether.inet.apple_hwcksum_rx=0

There’s one catch though. Disabling TCP checksum offloading effectively corrupts TCP checksum on the lo interface and, as you can well imagine, things stop working. In particular, you will not be able to start any graphical applications anymore… So it looks either playing with the network or working… There’s no free lunch.

At the end the solution with bro turned out to be simpler – you can tell it to ignore checksums with -C flag as everything works as on my linux box.

BTW: my favourite bro command

bro -C -i en0 conn http-request http-reply [http-headers] [http-body]

writes connection summaries to conn.log and http-related stuff to http.log. Depending on the amount of details you need, you can have: only requests, requests plus information about the results (HTTP return code and the size of the result). Additional modules record the browser headers or even the entire page loaded.

I used it some time ago, but have already forgotten how I did it and had to reinvent the wheel. Here it goes.

You sometimes need to start a background process in shell, which can die right away, in which case you want to know about it. Situations in which it is useful include init.d scripts, running some background processes… Here are two useful shell functions:

function pidactive () {
    #sends a signal which checks if the process is active (doesn't kill anything)
    kill -0 $1 2> /dev/null
    return
}

function pidkill () {
    echo "killing pid"
    kill $1 || return
    #adjust depending how long it takes to die gracefully
    sleep 1
    if pidactive $1; then
        #escalating
        kill -9 $1
    fi  
}

What I do in the script is the following:

<command> &
PID=$!
#wait for the process to startup or die...
sleep 5
if ! pidactive $PID; then
    wait $PID
    die "Command failed with $?"
fi

...
#kill the process before exiting
pidkill $PID