I’ve been using bro for quite a while on my server and consider is a great IDS. Actually, I’ve been using it mostly as a network analysis tool (connection summaries, tracking HTTP connections, analyzing headers, etc.), rather than an IDS itself, but I still think it’s great.

What has been bothering me most this time is that my cleanly-installed server with a proper package manager (I’m running Debian and I am very happy about it, regardless what some friends of mine say) is running a service installed in my home directory in a screen. In fact, as the server’s uptime is on average half a year, it’s not such a big problem, but it really bothered me

Almost a half a year ago, I started Bro’s ‘Debianization’ process, as one of my many procrastination projects (a.k.a. pet project), but I haven’t been active (maybe now that I defended my thesis I don’t need to procrastinate so much? ). Now during the Christmas break I finally managed to (almost) finish it!

The whole job turned out to be more difficult than I’d thought, but it works now. Here’s a proof:

tadekp@plum:~$ apt-cache show bro
Package: bro
Version: 1.1d-1
Priority: optional
Section: net
Maintainer: Tadeusz Pietraszek <tadek@pietraszek.org>
Depends: libc6 (>= 2.3.2.ds1-21), libgcc1 (>= 1:3.4.1-3), libncurses5 (>= 5.4-1), libpcap0.7, libssl0.9.7, libstdc++5 (>= 1:3.3.4-1), c-shell
Architecture: i386
Filename: ./bro_1.1d-1_i386.deb
Size: 3061038
Installed-Size: 8916
MD5sum: 880901a64a7fc44766e4645f445799a6
Description: Network Intrusion Detection System (NIDS)
 Bro is an open-source, Unix-based Network Intrusion Detection System (NIDS)
 that passively monitors network traffic and looks for suspicious traffic.
 .
 Bro detects intrusions by comparing network traffic against a customizable
 set of rules describing events that are deemed troublesome. These rules
 might describe specific attacks (including those defined by signatures)
 or unusual activities (e.g., certain hosts connecting to certain services
 or patterns of failed connection attempts).
 .
 Bro uses a specialized policy language that allows a site to tailor Bro's
 operation, both as site policies evolve and as new attacks are discovered.
 If Bro detects something of interest, it can be instructed to either generate
 a log entry, alert the operator in real-time, execute an operating system
 command (e.g., to terminate a connection or block a malicious host
 on-the-fly). In addition, Bro's detailed log files can be particularly
 useful for forensics.


tadekp@plum:~$ 

tadekp@plum:~$ /etc/init.d/bro status
Bro is running (pid: 2859)
Autorestart: ON
Running since: Mon Jan  1 16:11:37 CET 2007
Bro Version: 1.1d
Active log suffix: plum.07-01-01_16.11.33
tadekp@plum:~$ 

The package is in alpha stage now and I still get a few lintian errors (for example, the man page is missing), but otherwise is ok (even including the init.d scripts and checkpointing). If you’re interested in trying it out, please let me know.

I used it some time ago, but have already forgotten how I did it and had to reinvent the wheel. Here it goes.

You sometimes need to start a background process in shell, which can die right away, in which case you want to know about it. Situations in which it is useful include init.d scripts, running some background processes… Here are two useful shell functions:

function pidactive () {
    #sends a signal which checks if the process is active (doesn't kill anything)
    kill -0 $1 2> /dev/null
    return
}

function pidkill () {
    echo "killing pid"
    kill $1 || return
    #adjust depending how long it takes to die gracefully
    sleep 1
    if pidactive $1; then
        #escalating
        kill -9 $1
    fi  
}

What I do in the script is the following:

<command> &
PID=$!
#wait for the process to startup or die...
sleep 5
if ! pidactive $PID; then
    wait $PID
    die "Command failed with $?"
fi

...
#kill the process before exiting
pidkill $PID

While trying to set up a restricted shared shell account for SVN access I read about an interesting feature of OpenSSH. From this section of the SVN manual I learned that svn supports the following syntax in $HOME/.ssh/authorized_keys

  command="program" TYPE KEY COMMENT

Program is a command that will be executed instead of a shell when connected and it also supports different configuration options. For example, if you want the account to be really restricted you may want to pass the following options no-port-forwarding,no-agent-forwarding,no-X11-forwarding,no-pty. In my case, I used the following line in the configuration file:

command="/usr/bin/svnserve -t --tunnel-user=tadekp -r /var/lib/svn/svncommon",no-port-forwarding,no-agent-forwarding,no-X11-forwarding,no-pty ssh-dss AAAAB3N.... tadekp@server

Just a few points to remember:

• many commands can execute shell internally, watch out for these
• the command can modify authorized_keys file, the best way to prevent it is to make it read only on the filesystem level

After resuming my geotaggin script (see this post), I decided to do something useful with it. We’re using gallery2 to store our photos and with a googlemap plugin, but found it useful only for displaying a single pointer per album (see here). For a more fine-grained selection we needed something else.

Therefore, I decided to write my own plugin (yeah, there are already two out there, why not write my own? ) and also learn Gallery2 API. The idea is to display a google map at the bottom of each photo, showing exactly where the photo was taken. Yes, it photo-specific and there’s only one pointer on the map. I find it nonetheless very useful.

Here’s a sample output (you can also admire the beautiful scenery of Melchsee ). The plugin adds a new “block” in the template (therefore can be configured using a standard block management tool in Gallery2).

The position of the current photo is always in the middle (although you can move the map around, change the map type, zoom in and out etc.). The changes you make are stored as session cookies, and preserved between consecutive photo loads. Also, the whole panel can be hidden to speed up load and only shown on demand (show map|hide map).

Any comments? suggestions? ideas?

The plugin is currently in alpha stage, I will release it in a week or two (I want to create a webpage for it as well). In the meantime, if you’re interested in trying it out, drop me a line

BTW: I also found out that when iPhoto edits a photo, it converts Exif from Intel to Motorola (little endian -> big endian). There was a bug in exifer used in gallery, which corrpted the tags. The patch is only two lines long and can be found here (I also emailed the author):

--- gps.inc.orig        2006-08-31 10:25:27.000000000 +0200
+++ gps.inc     2006-08-31 10:36:37.000000000 +0200
@@ -116,13 +116,24 @@
                        $minutes = GPSRational(substr($data,16,16),$intel);
                        $hour = GPSRational(substr($data,32,16),$intel);



/* now we need a hack, since the whole data has been flipped in :103


the order here is sec:min:hour. However, in the motorla mode the data



has not been flipped and the order is h:m:s. This breaks compatibility



with Motorola exif. (Tadek) */

if($intel==1)
                    $data = $hour+$minutes/60+$seconds/3600;
else
                                $data = $seconds+$minutes/60+$hour/3600;
        } else if($tag=="0007") { //Time
                $seconds = GPSRational(substr($data,0,16),$intel);
                $minutes = GPSRational(substr($data,16,16),$intel);
                $hour = GPSRational(substr($data,32,16),$intel);

                /* I guess the same HACK as above. Tadek */

if ($intel==1)
                    $data = $hour.":".$minutes.":".$seconds;
else
$data = $seconds.":".$minutes.":".$hour;
            } else {
                    if($bottom!=0) $data=$top/$bottom;
                    else if($top==0) $data = 0;

After almost a year after I had done some initial experiments with geotagging my images. I decided to pursue this idea further. I also realized that taking an approach “It’s simple let’s write in Perl” is good, but “Let’s see if somebody hasn’t done it yet” is even better

First, I recall I had some problems with reading GPS data and hacked gpstransfer to do this. In the meantime, I found out that pygarmin is a nice and working interface to Garmin GPSs, which works. More recently I discovered a much better and more versatile tool gpsbabel. Like the tower of Babel it really does speak all the languages (including plurality of GPS and very exotic formats (see here). The user interface is a bit weird and not all types of information (i.e. waypoints, tracks) are supported in all formats. Sadly, if the format is not supported, the program does transfer all the data (which takes a while) and prints nothing just to confuse you

After a bit of experimenting I discovered a magic formula:

gpsbabel -t -i garmin -f /dev/ttyS0 -o psitrex -F <track_file>

“psitrex” stands for KuDaTa PsiTrex format, however, as exotic as it sounds, it is just a comma separated format. I tried a couple of others (actually all of them in my version (1.2.7) and the only ones I found useful were:

• psitrex -> an easy to parse CSV format
• gpx -> produces results in XML
• nmea -> looks ok, but does not show the start and end of each segment, which is useful for determining whether to interpolate data or not.

I also revisited my geotagging script. First, as Diego pointed out there’s a nice tool on Mac to do this: iPhototoGoogleEarth, which unfortunately doesn’t work on intel-Macs yet. For geotagging they use GPSPhotoLinker, which in turn uses gpsbabel. So at the end we use the same backend tool.

In the meantime I revisited my geotagging script and made it a bit more useful adding a bunch of options, debugging it, etc. Now that I actually use it on my photos, I had to make sure that it actually works

Here it is:

$ ./geotag.pl 
ERROR: Need track file to proceed - use -t <trackfile>
Usage:
  ./geotag.pl -t <track_file> [-b] [-f] [-s <seconds>] [-a <seconds>] [-n <seconds>] [-z <tz>] files_to_process...


  Where:
  -b -> keep backup,
  -f -> force, overwrites an existing EXIF tag (or removes it)
  -s <seconds> -> time shift (in case of time discrepancies)
  -a <seconds> -> approximate non-continuous segements in the tracklog (default 300s)
  -n <seconds> -> don't approximate but take the closest segment (default 10800s)
  -z <tz> -> use the follwoing timezone for your camera (default current timezone)

  <track_file> can be best created using gpsbabel (http://www.gpsbabel.org):
   (e.g. Serial Garmin GPS: gpsbabel -t -i garmin -f /dev/ttyS0 -o psitrex -F <track file>)    
   at ./geotag.pl line 288.

Now what do I do with the script? Well… I wrote a gallery2 plugin to display google maps. See this post.

Proofreading my thesis I had the following problem: Find all occurrences of “alert processing system” (should be “alert-processing system”), which can span over multiple lines.

I found out (thanks Diego!) a number of ways of doing this.

1. Writing my own perl program and defining INPUT_RECORD_SEPARATOR ($/)

2. Using agrep, in which I can use -d ‘delim’ as the separator. The advantage of agrep is that is can also find mispelled words (with a certain editing distance).

3. Using pcregrep!

I chose the last solution as it is the simplest one:

pcregrep -Mi 'alert\s+processing\s+system' <file>

IT WORKS!

I figured out how to quickly make a backup of my server:

ssh user@server "sudo dump -0 -f - /bin /boot /dev /etc/ /home /initrd /lib /opt /root /sbin /usr /var" > backup<date>.dump

The only problem is that dump does not easily allow you to exclude stuff (so I had to resort to moving things I do not want to backup to /nobackup and not listing it. Also, this solution works if sudo doesn’t ask for a password (you may need to open another sudo for that). I had to use sudo as I do not allow direct remote root access.

Ever wanted to do bulk operations on files, similar to xargs, but much more flexible? For example:

• rename all files .jpeg to .jpg
• remove a prefix from many file names?
• add a suffix/extension?
• remove a prefix/suffx/extension?

Here’s a script I wrote:

!/bin/bash

#

Pattern-based file rename

#(c)2006 by Tadeusz Pietraszek

#

Usage:

./mv-pattern -i a *.txt <- delete all 'a's in file names

./mv-pattern -i jpeg -o jpg *.jpeg <- rename all jpegs to jpg

./mv-pattern -o .txt <- add an extension

./mv-pattern -i .txt <- remove an extension

#

if [ $# -eq 0 ]
then
  echo "Usage: basename $0 [-i ]  [-o ]  [-c ] files"
  exit -1;
fi

INPATTERN="";
OUTPATTERN="";
COMMAND="echo";

while getopts "i:o:c:" Option
do
  case $Option in
    i ) INPATTERN=$OPTARG;;
    o ) OUTPATTERN=$OPTARG;;
    c ) COMMAND=$OPTARG;;
    * ) echo "Unimplemented option chosen. Has to be one of -i -o -c"; exit -1;;
  esac
done

if [ -z "$INPATTERN" ]; then
  echo "No input pattern. Are you sure it's what you want?";

exit -1;

fi

if [ -z "$INPATTERN" ]; then
  echo "No input pattern. Are you sure it's what you want?";

exit -1;

fi

shift $(($OPTIND - 1))

Decrements the argument pointer so it points to next argument.

echo "in: $INPATTERN, out: $OUTPATTERN";

rename

for FILE in  "$@" ; do
    if [ -f $FILE ]; then
        NEWFILE=echo $FILE | sed -re "s/(.*)$INPATTERN(.*)/\1$OUTPATTERN\2/";
        if [ "$FILE" != "$NEWFILE" ]; then
            $COMMAND $FILE $NEWFILE;
        fi;
    fi;
done

exit 0

I recently discovered a potential security problem with the configuration of Linux-based masquerading firewalls, which (I must admit I fell prey of).

Suppose I a server with one outgoing IP address, which has two network cards (eth0 – outgoing link and eth1 – internal link). I want to set up masquerading of the internal network 192.168.0.0/24.

The way I would proceed is:

1. Enable packet routing:

   net.ipv4.ip_forward = 1
   

2. Configure a masquearing rule:

   iptables -t nat -A POSTROUTING -j MASQUERADE
   

3. Set up a firewall rules:

   iptables -A INPUT -i lo -j ACCEPT
   iptables -A INPUT -i eth1 -j ACCEPT
   iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
   iptables -A INPUT -p tcp -m multiport --dports <ports_here> -j ACCEPT
   iptables -P INPUT DROP
   

Does it work and is it secure? Yes… well… not really. Suppose the atacker is on the same network as I am and sets my host as his router. He will then be able to:

1. Connect to my internal hosts (which will be helpfully masqueraded).
2. Use my server to masquerade his connections (only on the allowed ports, but still).
3. If the server has another interface (e.g. running an IPsec tunnel to a restricted network, it’s getting really scary.

What can I do:

1. Control the FORWARD chain:

   iptables  -A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
   iptables -A FORWARD -i eth1 -j ACCEPT
   iptables -P FORWARD DENY
   

2. Block the incoming connections in iptables -A INPUT -p tcp -m multiport --dports <ports_here> -j ACCEPT by specifying the destination IP address. However, this may be tricky, if the server uses a DHCP address (otherwise you wouldn’t be using MASQUERADE in the first place).

3. Change masquerade so that it has an interface specified: iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE. This only partially solves the problem, as the packets will still be forwarded to internal networks.

I recently wanted to make small changes to EPS charts I had once generated. It seems that EPS-editing is not so simple, but i have at least two solutions that works:

1. Import the file into CorelDraw and save back as EPS. I had previously been importing EPS files there and the results were amazingly good (BTW: choose “interpreted EPS”, not the one decoded by the system driver!).

2. Use pstoedit to save it to a .fig file, use xfig and then export back as EPS.

   pstoedit -f "fig:-startdepth 999" <file>.eps <file>.fig
   

I tried the last option so far and in kindof worked. The resulting file looked ok, but the bounding box was somewhat changed (that was probably the problem in the original file). Also the resulting EPS file was 10 times bigger then the original one. Well… nothing is perfect. Wonder how the CorelDraw solution works…