While playing with OSX I was wondering how to find out all the networks connections a particular process owns. On Linux I’d use netstat -p for this, which does not work on OSX.

It turns out that the solution is quite simple - lsof -i does the job and works on both Linux and OSX. Two other useful commands:

lsof -ai -p PID    # all connections/sockets owned by PID
lsof -i:PORT       # lists all connections/sockets with a particular PORT.

Interesting link on surviving traffic storms with Wordpress: not that I currently need it, but maybe in the future…

In a nutshell:

1. fine tuning of Apache (adjusting #processes, keep alives and ListenBacklog to values that match your machine’s constraints).
2. fine tuning of MySQL query caching
3. installing WP-Cache plugin + adaptive switching on of WP-cache plugin (only in heavy-load condition)
4. disabling some plugins (the ones that take up a lot of resources)
5. enabling Squid caching for static content.

One of my pet (a.k.a. procrastination) projects has been to visualize my server logs using Google Maps. In fact, this has been my ‘procrastination hub’ giving me excuses to work on a variety of pet projects, including:

• playing with Bro and packaging Bro for Debian
• playing with Apache logs and importing them to the relational database
• playing with Bro logs and importing them to the relational database
• learning Python and Javascript
• playing with Google Maps
• writing a web application to visualize the collected logs on Google maps
• creating a webpage documenting all the above.

As with procrastination projects, they are by definition never complete. I do have something working now, and you can see it in action (works best in a decent browser, but should show something in IE as well).

GeoWebStats

Visualizing Apache logs on a webpage. Here are three links (it might take a while to load them for the first time, so please be patient):

• Blog Visitors - last 7 days
• Gallery Visitors - last 7 days
• Homepage Visitors - last 7 days

The script is quite customizable (for example you can specify the regular expressions you want to filter on, group stuff) but for security resons those demo links are locked.

GeoBroStats

Simiarly to GeoWebStats, GeoBroStats visualizes raw TCP/UDP conections based on Bro conection summaries (this might also take a while to load):

• Today’s SMTP connections that transfered 10kB+ (mostly spammers)

The script is also quite customizable, but for security resons those demo links are locked.

Let me know what you think about it. I know that the user interface is very crude and needs some work. I have also almost finished GeoWebStat’s website, but knowing me, it will take a while

I’ve been using bro for quite a while on my server and consider is a great IDS. Actually, I’ve been using it mostly as a network analysis tool (connection summaries, tracking HTTP connections, analyzing headers, etc.), rather than an IDS itself, but I still think it’s great.

What has been bothering me most this time is that my cleanly-installed server with a proper package manager (I’m running Debian and I am very happy about it, regardless what some friends of mine say) is running a service installed in my home directory in a screen. In fact, as the server’s uptime is on average half a year, it’s not such a big problem, but it really bothered me

Almost a half a year ago, I started Bro’s ‘Debianization’ process, as one of my many procrastination projects (a.k.a. pet project), but I haven’t been active (maybe now that I defended my thesis I don’t need to procrastinate so much? :-)). Now during the Christmas break I finally managed to (almost) finish it!

The whole job turned out to be more difficult than I’d thought, but it works now. Here’s a proof:

tadekp@plum:~$ apt-cache show bro
Package: bro
Version: 1.1d-1
Priority: optional
Section: net
Maintainer: Tadeusz Pietraszek <tadek@pietraszek.org>
Depends: libc6 (>= 2.3.2.ds1-21), libgcc1 (>= 1:3.4.1-3), libncurses5 (>= 5.4-1), libpcap0.7, libssl0.9.7, libstdc++5 (>= 1:3.3.4-1), c-shell
Architecture: i386
Filename: ./bro_1.1d-1_i386.deb
Size: 3061038
Installed-Size: 8916
MD5sum: 880901a64a7fc44766e4645f445799a6
Description: Network Intrusion Detection System (NIDS)
 Bro is an open-source, Unix-based Network Intrusion Detection System (NIDS)
 that passively monitors network traffic and looks for suspicious traffic.
 .
 Bro detects intrusions by comparing network traffic against a customizable
 set of rules describing events that are deemed troublesome. These rules
 might describe specific attacks (including those defined by signatures)
 or unusual activities (e.g., certain hosts connecting to certain services
 or patterns of failed connection attempts).
 .
 Bro uses a specialized policy language that allows a site to tailor Bro's
 operation, both as site policies evolve and as new attacks are discovered.
 If Bro detects something of interest, it can be instructed to either generate
 a log entry, alert the operator in real-time, execute an operating system
 command (e.g., to terminate a connection or block a malicious host
 on-the-fly). In addition, Bro's detailed log files can be particularly
 useful for forensics.


tadekp@plum:~$ 

tadekp@plum:~$ /etc/init.d/bro status
Bro is running (pid: 2859)
Autorestart: ON
Running since: Mon Jan  1 16:11:37 CET 2007
Bro Version: 1.1d
Active log suffix: plum.07-01-01_16.11.33
tadekp@plum:~$ 

The package is in alpha stage now and I still get a few lintian errors (for example, the man page is missing), but otherwise is ok (even including the init.d scripts and checkpointing). If you’re interested in trying it out, please let me know.

I used it some time ago, but have already forgotten how I did it and had to reinvent the wheel. Here it goes.

You sometimes need to start a background process in shell, which can die right away, in which case you want to know about it. Situations in which it is useful include init.d scripts, running some background processes… Here are two useful shell functions:

function pidactive () {
    #sends a signal which checks if the process is active (doesn't kill anything)
    kill -0 $1 2> /dev/null
    return
}

function pidkill () {
    echo "killing pid"
    kill $1 || return
    #adjust depending how long it takes to die gracefully
    sleep 1
    if pidactive $1; then
        #escalating
        kill -9 $1
    fi  
}

What I do in the script is the following:

<command> &
PID=$!
#wait for the process to startup or die...
sleep 5
if ! pidactive $PID; then
    wait $PID
    die "Command failed with $?"
fi

...
#kill the process before exiting
pidkill $PID