Storing and plotting connection summaries in the database
Since a while I have been (experimentally) running Bro on my server collecting connection summaries. This is a very nice feature of Bro, generating really accurate TCP flows (including information whether the connection was terminated correctly or not and others). The big advantage of connection summaries (over e.g. NetFlow) is their high accuracy and yet compact representation (for over 2 months ov data, I have only 21MB (250k lines) of data.
I wrote a simple perl parser parsing Bro’s connection status and writing it to a relational database (in this case Postgres). The parser can also run in “query” mode, producing a comma/space separated data for easy visualization (using AfterGlow or even GnuPlot).
Here’s how I run my collection:
sudo bin/bro -i eth0 -f "host 85.10.194.212" conn
And parsing:
tail -f conn.log | ../conn2db2csv.pl -i -d "dbi:Pg:dbname=bro;host=localhost" -u bro -p <password>
And plotting:
./conn2db2csv.pl -q "select time::date,dport, sum(brecv) from conn where dip='85.10.194.212' group by 1,2 having(sum(brecv) >0) order by 1,2" -d "dbi:Pg:dbname=bro;host=localhost" -u bro -p <password>
This can be nicely used by gnuplot. For example to plot daily traffic on let’s say port 22:
./conn2db2csv.pl -q "select time::date, sum(brecv) from conn where dip='85.10.194.212' and dport = 22 group by 1 having(sum(brecv) >0) order by 1" -d "dbi:Pg:dbname=bro;host=localhost" -u bro -p <password> -s ' ' > testplot
gnuplot set timefmt "%Y-%m-%d %H:%M:%S" set xdata time plot 'testplot' using 1:3
January 20th, 2007 at 4:10 am
I have just started playing around with Bro. Very cool tool and seems promising for generating network traffic stats. But currently I don’t know how to collect information such as: packet interarrival mean, packet size mean. Tadek, since you are proficient with it, could you provide some tips ? Tks a lot. D.