Archive for the 'Personal' Category

Two friends: GeoWebStats and GeoBroStats – visualizing Apache and Bro logs with Google Maps

Tuesday, January 2nd, 2007

One of my pet (a.k.a. procrastination) projects has been to visualize my server logs using Google Maps. In fact, this has been my ‘procrastination hub’ giving me excuses to work on a variety of pet projects, including:

  • playing with Bro and packaging Bro for Debian
  • playing with Apache logs and importing them to the relational database
  • playing with Bro logs and importing them to the relational database
  • learning Python and Javascript
  • playing with Google Maps
  • writing a web application to visualize the collected logs on Google maps
  • creating a webpage documenting all the above.

As with procrastination projects, they are by definition never complete. I do have something working now, and you can see it in action (works best in a decent browser, but should show something in IE as well).

GeoWebStats

Visualizing Apache logs on a webpage. Here are three links (it might take a while to load them for the first time, so please be patient):

The script is quite customizable (for example you can specify the regular expressions you want to filter on, group stuff) but for security resons those demo links are locked.

GeoBroStats

Simiarly to GeoWebStats, GeoBroStats visualizes raw TCP/UDP conections based on Bro conection summaries (this might also take a while to load):

The script is also quite customizable, but for security resons those demo links are locked.

Let me know what you think about it. I know that the user interface is very crude and needs some work. I have also almost finished GeoWebStat’s website, but knowing me, it will take a while ;-)

Dr. Tadek

Friday, December 8th, 2006

Woohoo!

After 3.5 years of work, 0.5 year of writing, 5 months of waiting, 2 months of nerves, 2 weeks of getting really nervous and 1.5 hours of defense it finally came through!

That means that after I publish my thesis, I can be called Dr. Tadek ;-)

I feel a big relief to have successfully closed this chapter of my life. To celebrate this we opened a 1975 Colheita Port, which we have kept for over a year for this occasion (I’ve been saying that I will open it only if and when I graduate).

Now it’s time to do all the stuff I have be putting of until I finish and of course enjoying the first really free weekend!

Tadek’s new mobile: 076 394 2998

Wednesday, October 25th, 2006

After more than 6 years with my current S35 (and something like 3 years with a crappy Orange prepaid card) I purchased a proper mobile (K800i) with a contract. Personally, I’ve been reasonably happy with my old mobile, in spite of Annie’s hostility towards this seemingly innocent device (this may have something to do with the high-pitched sound it produces when the battery dies in the middle of the night) and friends’ teasing remarks (my favourite is: “Noooooo, I can’t believe your mobile is SIX years old!”… I wonder what they would say knowing how old my car is…), but I will definitely benefit from things like a built-in 3MPix camera, Bluetooth or Edge for data transfer.

More importantly, my new mobile number has changed and is now 076 394 2998 (076 FYI AXYU). My current mobile 078 857 3165 is no longer valid.

As a final remark, I got a sunrise relax plan, which means that I am charged by hours (something like 0.49CHF/hour or 0.05CHF for <5s) when making calls to the sunrise network or fixnet. Therefore, if I’m calling your sunrise or fixnet, I might be either extremely talkative or very succint. If you have another mobile, the latter is far more likely ;-)

Sailing the Greek islands – planning vacation ahead

Sunday, September 10th, 2006

Planning ahead the next vacation (haven’t had much vacation this year, so it’s nice to dream ;-) ), sailing in the Greek islands, here are some links I found:

I found a bunch of charter websites (just googling for “greece bareboat charter gets you a lot of results) but still need some time to sort them through. I also need to figure out which the boat I want. So far it looks that a 6-8 people boat is an expense on the order of 1500-2000 EUR/week. Also, you can get an early-booking discount until sometime in February.

Any experiences/suggestions/recommendations about chartering a boat in Greece?

Goodbye IBM, welcome Google

Wednesday, August 30th, 2006

After 3.5 years of work on my PhD and two months after submitting my thesis I left IBM Zurich Research Laboratory I left the Zurich Lab. Many thanks for the superb farewell lunch and all warm words!

Now, I am enojoying my week long vacation at home (or more precisely, trying to enojoy vacation while touching up on my MLJ submission. Still not done yet… grrr) and getting mentally ready for a new start (and my PhD defense in some time).

On Monday, I am starting a real job at Google in Zurich. I’m a bit anxious, but very much looking forward to it.

Wow. My blog is getting really personal this time… that’s second to last time, promise ;-)

Pusterla Elektronik

Tuesday, August 22nd, 2006

Ever wondered where to get some electronic components in Zurich and was unhappy with what quadruple “M” Migros offers in the hobby section? Or looking for a non-typical power supply for your favorite toy that just died? Or need some fancy cables? Or maybe just want to look at cool stuff? ;-)

I found a really nice electronic store in Zurich – Pusterla Elektronik to help you.

Thesis Submitted!

Wednesday, July 5th, 2006

After almost 3.5 years of work I submitted my PhD thesis today. I feel relieved, although I still haven’t fully realized this yet. It will come… On the funny side, now after having (hopefully) done most of the work, I am currently not even at the first out of 11-point Ablaufplan describing the quest of a poor PhD student. However, now that the machinery has been started, actionable items for me start at point 8 (“pass the defense”), which will take place some time in October ;-)

Finishing the blog entry, here’s a funny thing that happened to me on the train. I was carrying the a cardboard box, filled up with 5 copies of a nicely printed and bound dissertation. During the passport control (rarely happens, but today was my lucky day) a very bored customs officer got interested in what was in the box. The conversation followed like this.

Customs Officer: So, what is in this box?
Me: Papers.
C: How much are they worth?
M: Well… nothing? (I did not want to go into details of three-and-a-half years of pain or how much I was paid during that time)
C: Can I see?
M: Sure.
C: (opens the box takes one and starts browsing) Are they all the same?
M: Yes. Would you like to read? ;-)
C: (at this time I got a kind of look which tells you that you shouldn’t be joking with a customs official). So… were they printed in Switzerland?
M: Yes.
C: And you say they are worth “nothing”?
M: Exactly.
C: Why were they printed in Switzerland and are being brought here?
M: Because I work there?
C: Well…OK. Do you carry anything else that is also worth “nothing”?
M: No, I don’t think so.
(custom officer leaves)

On the perils of masquerading with Linux

Sunday, April 2nd, 2006

I recently discovered a potential security problem with the configuration of Linux-based masquerading firewalls, which (I must admit I fell prey of).

Suppose I a server with one outgoing IP address, which has two network cards (eth0 – outgoing link and eth1 – internal link). I want to set up masquerading of the internal network 192.168.0.0/24.

The way I would proceed is:

  1. Enable packet routing:

    net.ipv4.ip_forward = 1
    
  2. Configure a masquearing rule:

    iptables -t nat -A POSTROUTING -j MASQUERADE
    
  3. Set up a firewall rules:

    iptables -A INPUT -i lo -j ACCEPT
    iptables -A INPUT -i eth1 -j ACCEPT
    iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
    iptables -A INPUT -p tcp -m multiport --dports <ports_here> -j ACCEPT
    iptables -P INPUT DROP
    

Does it work and is it secure? Yes… well… not really. Suppose the atacker is on the same network as I am and sets my host as his router. He will then be able to:

  1. Connect to my internal hosts (which will be helpfully masqueraded).
  2. Use my server to masquerade his connections (only on the allowed ports, but still).
  3. If the server has another interface (e.g. running an IPsec tunnel to a restricted network, it’s getting really scary.

What can I do:

  1. Control the FORWARD chain:

    iptables  -A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
    iptables -A FORWARD -i eth1 -j ACCEPT
    iptables -P FORWARD DENY
    
  2. Block the incoming connections in iptables -A INPUT -p tcp -m multiport --dports <ports_here> -j ACCEPT by specifying the destination IP address. However, this may be tricky, if the server uses a DHCP address (otherwise you wouldn’t be using MASQUERADE in the first place).

  3. Change masquerade so that it has an interface specified: iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE. This only partially solves the problem, as the packets will still be forwarded to internal networks.

Anti-virus and anti-spam measures on my server

Tuesday, February 7th, 2006

After having thought about it for at least half a year and having researched the topic thoroughly for a good weekend, I finally got to implementing anti-spam and anti-virus measures on my server. It turned out to be more complex than I had initially thought (as always), but it seems to be working now.

To give a bit more background, I am running Postfix with Courier-IMAP and PostgreSQL as database backend. E-mail accounts reside in a virtual folder and have no corresponding Unix accounts.

I decided to use maildrop (I discussed Postfix and procmail issues here) and followed this tutorial, with the following exceptions:

  • I had to backport a few packages to sarge (wrote about it here).
  • I found out by trial and error that two packages courier-maildrop and maildrop have the same program working differently (essentially, maildrop from the maildrop package works, the other one doesn’t!)
  • I added a custom clamAV source to my sources.list files:

    deb http://ftp2.de.debian.org/debian-volatile sarge/volatile main
    
  • I wrote my own /etc/maildroprc

The idea is to have e-mail moved automatically to a folder containing spam if (and only if) such a folder exists. What I came up with is the following

This is the folder into which spam messages are delivered

SPAMFOLDER="$DEFAULT/.caughtspam/"

run the message through SpamAssassin

exception { xfilter "/usr/bin/spamc -u $LOGNAME" }

if the message is marked as spam AND SPAMFOLDER exists - deliver there

I have no idea how to check it other than executing [ -d ] in a shell

SPAMFOLDEROK=[ -d $SPAMFOLDER ]; echo $? if ( /^X-Spam-Flag:.*YES/ && $SPAMFOLDEROK == 0 ) { exception { to $SPAMFOLDER } }

What still needs to be done is:

  • automatic training on users’ emails (to enable per-user training)
  • inclusion of user-specific rules (still need to thnik about it a bit as it has serious security implications).

Useful links:

Postfix Virtual Mailboxes and Procmail Filtering

Sunday, February 5th, 2006

For some time I’ve been running an ISP-grade e-mail hosting system on our server (it might be a bit of an overkill, I know), using Postfix, and Courier IMAP and Postgres as database backend. This is not the topic of this post, but followed this tutorial while setting it up and changed some MySQL-specific things to Postgres (BTE: postfix wiki article also discusses this topic).

The system works fine, but what’s been on my mind is how to enable server-side e-mail filters (rules, etc.). A simple example could be to deliver spam messages directly to “spam” folder, but other things could also be interesting (e.g. rule based filtering, autoreplying, folder sorting, etc.). All in all, what I was looking for is “procmail for virtual mailboxes”.

The most obvious option mailbox_command = /usr/bin/procmail in main.cf doesn’t work, because it only refers to local delivery (done using local, not virtual command). Looking into this matter a bit further, here are the possible soulutions I found on the web:

Change virtual delivery to local

In this case postfix should just work, however you need to make sure that it delivers mail to correct mailboxes (it’s not trivial). Also, if users have per-user procmail files, they can probably easily access each others mailboxes (essentially no file system-level permissions here).

This post and the followup discussion discuss this topic (in short, setting virtual_transport=local or virtual_transport=postfix is not such an easy solution).

There’s an interesting discussion here (check all the e-mails) and one post suggests virtual=local. Again, I’m not sure how well it works. Essentially, the problem is that procmail is not aware of users stored in the database.

Change virtual delivery to maildrop

Maildrop can connect to the database and check user account there (this option is supposedly not enabled in the debian package).

Links:

Use Courier’s MTA and procmail

This site briefly mentions how to enable it. Haven’t tried it.


To summarize, I think that the most reliable option is to use maildrop with database support. I will give it a try…