Tadek’s Blog

I recently made a really stupid mistake. I wanted to do a simple variable assignment and run a command. What I did was the following:

VAR=value command $VAR

Obviously, this doesn’t work! As this is a single command, the shell does variable expansion __before__ the assignment takes place, so the variable is null (or whatever it was before). Obviously, the variable is set correctly within the command’s execution environment, but it’s too late then

What I should have done in this case was:

VAR=value; command $VAR

Sadly, the command did not complain about my mistake and I only realized it after a lost night of computation. Phew!

I recently wanted to play the [apple keynote](http://www.apple.com/iphone/keynote) and, to my disappointment, the keynote was silent. Tried quitting, restarting, putting my computer to sleep and waking up again…. the usual sort of things a computer literate does when something doesn’t work All in vein… Surprisingly, the QuickTime plugin in Safari seemed to work fine.

After a bit of googling, I found that the [solution](http://www.macfixitforums.com/showflat.php?Cat=&Board=tiger&Number=794260&page=0&view=collapsed&sb=5&o=31&fpart=1) was to __run the GarageBand and close it__. w00t it worked! “Any sufficiently advanced technology is indistinguishable from magic.” (A. Clarke).

* (playing with the camera) Cool. You can __change__ the format of the CF card… Whops….
* My CF card got corrupted.
* The photos I thought had been already uploaded to the gallery were deleted from both the CF card and the harddisk. Moreover, I already took 50 photos on the CF card after reformatting it.

Time and time again, I realize how precious my photos are, only after they are gone. The last time they were my PhD defense photos… Auch. Fortunately, they are gone, but not forever

You will find several tools for photo recovery on Google, but most of them either commercial or give you a only a “free preview”. Thinking about it, it’s a very good business model: people are very willing to swipe their card if they can see that they get their photos back. Should have written one as well Fortunately, there are also free ones, two of them I tried by myself.

[PC Inspector Smart Recovery](http://www.pcinspector.de/smart_media_recovery/welcome.htm) is an excellent and easy-to-use Windows application for photo recovery. It is extremely easy to use and does an amazing job. It is also completely free, but I think is fair to reward the author with a PayPal donation.

[PhotoRec](http://www.cgsecurity.org/wiki/PhotoRec) is a cross-platform program for photo recovery. In fact, it runs on Dos/Windows/Linux/BSD/Solaris/MacOSX, which is really impressive. It also supports a whole range of filesystems, including FAT/NTFS/Ext2/3/HFS+. It has a simple (n)curses based interface, which is a bit of a disappointment in the age of cool animated GUIs, but it’s also relatively easy to use. Also, it has an impressive range of configuration options. And it works on Linux and my Mac. I no longer need Windows! Simialrly, the program is really free (including the sourcre) and you can rewerd the author with a donation.

Both tools worked for me smoothly and (unlike some other commercial ones I tried which ran out of memory) managed to recover almost one thousand photos from my 4GB CF card, including the wanted defense photos I don’t know which of the tools is better and I recommend both (although I would lean slightly towards the PhotoRec as I don’t have Windows anymore). Also, if you don’t get what you’re looking for, you can try both of them (as a rule all image/disk recovery tools are read-only so you can try all of them many times with no risk).

One of my pet (a.k.a. [procrastination](http://www.thefreedictionary.com/procrastination)) projects has been to visualize my server logs using [Google Maps](http://maps.google.com). In fact, this has been my ‘procrastination hub’ giving me excuses to work on a variety of pet projects, including:

* playing with [Bro](http://bro-ids.org) and packaging Bro for Debian
* playing with Apache logs and importing them to the relational database
* playing with Bro logs and importing them to the relational database
* learning Python and Javascript
* playing with Google Maps
* writing a web application to visualize the collected logs on Google maps
* creating a webpage documenting all the above.

As with procrastination projects, they are by definition never complete. I do have something working now, and you can see it in action (works best in a [decent browser](http://www.mozilla.com/en-US/firefox/), but should show something in IE as well).

### GeoWebStats
Visualizing Apache logs on a webpage. Here are three links (it might take a while to load them for the first time, so please be patient):

* [Blog Visitors - last 7 days](http://plum.ibao.net/webstats/?vhost=tadek.pietraszek.org&regexpmatch=%2Fblog&regexpnomatch=%2Fblog%2Fwp-%7C%2Fblog%2Ffeed&cutoff=10&start=-7&cc_locked=vhost%2Cregexpmatch%2Cregexpnomatch%2Cstart%2Cend%2Cstriplevel&cc_hash=6049f98622a55ac5cb3b87dd535820d0&submit=1)
* [Gallery Visitors - last 7 days](http://plum.ibao.net/webstats/?vhost=gallery.ibao.net&regexpnomatch=%5E%2Fd%2F%5B0-9%5D%2B%7C%5E%2Fc%2F%7C%5E%2Findex.php%7C%2Ffavicon.ico&cutoff=5&start=-7&cc_locked=vhost%2Cregexpmatch%2Cregexpnomatch%2Cstart%2Cend%2Ccutoff%2Cstriplevel&cc_hash=e5366f30cae72af7a33f166394e96bc6&submit=1)
* [Homepage Visitors - last 7 days](http://plum.ibao.net/webstats/?vhost=tadek.pietraszek.org&regexpnomatch=%2Fblog%7C%2Ffavicon.ico%7C%5E%2Fimages&striplevel=2&start=-7&cc_locked=vhost%2Cregexpmatch%2Cregexpnomatch%2Cstart%2Cend%2Ccutoff%2Cstriplevel&cc_hash=992cd263f64643edf909c6eea6a9624e&submit=1)

The script is quite customizable (for example you can specify the regular expressions you want to filter on, group stuff) but for security resons those demo links are locked.

### GeoBroStats
Simiarly to GeoWebStats, GeoBroStats visualizes raw TCP/UDP conections based on Bro conection summaries (this might also take a while to load):

* [Today's SMTP connections that transfered 10kB+ (mostly spammers) ](http://plum.ibao.net/brostats/?service=smtp&cutoff=10&start=0&cc_locked=service%2Cport%2Cstate%2Ccutoff%2Cstart%2Cend&cc_hash=3f41b93fc42c40fe3d60920af7e207f5&submit=1)

The script is also quite customizable, but for security resons those demo links are locked.

Let me know what you think about it. I know that the user interface is very crude and needs some work. I have also almost finished GeoWebStat’s website, but knowing me, it will take a while

I recently had to write some Polish text on my MacBook Pro and discovered that the Polish keyboard is messed up. In fact, coming from a PC world I’ve always thought Mac’s keyboards are messed up (e.g., lack of Home/End PageUp/PageDown, which can be simulated by some weird and application-dependent two/three key combination, an almost completely useless Enter/Rename key, an annoying Eject key, which pressed accidentally generates an eject sound regardless whether you have something in your drive or not), but this time I got annoyed.

To give a bit of background, in Poland, we use nine additional letters, namely ęóąśłżźćń (and their uppercase counterparts) and historically typewriter’s keyboard had them allocated at the right side (where brackets and quotes are). Now, unless you’re a typewriter, this is not very useful (especially if you need the braces and quotes more often) and we have two Polish keyboard mappings: a typewriter’s keyboard and a programmer’s keyboard (with Polish letters generated with an Alt+<Latin letter>). As we have two z-derivatives: żź one of them is Alt+z (the more common ż) and the other is Alt+x (the less common ź).

Playing with my Mac I discovered that żź are swapped. I am not sure if there’s any rationale for it (apparently it was ok in OS9 and only changed in OSX), maybe it’s easier to press Alt+x (which gives a more commonly used character), in particular that on a PC it’s a right Alt, not the left one (in fact, I was trying to get it to be more ergonimic, I would remap the right Enter to Alt), but I found it confusing. To get a feeling what it’s like, imagine what if Apple replaced a Control key with Enter or PageUp with an eject button. Whops… they already did it. Imagine something else then

Doing a bit of research I found discovered I am not the only one annoyed with it. Somebody made a correct programmer’s keyboard and which can be downloaded from [here](http://www.srebrnysen.co.uk/e4aws/downloads/files/poprawiona_klawiatura_programisty_dla_OSX_10.4_v1.4.zip).
There are two versions: one replacing a system file and one installing a local keyboard for a user. I took the latter approach and it works great!

I’ve been using [bro](http://bro-ids.org) for quite a while on my server and consider is a great IDS. Actually, I’ve been using it mostly as a network analysis tool (connection summaries, tracking HTTP connections, analyzing headers, etc.), rather than an IDS itself, but I still think it’s great.

What has been bothering me most this time is that my cleanly-installed server with a proper package manager (I’m running Debian and I am very happy about it, regardless what some friends of mine say) is running a _service_ installed in my home directory in a screen. In fact, as the server’s uptime is on average half a year, it’s not such a big problem, but it really bothered me

Almost a half a year ago, I started Bro’s ‘Debianization’ process, as one of my many procrastination projects (a.k.a. pet project), but I haven’t been active (maybe now that I defended my thesis I don’t need to procrastinate so much? ). Now during the Christmas break I finally managed to (almost) finish it!

The whole job turned out to be more difficult than I’d thought, but it works now. Here’s a proof:

tadekp@plum:~$ apt-cache show bro
Package: bro
Version: 1.1d-1
Priority: optional
Section: net
Maintainer: Tadeusz Pietraszek <tadek@pietraszek.org>
Depends: libc6 (>= 2.3.2.ds1-21), libgcc1 (>= 1:3.4.1-3), libncurses5 (>= 5.4-1), libpcap0.7, libssl0.9.7, libstdc++5 (>= 1:3.3.4-1), c-shell
Architecture: i386
Filename: ./bro_1.1d-1_i386.deb
Size: 3061038
Installed-Size: 8916
MD5sum: 880901a64a7fc44766e4645f445799a6
Description: Network Intrusion Detection System (NIDS)
 Bro is an open-source, Unix-based Network Intrusion Detection System (NIDS)
 that passively monitors network traffic and looks for suspicious traffic.
 .
 Bro detects intrusions by comparing network traffic against a customizable
 set of rules describing events that are deemed troublesome. These rules
 might describe specific attacks (including those defined by signatures)
 or unusual activities (e.g., certain hosts connecting to certain services
 or patterns of failed connection attempts).
 .
 Bro uses a specialized policy language that allows a site to tailor Bro's
 operation, both as site policies evolve and as new attacks are discovered.
 If Bro detects something of interest, it can be instructed to either generate
 a log entry, alert the operator in real-time, execute an operating system
 command (e.g., to terminate a connection or block a malicious host
 on-the-fly). In addition, Bro's detailed log files can be particularly
 useful for forensics.

tadekp@plum:~$

tadekp@plum:~$ /etc/init.d/bro status
Bro is running (pid: 2859)
Autorestart: ON
Running since: Mon Jan  1 16:11:37 CET 2007
Bro Version: 1.1d
Active log suffix: plum.07-01-01_16.11.33
tadekp@plum:~$

The package is in __alpha__ stage now and I still get a few lintian errors (for example, the man page is missing), but otherwise is ok (even including the init.d scripts and checkpointing). If you’re interested in trying it out, please let me know.