Apache2.0 -> Apache 2.2 upgrade issues

March 4th, 2008

While upgrading apache 2.0 -> 2.2 I found that two configuration options have changed:

* `AuthAutoritative` -> `AuthBasicAuthoritative` (needs to be set to Off for mod_auth_pgsql to work).
* `AuthDigestFile` -> `AuthUserFile`

Plus there has been a log of changes in `apache2.conf`. I looked at the changes we made (keeping /etc/ in SVN comes in handy, in spite of occasional pain) and pasted them to the vanilla apache2.conf that came with 2.2.

WTH: eth0 got renamed to eth2 after an upgrade

March 4th, 2008

While upgrading a __remote server__ from sarge to etch including the new kernel, the server did not come up. After attaching a console (thanks Hetzner!) I found out that the network interface got mysteriously renamed to eth2!

After snooping around a bit, I found out that the culprit was udev, more specifically `/etc/udev/rules.d/z25_persistent-net.rules` which says:

 # This file was automatically generated by the /lib/udev/write_net_rules
 # program, probably run by the persistent-net-generator.rules rules file.
 #
 # You can modify it, as long as you keep each rule on a single line.
 # MAC addresses must be written in lowercase.
 # PCI device 0x1106:0x3065 (via-rhine)
 SUBSYSTEM=="net", DRIVERS=="?*", ATTRS{address}=="00:0c:76:af:2f:9d", NAME="eth0"

It also contained two entries for bogus eth0 and eth1 (usb dongle got identified as a network card?). After removing the and relabeling interfaces everything is back to normal now.

Patching debian packages

March 4th, 2008

While upgrading my server, I had to create a new version of Cyrus-SASL packages with the [crypt patch](http://frost.ath.cx/software/cyrus-sasl-patches/). This turned out to be more difficult as I thought as just patching the raw source would conflict with debian patches applied after that and the package building would fail. Fortunately, I learned about (dpatch)[http://www.tuxmaniac.com/blog/2008/01/25/dpatch-just-superb-a-short-how-to/], which is a standard mechanism for patching stuff in Debian.

Once I figured out how to do it, it tunred out to be very simple:

1. Unpack the original orig.tar.gz
2. Apply the Debian diff (this would create a debian subdirectory)
3. Edit changelog file to add a new release (so that we can keep track of it).
4. Run `dpatch-edit-patch fixing_foo` to create a patch. This will create a shell with mounted source package and any changes you make there will be included in the diff.
5. __Exit the shell without changing anything__. You will change things after making sure that the patch is applied in the correct order.
6. Edit `00list` to set the patch ordering.
7. Run `dpatch-edit-patch` again now, with all the previous patches applied.
8. Make the necessary changes.
9. Build the target package with `debuild-pbuilder` (this will download all the dependencies too).
10. Install the pakcages with `dpkg -i` (I could also create my private apt-repository for this. I wrote about in in a blog entry about pbuilder).
11. Pin the packages so that they will not get upgraded `echo “package hold” | dpkg –set-selection`

Saturday procrastination – how to recognize a good programmer

January 13th, 2008

I found this essay on Slashdot yesterday: [how to recognize a good programmer](http://www.inter-sections.net/2007/11/13/how-to-recognise-a-good-programmer/). Some of the comments were quite insightful, but many followed the following pattern.


"I am a good programmer myself and I match X% of the qualities you mentioned. Therefore, your article is excellent/good/not so good/crap (depending on the value of X) ;-)

I also read two other articles on the topic:

* [Confessions of a Terrible Programmer](http://blog.kickin-the-darkness.com/2007/09/confessions-of-terrible-programmer.html).
* [Where are Software Engieers of Tomorrow](http://www.stsc.hill.af.mil/CrossTalk/2008/01/0801DewarSchonberg.html).

md5^-1(hash)

November 22nd, 2007

I just came across this website: , , They have a large dictionaries of unsalted MD5/SHA passwords. I played with it for a little bit. It’s amazing and scary how little entropy there is in a simple dictionary-based passwords with few modifications! Another conclusion is that you should never use unsalted passwords in your programs.

sed and awk – my two old friends

September 30th, 2007

Writing some shell scripts I needed to do some a little fancier variable substitution than the standard shell offers. The heavyweight solution would be to write a perl one-liner, but this is, well…, heavyweight? ;-)

Here’s a couple of patterns I used:

  • --parameter=$(sed -re 's/ /,/g' -e 's/(^|,)/\1file:/g' <<<$INPUT) - replaces spaces with commas and prepends file to every file.
  • --parameter=$(awk '{split($0, a, /@/); printf "%s-?????-of-%05d", a[1], a[2]} <<<$INPUT)'
  • - replaces file@5 with file-?????-of-00005

  • --parameter=$(awk '{sub(/.*:/, ""); print $0}' <<<$INPUT) - removes everything before the colon.

Parsing parameters in bash – a getopt template

August 16th, 2007

Writing some bash scripts that parse command lines, I wrote this handy template with getopt. It is easy to apply even for simplest scripts.

OPTION_SPEC="help,flag1,flag2_params:"
PARSED_OPTIONS=$(getopt -n "$0" -a -o h --long $OPTION_SPEC -- "$@")
OPTIONS_RET=$?
eval set -- "$PARSED_OPTIONS"

# Parsing error or no flags
if [ $OPTIONS_RET -ne 0 ] || [ $# -le 0 ]; then
  usage;
  die;
fi

while [ $# -ge 1 ]; do
  case $1 in
    --help | -h )  usage; die;;
    --flag1 )  FLAG1=1;;
    --flag2_params )  shift; FLAG2_PARAMS="$1";;
    -- ) shift;;
    * ) echo "ERROR: unknown flag $1"; usage; die;;
  esac
  shift
done

No more unannotated $ns in my scripts!

Date of yesterday in bash?

August 16th, 2007

I recently had to hack a small shell script that would read files in a directory structure generated based on the date, something like 2007/08/16. The trick was that the script would look at yesterday’s file or files generated a few days ago.

A quick search on info and here’s the magic command

FILE="...$(date -d 'yesterday' +%Y/%m/%d)"

Interestingly, you can also use things like 3 days ago, next Monday, 2 months etc.
Cool!

Finding top-N items in a stream

July 4th, 2007

How to (approximately) generate a top-N items list without counting the number of occurrences of all instances?
Two interesting papers I found on the topic: and . I also somebody’s seminar [powerpoint presentation](http://www.math.tau.ac.il/~haimk/seminar03/FindingFrequentItemsInDataStreams.ppt) explaining it.

Listing socket/network connection owners on OSX

July 4th, 2007

While playing with OSX I was wondering how to find out all the networks connections a particular process owns. On Linux I’d use `netstat -p` for this, which does not work on OSX.

It turns out that the solution is quite simple – `lsof -i` does the job and works on both Linux and OSX. Two other useful commands:

lsof -ai -p PID # all connections/sockets owned by PID
lsof -i:PORT # lists all connections/sockets with a particular PORT.