Archive for the 'Linux' Category

Talking TLS to SMTP

Tuesday, December 2nd, 2008

I recently wanted to test TLS with SMTP. I followed instructions on http://qmail.jms1.net/test-auth.shtml and got it to work in less than 30s ;-)

In short:

perl -MMIME::Base64 -e 'print encode_base64("\000user\000password")'
openssl s_client -starttls smtp -crlf -connect <ip>:<port>
auth <auth_command>
mail from:<tadek@pietraszek.org>
rcpt to:<tadek@pietraszek.org>
data
...
.

Interestingly, when I tried typing RCPT s_client would interpret it as “renegotiate”, which confused me a bit, but you can inhibit it with --quiet or type it in lowercase like I did ;-)

T.

xdu: analyzing disk-space usage

Thursday, November 27th, 2008

I recently started getting nagmails about running out of quota on my home directory. Being a very messy user, I had no idea where the space went. I tried playing with du manually, but it’s much easier with xdu:

sudo apt-get install xdu
du > /tmp/blah
xdu -n /tmp/blah

Installing SVN with Apache2 support on my server.

Thursday, March 6th, 2008

I just enabled SVN access on my server through the web interface. It was quite easy and, since we have a Postgres DB authentication, there’s no need to edit inconvenient password files ;-)

All I had to do was to enable the SVN module: link dav_svn.{load|conf} in mods-enabled and add the following line to the SSL-ed vhost.

<Location "/svn/foo">
   SVNPath /var/lib/svn/foo
   Dav svn
  <LimitExcept OPTIONS GET>
    Require user user@example.com user2@example.com
  </LimitExcept>
</Location>

Since the SSL-ed vhost already requires authentication, I didn’t have to change anything. I also had to create an SVN repository svnadmin create --fs-type fsfs /var/lib/svn/foo and change the permissions to www-data.

The checkout command is:

svn –username user@example.com –password my_secret_password co https://my.example.com/svn/foo

SVN caches the username and password, so any further operations are done without prompting you for it. If you don’t like it, you can disable it with --no-auth-cache.

Finally, one annoying thing. Initially, I would just try to connect without –username and SVN would first try my Unix user name and then ask for it. Unfortunately, some (but not all) users I tried in this way would get a mysterious:

svn: PROPFIND request failed on '/svn/foo'
svn: PROPFIND of '/svn/foo': authorization failed (https://example.com)

WTH?

Apache2.0 -> Apache 2.2 upgrade issues

Tuesday, March 4th, 2008

While upgrading apache 2.0 -> 2.2 I found that two configuration options have changed:

  • AuthAutoritative -> AuthBasicAuthoritative (needs to be set to Off for mod_auth_pgsql to work).
  • AuthDigestFile -> AuthUserFile

Plus there has been a log of changes in apache2.conf. I looked at the changes we made (keeping /etc/ in SVN comes in handy, in spite of occasional pain) and pasted them to the vanilla apache2.conf that came with 2.2.

WTH: eth0 got renamed to eth2 after an upgrade

Tuesday, March 4th, 2008

While upgrading a remote server from sarge to etch including the new kernel, the server did not come up. After attaching a console (thanks Hetzner!) I found out that the network interface got mysteriously renamed to eth2!

After snooping around a bit, I found out that the culprit was udev, more specifically /etc/udev/rules.d/z25_persistent-net.rules which says:

 # This file was automatically generated by the /lib/udev/write_net_rules
 # program, probably run by the persistent-net-generator.rules rules file. 
 #
 # You can modify it, as long as you keep each rule on a single line.
 # MAC addresses must be written in lowercase.
 # PCI device 0x1106:0x3065 (via-rhine)
 SUBSYSTEM=="net", DRIVERS=="?*", ATTRS{address}=="00:0c:76:af:2f:9d", NAME="eth0"

It also contained two entries for bogus eth0 and eth1 (usb dongle got identified as a network card?). After removing the and relabeling interfaces everything is back to normal now.

Patching debian packages

Tuesday, March 4th, 2008

While upgrading my server, I had to create a new version of Cyrus-SASL packages with the crypt patch. This turned out to be more difficult as I thought as just patching the raw source would conflict with debian patches applied after that and the package building would fail. Fortunately, I learned about (dpatch)[http://www.tuxmaniac.com/blog/2008/01/25/dpatch-just-superb-a-short-how-to/], which is a standard mechanism for patching stuff in Debian.

Once I figured out how to do it, it tunred out to be very simple:

  1. Unpack the original orig.tar.gz
  2. Apply the Debian diff (this would create a debian subdirectory)
  3. Edit changelog file to add a new release (so that we can keep track of it).
  4. Run dpatch-edit-patch fixing_foo to create a patch. This will create a shell with mounted source package and any changes you make there will be included in the diff.
  5. Exit the shell without changing anything. You will change things after making sure that the patch is applied in the correct order.
  6. Edit 00list to set the patch ordering.
  7. Run dpatch-edit-patch again now, with all the previous patches applied.
  8. Make the necessary changes.
  9. Build the target package with debuild-pbuilder (this will download all the dependencies too).
  10. Install the pakcages with dpkg -i (I could also create my private apt-repository for this. I wrote about in in a blog entry about pbuilder).
  11. Pin the packages so that they will not get upgraded echo "package hold" | dpkg --set-selection

sed and awk – my two old friends

Sunday, September 30th, 2007

Writing some shell scripts I needed to do some a little fancier variable substitution than the standard shell offers. The heavyweight solution would be to write a perl one-liner, but this is, well…, heavyweight? ;-)

Here’s a couple of patterns I used:

  • --parameter=$(sed -re 's/ /,/g' -e 's/(^|,)/\1file:/g' <<<$INPUT) - replaces spaces with commas and prepends file to every file.
  • --parameter=$(awk '{split($0, a, /@/); printf "%s-?????-of-%05d", a[1], a[2]} <<<$INPUT)'
  • - replaces file@5 with file-?????-of-00005
  • --parameter=$(awk '{sub(/.*:/, ""); print $0}' <<<$INPUT) - removes everything before the colon.

Listing socket/network connection owners on OSX

Wednesday, July 4th, 2007

While playing with OSX I was wondering how to find out all the networks connections a particular process owns. On Linux I’d use netstat -p for this, which does not work on OSX.

It turns out that the solution is quite simple – lsof -i does the job and works on both Linux and OSX. Two other useful commands:

lsof -ai -p PID    # all connections/sockets owned by PID
lsof -i:PORT       # lists all connections/sockets with a particular PORT.

Link: Surviving traffic storms with WordPress

Wednesday, May 2nd, 2007

Interesting link on surviving traffic storms with WordPress: not that I currently need it, but maybe in the future… ;)

In a nutshell:

  1. fine tuning of Apache (adjusting #processes, keep alives and ListenBacklog to values that match your machine’s constraints).
  2. fine tuning of MySQL query caching
  3. installing WP-Cache plugin + adaptive switching on of WP-cache plugin (only in heavy-load condition)
  4. disabling some plugins (the ones that take up a lot of resources)
  5. enabling Squid caching for static content.

Two friends: GeoWebStats and GeoBroStats – visualizing Apache and Bro logs with Google Maps

Tuesday, January 2nd, 2007

One of my pet (a.k.a. procrastination) projects has been to visualize my server logs using Google Maps. In fact, this has been my ‘procrastination hub’ giving me excuses to work on a variety of pet projects, including:

  • playing with Bro and packaging Bro for Debian
  • playing with Apache logs and importing them to the relational database
  • playing with Bro logs and importing them to the relational database
  • learning Python and Javascript
  • playing with Google Maps
  • writing a web application to visualize the collected logs on Google maps
  • creating a webpage documenting all the above.

As with procrastination projects, they are by definition never complete. I do have something working now, and you can see it in action (works best in a decent browser, but should show something in IE as well).

GeoWebStats

Visualizing Apache logs on a webpage. Here are three links (it might take a while to load them for the first time, so please be patient):

The script is quite customizable (for example you can specify the regular expressions you want to filter on, group stuff) but for security resons those demo links are locked.

GeoBroStats

Simiarly to GeoWebStats, GeoBroStats visualizes raw TCP/UDP conections based on Bro conection summaries (this might also take a while to load):

The script is also quite customizable, but for security resons those demo links are locked.

Let me know what you think about it. I know that the user interface is very crude and needs some work. I have also almost finished GeoWebStat’s website, but knowing me, it will take a while ;-)