Unit testing is great - but don't forget about configuration testing

Recently a friend of mine pointed out that I had an error on one of my pages.  It took me almost 45 minutes to figure out what was happening.  I wasn’t able to reproduce the defect in my development environment.  The version of Python installed was exactly the same.  I tried executing the problematic piece of code on the production server and it did not reproduce the problem.  All of my unit tests passed… I was at a loss as to the source of the problem.

It just so happend that a few months ago I had turned on a particular configuration option in Apache that influenced the way Python works.  The reason why I wasn’t able to reproduce the problem in dev was because it does not use a production configuration.  The reason why I wasn’t able to reproduce the problem using a Python interpreter is because it doesn’t care about how mod_python works.  Once I figured out what was wrong the fix was very simple, in fact all I had to change were two letters.

The moral of the story is: Testing is good, unit testing is great, but don’t forget to test your configuration  :)