Unit testing is great - but don't forget about configuration testing
Recently a friend of mine pointed out that I had an error on one of my pages. It took me almost 45 minutes to figure out what was happening. I wasn’t able to reproduce the defect in my development environment. The version of Python installed was exactly the same. I tried executing the problematic piece of code on the production server and it did not reproduce the problem. All of my unit tests passed… I was at a loss as to the source of the problem.
It just so happened that a few months ago I had turned on a particular configuration option in Apache that influenced the way Python works. The reason why I wasn’t able to reproduce the problem in dev was because it does not use a production configuration. The reason why I wasn’t able to reproduce the problem using a Python interpreter is because it doesn’t care about how mod_python works. Once I figured out what was wrong the fix was very simple, in fact all I had to change were two letters.
The moral of the story is: Testing is good, unit testing is great, but don’t forget to test your configuration :)