In defence of unit testing

posted 05 February 2009

Bill Moorier says that unit testing is one of the least useful forms of testing. His thesis boils down to, basically, "unit testing checks a bunch of things you already know aren't wrong", because you've already written the tests. He says stuff like good logging, systems monitoring, and user reporting tools are significantly better at finding bugs. And he's right, but that doesn't mean unit testing is a waste of time.

Unit testing doesn't help you find bugs very often, it prevents you writing bugs in the first place.

I'm not an advocate of writing unit tests in advance of the software: the system changes and evolves as you build it, so unit tests you wrote at the start would be unlikely to work with the software you eventually ended up with. However, I'm a big fan of writing unit tests at the same time as your software. Every function or module you write, create a test for it to make sure it's done. This has a bunch of nice side-effects:

  1. Each module is at the very least smoke-tested
  2. The tests act as examples of how to use the modules, that other devs can look at.
  3. The unit tests check edge cases, not just the happy path

Smoke testing sound obvious, but it is amazing how often real software falls down the first time you use a new module. Developers leave out obvious things, like returning the value they just spent 100 lines of code calculating, or they call the key in their hash the wrong thing, or a dozen other simple mistakes that would not be spotted by a compiler.

Unit tests as documentation are even more useful. An enormous number of bugs in a large software development team are caused by one developer misunderstanding how to use code written by another. In real-world development nobody ever has time to write documentation, but having a working example is even better than documentation, because the other developer can just copy and paste the test code and modify it a little to make it do what she wants, saving everybody time.

And edge cases are another great one. In YAP we had a big push in the middle of last year to expand the code coverage of our unit tests, i.e. to make them cover edge cases. Once again, it was amazing how many times an "else" clause of a little-used "if", having never been hit before, would just fall over or throw execution-wrecking exceptions.

But the thing I like most about unit tests is that they provide certainty. Just finished a complex refactoring and not sure whether you got everything? Run the unit tests: now you know that anything that used to work still works. Has somebody changed a library you rely upon? Run your unit tests, and you know your code works against the latest dependencies. Another dev complaining that your code doesn't work the way you said it would? Run your unit tests, and you know that they must be using it wrong, because you've got a working example, right there.

Of course, in web development unit tests are tricky -- it's almost impossible to write reliable tests for front-end web development. Luckily, the growing popularity of the MVC pattern means you can write tests for your model, and usually for your controllers too, and that's two-thirds of your code. But I still think they're great, and totally worthwhile. Unit testing doesn't find bugs in the wild, it prevents them getting out the door in the first place.