Unit Tests, Heroes and Duct Tape

Recently, a friend twittered about an article by Joel Spolsky entitled The Duct Tape Programmer:

OMG, Spolsky is praising his hero--WHO DOESN'T LIKE UNIT TESTS! Can it be that Spolsky doesn't either?

I read the article and it could be that red flags go off in my head when I see generalities, or it could be because I was at a seminar on project management last week, or it could be that my outlook on things has changed in my old age, but I couldn't help but think of all the hidden assumptions.

Unit Tests

So, why do people write unit tests? What's the point? Are they just wasting their time? I mean, it's writing twice the code for the same functionality, right? Not quite. People write unit tests because unit tests are a useful tool that helps them accomplish one of their goals: to write reliable, maintainable software. How do unit tests obtain this? By having tests that show that the code does what it's supposed to do, they have confidence that the code continues to do what it should as changes are introduced because they can simply run the test suite to verify. And when tests that previously worked fail, they know they've introduced a bug in their changes and can quickly isolate and fix it.

The Test Driven Development (TDD) and Behavior Driven Development (BDD) people have brought these ideas to the front of the development cycle. If tests give you confidence in your code, why not write them first? Then as you develop your software, you can always and from the beginning have confidence that it does what it's supposed to do. The main difference between TDD and BDD is that with BDD the "tests" are expressed more in the language of the customer's problem domain than the language of the programmer. And thus there's more programmatic scaffolding to interpret the "tests". But the basic mechanisms are the same.

[It seems to me though that you had better pay really close attention to your design to avoid code churn where you may repeatedly write tests, then write implementations, but later delete both because that code ultimately didn't fit into your design. Sometimes you have to write code to vet aspects of a design before you know that's the design you're going to go with. This exploratory programming doesn't need tests up front IMHO]

In any case, at the end of the software project you have 2 sets of software: the software that you're going to provide to your customer and a set of unit tests that prove (to yourself and other programmers but not necessarily the customer) that the software does the things that it is supposed to do.

This is very similar to how scientific advancements are made. A scientist performs an experiment that gives some novel result. Then he publishes how he did it so that other scientists may reproduce his work. If others can not reproduce his work, then there may be something fishy going on or maybe the experiment was flawed in some way. If other people can reproduce his work, then our confidence in the validity of the experiment and its result grows. Unit tests are experimental proof that the software does what it is supposed to do (at least with regard to the things that are expressed as unit tests).

That's also why you find that code coverage tools are popular with people who write tests for their software. The tests may not exercise every bit of code in the system and a code coverage tool tells which subroutines or expressions or statements are executed. The more code that's exercised when running the tests, the more confidence that the entire software is operating as it should.

Having many unit tests that execute a high percentage of the code add up to show that statistically, the software performs as expected.

On Heroes

Spolsky's hero from the article is Jamie Zawinski (of xemacs and netscape fame) and is actually quoted as saying (in regards to unit tests) "Given a leisurely development pace, that’s certainly the way to go." So it's not so much that he doesn't like unit tests, as that he doesn't like unit tests for a particular type of development: get it out the door quick before the deadline.

But that's also a very short-sighted view of software development.

Does the programmer's obligation to the software end upon delivery to the customer? Sometimes. If, like Jamie Zawinski, your job is to get the product out *now*. Before the next guy. Before you're obsolete. Before the deadline that the sales people gave the client. Then, yes. Or if you're a mercenary programmer, it certainly does. You write some software that does a task, you give it to the customer and you're done. You're out of there. You're on to the next customer.

However, if you're not a mercenary programmer, then you're probably more interested in developing relationships. Sometimes it's a relationship with your customers; as when you develop a custom application for a customer and continue performing enhancements and upgrades long after you've delivered the initial product. Sometimes it's a relationship with your software; as when you're working on a software product that you wish to sell to the masses. Either way, you will invest more time and energy revisiting programs you've written in the past.

If you're a hot-shot programmer (HSP), you can write perfect code. You don't need tests because you're so meticulous and careful and just plain good at programming that you quickly find bugs and squash them. That's just how you do things. Jamie Zawinski is probably one of these people. Donald Knuth is famously one of these people. He is so confident in the perfection of his work (whether code or print) that he gives out monetary rewards that double as each bug is found.

And that's fine ... if you believe that HSPs are infallible and if the only programmer who will ever touch the code is an HSP or if all of the programmers that ever touch the code are HSPs. From the point of view of an HSP it doesn't matter if anyone else touches the code. Because as an HSP, you know that if you touch the code, all will be made well. But that's often not reality. Reality is that there are probably different programmers of varying skill that will ultimately touch the code over its lifetime.

From the point of view of the company that employs the HSP, they would be taking a risk not having some form of code verification as changes are introduced. Because from the employer's stand point, that HSP might be the only one on the team so other non-HSPs will definitely touch the code and they won't be as meticulous or careful. The HSP might not work at the company forever, and when he leaves, who's to say that the next guy the company hires will be an HSP or not? Odds are fairly good that he (or she) won't be. See Truck Number or Bus Number for more details on the risks involved.

Tools like unit tests help mitigate against some of these risks by codifying the correctness of the software into a test suite that can be run at any time by anyone.

On Context

These are some of the things that went through my head, but all of this is probably tangential to the point of Spolsky's article which really seems like a reaction against those people that have taken some good ideas and turned them into some sort of religion. People that latch on to the latest set of buzz-word technology and apply it everywhere. People that think that units tests are more important than the software they test. (If you're one of these people, tell me, which does the customer want?)

Spolsky praises Zawinski for his Worse is Better approach. I do too. Because it's very pragmatic. Write something useful, then let people use it. It may not be perfect, but you can iteratively refine it as needed. And do you refine it until perfection? Hell no! The perfect is the enemy of the good. Voltaire knew this 230-something years ago. Perfection is a chimera that leads you madness. The point is to just get useful things. So, you do the simplest thing possible to make the software more useful and that's it. Stop trying to predict the future. You can't. Stop trying to design for things that you imagine will be useful. Only design/write things that are actually useful now.

Oddly, these are the same ideas I hear from the practitioners of agile software development. The only problem is that when you name something, you can't be sure the name will mean what you want it to mean. I think these days when people say "agile", what's heard is "the Agile religion" rather than "that bunch of useful ideas".

Conclusion

So, with regard to unit tests ... a man with two clocks never knows what time it is, but a man with thousands of clocks invokes the law of large numbers and has a good statistical chance of knowing the time within a sufficiently small epsilon. (I hope everyone can draw the analogy :-) And if you're that man with thousands of clocks, don't get caught up in making clocks for the sake of making clocks.

Ultimately, who cares if Spolsky or Zawinski or I like or don't like unit tests? At the end of the day, to be a good developer or development company you have to find out what works for you or your team for a particular problem. Sometimes that might involve unit tests; sometimes that might mean you're a mercenary programmer; sometimes that might mean you're doing TDD. Or maybe something else entirely. Use what works.

And that's all I have to say about that. :-)