Saturday, September 26, 2009

Unit Tests, Heroes and Duct Tape

Recently, a friend twittered about an article by Joel Spolsky entitled The Duct Tape Programmer:
OMG, Spolsky is praising his hero--WHO DOESN'T LIKE UNIT TESTS! Can it be that Spolsky doesn't either?
I read the article and it could be that red flags go off in my head when I see generalities, or it could be because I was at a seminar on project management last week, or it could be that my outlook on things has changed in my old age, but I couldn't help but think of all the hidden assumptions.

Unit Tests

So, why do people write unit tests? What's the point? Are they just wasting their time? I mean, it's writing twice the code for the same functionality, right? Not quite. People write unit tests because unit tests are a useful tool that helps them accomplish one of their goals: to write reliable, maintainable software. How do unit tests obtain this? By having tests that show that the code does what it's supposed to do, they have confidence that the code continues to do what it should as changes are introduced because they can simply run the test suite to verify. And when tests that previously worked fail, they know they've introduced a bug in their changes and can quickly isolate and fix it.

The Test Driven Development (TDD) and Behavior Driven Development (BDD) people have brought these ideas to the front of the development cycle. If tests give you confidence in your code, why not write them first? Then as you develop your software, you can always and from the beginning have confidence that it does what it's supposed to do. The main difference between TDD and BDD is that with BDD the "tests" are expressed more in the language of the customer's problem domain than the language of the programmer. And thus there's more programmatic scaffolding to interpret the "tests". But the basic mechanisms are the same.

[It seems to me though that you had better pay really close attention to your design to avoid code churn where you may repeatedly write tests, then write implementations, but later delete both because that code ultimately didn't fit into your design. Sometimes you have to write code to vet aspects of a design before you know that's the design you're going to go with. This exploratory programming doesn't need tests up front IMHO]

In any case, at the end of the software project you have 2 sets of software: the software that you're going to provide to your customer and a set of unit tests that prove (to yourself and other programmers but not necessarily the customer) that the software does the things that it is supposed to do.

This is very similar to how scientific advancements are made. A scientist performs an experiment that gives some novel result. Then he publishes how he did it so that other scientists may reproduce his work. If others can not reproduce his work, then there may be something fishy going on or maybe the experiment was flawed in some way. If other people can reproduce his work, then our confidence in the validity of the experiment and its result grows. Unit tests are experimental proof that the software does what it is supposed to do (at least with regard to the things that are expressed as unit tests).

That's also why you find that code coverage tools are popular with people who write tests for their software. The tests may not exercise every bit of code in the system and a code coverage tool tells which subroutines or expressions or statements are executed. The more code that's exercised when running the tests, the more confidence that the entire software is operating as it should.

Having many unit tests that execute a high percentage of the code add up to show that statistically, the software performs as expected.

On Heroes

Spolsky's hero from the article is Jamie Zawinski (of xemacs and netscape fame) and is actually quoted as saying (in regards to unit tests) "Given a leisurely development pace, that’s certainly the way to go." So it's not so much that he doesn't like unit tests, as that he doesn't like unit tests for a particular type of development: get it out the door quick before the deadline.

But that's also a very short-sighted view of software development.

Does the programmer's obligation to the software end upon delivery to the customer? Sometimes. If, like Jamie Zawinski, your job is to get the product out *now*. Before the next guy. Before you're obsolete. Before the deadline that the sales people gave the client. Then, yes. Or if you're a mercenary programmer, it certainly does. You write some software that does a task, you give it to the customer and you're done. You're out of there. You're on to the next customer.

However, if you're not a mercenary programmer, then you're probably more interested in developing relationships. Sometimes it's a relationship with your customers; as when you develop a custom application for a customer and continue performing enhancements and upgrades long after you've delivered the initial product. Sometimes it's a relationship with your software; as when you're working on a software product that you wish to sell to the masses. Either way, you will invest more time and energy revisiting programs you've written in the past.

If you're a hot-shot programmer (HSP), you can write perfect code. You don't need tests because you're so meticulous and careful and just plain good at programming that you quickly find bugs and squash them. That's just how you do things. Jamie Zawinski is probably one of these people. Donald Knuth is famously one of these people. He is so confident in the perfection of his work (whether code or print) that he gives out monetary rewards that double as each bug is found.

And that's fine ... if you believe that HSPs are infallible and if the only programmer who will ever touch the code is an HSP or if all of the programmers that ever touch the code are HSPs. From the point of view of an HSP it doesn't matter if anyone else touches the code. Because as an HSP, you know that if you touch the code, all will be made well. But that's often not reality. Reality is that there are probably different programmers of varying skill that will ultimately touch the code over its lifetime.

From the point of view of the company that employs the HSP, they would be taking a risk not having some form of code verification as changes are introduced. Because from the employer's stand point, that HSP might be the only one on the team so other non-HSPs will definitely touch the code and they won't be as meticulous or careful. The HSP might not work at the company forever, and when he leaves, who's to say that the next guy the company hires will be an HSP or not? Odds are fairly good that he (or she) won't be. See Truck Number or Bus Number for more details on the risks involved.

Tools like unit tests help mitigate against some of these risks by codifying the correctness of the software into a test suite that can be run at any time by anyone.

On Context

These are some of the things that went through my head, but all of this is probably tangential to the point of Spolsky's article which really seems like a reaction against those people that have taken some good ideas and turned them into some sort of religion. People that latch on to the latest set of buzz-word technology and apply it everywhere. People that think that units tests are more important than the software they test. (If you're one of these people, tell me, which does the customer want?)

Spolsky praises Zawinski for his Worse is Better approach. I do too. Because it's very pragmatic. Write something useful, then let people use it. It may not be perfect, but you can iteratively refine it as needed. And do you refine it until perfection? Hell no! The perfect is the enemy of the good. Voltaire knew this 230-something years ago. Perfection is a chimera that leads you madness. The point is to just get useful things. So, you do the simplest thing possible to make the software more useful and that's it. Stop trying to predict the future. You can't. Stop trying to design for things that you imagine will be useful. Only design/write things that are actually useful now.

Oddly, these are the same ideas I hear from the practitioners of agile software development. The only problem is that when you name something, you can't be sure the name will mean what you want it to mean. I think these days when people say "agile", what's heard is "the Agile religion" rather than "that bunch of useful ideas".


So, with regard to unit tests ... a man with two clocks never knows what time it is, but a man with thousands of clocks invokes the law of large numbers and has a good statistical chance of knowing the time within a sufficiently small epsilon. (I hope everyone can draw the analogy :-) And if you're that man with thousands of clocks, don't get caught up in making clocks for the sake of making clocks.

Ultimately, who cares if Spolsky or Zawinski or I like or don't like unit tests? At the end of the day, to be a good developer or development company you have to find out what works for you or your team for a particular problem. Sometimes that might involve unit tests; sometimes that might mean you're a mercenary programmer; sometimes that might mean you're doing TDD. Or maybe something else entirely. Use what works.

And that's all I have to say about that. :-)

Monday, September 7, 2009

Things I wish were different in Padre

I've been using Padre as my primary editor for the last several days and I have to say that it's awesome. I've thoroughly enjoyed using it. However, there are some small things that tend to annoy so I thought I'd at least write them down so that, if nothing else, I'll have a list of potential things to hack into Padre if the mood takes me. And I'm making the list public in this way so that maybe even the LazyWeb will do the work for me :-)

In no particular order ...
  • The find dialog box appears right in the middle of the window and I always have to move it out of the way to see the text underneath. I think something like Firefox's find-at-the-bottom status bar would work well instead.

  • The mechanism for changing what language Padre thinks the document is written in is a little bit too cumbersome. There are times when I'm looking at some random perl file, but Padre doesn't figure out that it's Perl. So I have to go View -> View Document As -> and then select the appropriate language. I'd like something a little more Textmate-ish where I can click on the text in the status bar that indicates the language of the document and be able to select from the list there.

  • I wish there were a keystroke for deleting an entire line. There may actually be one, but I don't know what it is and I can't seem to find it.

  • The documentation could use a little work. :-) In searching for keystrokes that might delete a line, I stumbled across Control-D which apparently duplicates the current line. Later I was looking through the help and came across this:
    (TODO What is Ctrl-D ?, duplicate the current line?)

  • I can't seem to copy from the DocBrowser

  • DocBrowser Search doesn't seem to do anything useful as everything I search for ends in "no results found for ..."

  • Not really a Padre problem, but the vi plugin doesn't seem to support many editing keystrokes that I use all the time. I've been using Padre with the vi plugin turned off because of this, but I really wish one of two things would come to pass:
    1. the vi plugin supported more of the keystrokes I tend to use in vi or
    2. Padre provides some keystrokes of its own to do the things that I tend to want to do (like delete an entire line).
    Using Padre for the past week or so has made me realize that I'm not so tied to vi as I often think I am and that I could be happy with #2

  • There have been times when I'm clicking through a directory structure and the ".." entry at the top disappears. I'm not sure exactly when it happens, but it seems to happen whenever I click into a Catalyst project directory. I have to change the listing mode view to "navigate" to get it back and be able to navigate to the parent dir.

  • I really wish that padre would background itself when run from the command line. And that padre . would start the directory browser in the current directory.

Okay ... enough of that. These are just the little things that I've run across; nothing major. But it's the little things that affect your quality of life. Right now Padre is awesome but could use a few small QoL adjustments to make me happier. weirdness

The other day something strange happened. I tried azawawi's one-liner for upgrading Padre on my linux desktop. What happened was that the latest Padre (0.45) was downloaded and installed, but then so was version 0.40. I didn't realize that this had happened immediately, but once the upgrade was finished, I wanted to see something different, so I fired up Padre and went to Help -> About. I was shocked to see "Padre 0.40". So then I typed padre --version on the command line and again, it told me "Perl Application Development and Refactoring Environment 0.40". Still disbelieving, I asked on #padre how to tell the version of Padre installed and they, of course, said padre --version.

It was at this point that I realized I still had the window history available in the window I had typed the command originally. I scrolled back through the output and there it was, plain as day; first in the original upgrade list was Padre 0.45 to upgrade Padre itself and then Padre 0.40 to upgrade Padre::Wx::Menu::Experimental (whatever that is). Weird. So, I ran the one-liner again while paying close attention to the output this time. This time it installed Padre 0.45 as expected ... and then installed Padre 0.44 because of Padre::Plugin::Perl5.

I don't know what was going on, but perhaps it was an old version of because when I ran the same one-liner on my laptop, it worked flawlessly (I'll have to check the desktop tomorrow as it's at work and I'm not :). I have had weirdness before on my desktop since it was where I first installed Padre almost a year ago, so maybe that factored in somehow too. Anyway, in the end I just installed Padre 0.45 specifically and only and that worked just fine.