The Craftsman and the Bazaar

Many of you will be familiar with that seminal work The Cathedral and the Bazaar, a manifesto of the open source movement. If not, I recommend you read it. Today I want to talk about a different perspective: the Craftsman and the Bazaar.

The craftsman is an expert in his trade who builds to last. He creates quality heirlooms that people will be treasuring and restoring in centuries to come, long after it is forgotten who made or originally commissioned them. The bazaar sells trashy geegaws that don’t last. They suit our society driven by commodity, novelty and innovation.

Mainframe systems are recently retired or still around after thirty years of service. As our industry matures (and IT technology matures) eventually the enormous rate of change will slow and systems will be expected to stay around for even longer. Change in the aircraft industry has slowed: look at how long the 737, A320 and 747 have been around and still undisplaced as the workhorses of the international skies. Automobile manufacturers try desperately to find novelty in a field that has pretty much exhausted it. New bridge technologies don’t come out every year or two, so bridges are expected to stand for the lifetime of their materials or their usefulness. Software doesn’t wear out per se, though its quality will degrade over time with constant maintenance, especially if architecture and/or documentation are poor.

Ah, documentation. Hated by most, done (properly) by few. In the brave new IT world we apparently need it less - which brings me back to open source. My experience of open source code is that in-code comments are rare, and accompanying documentation minimal. If you look beyond usage notes, documentation is usually non-existent.

Why were certain design decisions taken and what were the options and considerations? What are the critical aspects of the architecture that must be preserved in future? What is the architecture, the standards to which it is built? What mistakes have been made and fixed, so as not to repeat them? A code craftsman, building for the future, would consider this documentation as essential as an install script.

Open source developers, on the other hand, are writing for the bazaar: their targets are to get something working, to get immediate uptake (and recognition), to have fun coding. Their products are cool, shiny, eye-catching, reflecting the latest fashions (Ajax, XHTML, video, Google Maps, RSS…).

They are like the toys my son buys from the $1 shop: made cheap, made to be picked up, not made to last more than a day in the front yard. They are so cheap they are effectively free, and he won’t be handing them down to his children. Contrast them with his Märklin trains, his wooden Thomas trains, or his Lego. These are designed by professional engineers. They are robust, useful, compellingly fun, and compatible throughout their range. The play value lasts for years, long after the novelty is gone. And in twenty or thirty years he will be pulling them out of the attic for the next generation. These are craftsman toys.

Am I saying he will be dusting off the old Adobe Photoshop [insert your favourite well-engineered software here] disks in thirty years too? Well, no, I doubt that somehow. Unlike the other engineering disciplines, IT is still caught up in a pell-mell rush of evolving technology that means the same considerations don’t apply as they do to bridges or toys. Even craftsman software (if there is such a thing) becomes obsolete quickly.

For that reason, what the open source people are doing is OK right now, I guess. This entire website rests on the LAMP stack (Linux, Apache, MySQL, PHP …. and Drupal) so I must think open source is useful. There is little justification for craftsmanship in anything that is going to be obsolete so quickly.

What concerns me though, is the habits we are forming and the direction open source is taking. Many in that movement will be completely mystified by these comments. “Surely open source code is better, of higher quality, than most other code? Developed by a large pool of enthusiastic and committed programmers, designed free of commercial imperatives, tested by thousands of others…”

Software is different from bridges or toys or furniture. It is different to most other artefacts produced by Man, in that it changes, constantly. It evolves to meet new requirements; it shifts to adopt new technologies. Documenting why and how a bridge was built is mildly important. Documenting code is essential. And as we start to move into a new phase of the Information Age, a maturing phase, we will expect our software to stick around. The obsolescence cycle will get longer. The code itself will become one tiny facet of the artefact that is a piece of software. On its own it will be worthless. It will be one cell in a matrix of requirements and designs, frameworks and architectures, minutes and discussions, decisions and compromises.

If the human race is to have any hope at the end of the 21st Century, we will have rediscovered Quality by then. The trashy disposable pap that characterises Western civilisation at the start of this century will be seen as an embarrassing diversion, like the Naughty Nineties at the turn of the last one.

Along with that broader trend, IT will grow up. IT is engineering and finally starting to admit it (ITIL is a good positive example of the new professionalism that I may discuss one day). Engineering does not set to to be fun, creative, spontaneous. Enginweering is sober, disciplined, frugal and conservative. In the next few decades, software engineering will adopt the attitudfes, practices and professionalism of the other branches of engineering.

Somewhere between now and then, software is going to have to start being built to last just like every other constructed system. Like any business infrastructure, we will demand we get maximum return on our investment before we decommission it.

I like open source software. It is human, personal, sharing, communal, fun ... and cheap. Much of it is also, by engineering standards, trashy junk. I don’t think Linux is junk, nor other of the big foundation systems that get enormous input from many people. But I worry about the code where each module is written by one or two amateurs in their spare time, which describes huge quantities of the open source stuff out there. I work in that kind of code a lot, and much of it is pretty rough. But more importantly, most of even the well-crafted code is just that: code. Little or nothing else exists to assist when the current volunteer has babies and a promotion and suddenly isn’t interested in unpaid code development and someone else steps into the breach. When the individuals move on, much of this code will fall apart like a twenty-dollar remote-controlled car or a no-name DVD player or an Ikea bookcase. You get this stuff on trestle tables in the Bazaar and it isn’t going to last long. It does not represent the future of software (I hope). The future is going to need the work of Craftsmen.