The Craftsman and the Bazaar

Many of you will be familiar with that seminal work The Cathedral and the Bazaar, a manifesto of the open source movement. If not, I recommend you read it. Today I want to talk about a different perspective: the Craftsman and the Bazaar.

The craftsman is an expert in his trade who builds to last. He creates quality heirlooms that people will be treasuring and restoring in centuries to come, long after it is forgotten who made or originally commissioned them. The bazaar sells trashy geegaws that don’t last. They suit our society driven by commodity, novelty and innovation.

Mainframe systems are recently retired or still around after thirty years of service. As our industry matures (and IT technology matures) eventually the enormous rate of change will slow and systems will be expected to stay around for even longer. Change in the aircraft industry has slowed: look at how long the 737, A320 and 747 have been around and still undisplaced as the workhorses of the international skies. Automobile manufacturers try desperately to find novelty in a field that has pretty much exhausted it. New bridge technologies don’t come out every year or two, so bridges are expected to stand for the lifetime of their materials or their usefulness. Software doesn’t wear out per se, though its quality will degrade over time with constant maintenance, especially if architecture and/or documentation are poor.

Ah, documentation. Hated by most, done (properly) by few. In the brave new IT world we apparently need it less - which brings me back to open source. My experience of open source code is that in-code comments are rare, and accompanying documentation minimal. If you look beyond usage notes, documentation is usually non-existent.

Why were certain design decisions taken and what were the options and considerations? What are the critical aspects of the architecture that must be preserved in future? What is the architecture, the standards to which it is built? What mistakes have been made and fixed, so as not to repeat them? A code craftsman, building for the future, would consider this documentation as essential as an install script.

Open source developers, on the other hand, are writing for the bazaar: their targets are to get something working, to get immediate uptake (and recognition), to have fun coding. Their products are cool, shiny, eye-catching, reflecting the latest fashions (Ajax, XHTML, video, Google Maps, RSS…).

They are like the toys my son buys from the $1 shop: made cheap, made to be picked up, not made to last more than a day in the front yard. They are so cheap they are effectively free, and he won’t be handing them down to his children. Contrast them with his Märklin trains, his wooden Thomas trains, or his Lego. These are designed by professional engineers. They are robust, useful, compellingly fun, and compatible throughout their range. The play value lasts for years, long after the novelty is gone. And in twenty or thirty years he will be pulling them out of the attic for the next generation. These are craftsman toys.

Am I saying he will be dusting off the old Adobe Photoshop [insert your favourite well-engineered software here] disks in thirty years too? Well, no, I doubt that somehow. Unlike the other engineering disciplines, IT is still caught up in a pell-mell rush of evolving technology that means the same considerations don’t apply as they do to bridges or toys. Even craftsman software (if there is such a thing) becomes obsolete quickly.

For that reason, what the open source people are doing is OK right now, I guess. This entire website rests on the LAMP stack (Linux, Apache, MySQL, PHP …. and Drupal) so I must think open source is useful. There is little justification for craftsmanship in anything that is going to be obsolete so quickly.

What concerns me though, is the habits we are forming and the direction open source is taking. Many in that movement will be completely mystified by these comments. “Surely open source code is better, of higher quality, than most other code? Developed by a large pool of enthusiastic and committed programmers, designed free of commercial imperatives, tested by thousands of others…”

Software is different from bridges or toys or furniture. It is different to most other artefacts produced by Man, in that it changes, constantly. It evolves to meet new requirements; it shifts to adopt new technologies. Documenting why and how a bridge was built is mildly important. Documenting code is essential. And as we start to move into a new phase of the Information Age, a maturing phase, we will expect our software to stick around. The obsolescence cycle will get longer. The code itself will become one tiny facet of the artefact that is a piece of software. On its own it will be worthless. It will be one cell in a matrix of requirements and designs, frameworks and architectures, minutes and discussions, decisions and compromises.

If the human race is to have any hope at the end of the 21st Century, we will have rediscovered Quality by then. The trashy disposable pap that characterises Western civilisation at the start of this century will be seen as an embarrassing diversion, like the Naughty Nineties at the turn of the last one.

Along with that broader trend, IT will grow up. IT is engineering and finally starting to admit it (ITIL is a good positive example of the new professionalism that I may discuss one day). Engineering does not set to to be fun, creative, spontaneous. Enginweering is sober, disciplined, frugal and conservative. In the next few decades, software engineering will adopt the attitudfes, practices and professionalism of the other branches of engineering.

Somewhere between now and then, software is going to have to start being built to last just like every other constructed system. Like any business infrastructure, we will demand we get maximum return on our investment before we decommission it.

I like open source software. It is human, personal, sharing, communal, fun ... and cheap. Much of it is also, by engineering standards, trashy junk. I don’t think Linux is junk, nor other of the big foundation systems that get enormous input from many people. But I worry about the code where each module is written by one or two amateurs in their spare time, which describes huge quantities of the open source stuff out there. I work in that kind of code a lot, and much of it is pretty rough. But more importantly, most of even the well-crafted code is just that: code. Little or nothing else exists to assist when the current volunteer has babies and a promotion and suddenly isn’t interested in unpaid code development and someone else steps into the breach. When the individuals move on, much of this code will fall apart like a twenty-dollar remote-controlled car or a no-name DVD player or an Ikea bookcase. You get this stuff on trestle tables in the Bazaar and it isn’t going to last long. It does not represent the future of software (I hope). The future is going to need the work of Craftsmen.


Are you really talking about open source?

I feel the same frustrations that you do re: technology -- even about the same technologies that you list (Ajax, XHTML, video, Google Maps, RSS…). Where we disagree is that you think the problem is open-source and I think that has nothing to do with it.

Look at your list: Ajax, XHTML, video, Google Maps, RSS…. Either they aren't "source" at all or they're straight-up commercial hype from the Silicon Madison avenue (quite the opposite of open-source).

I think what you are decrying is shallow commercialized faddism. A destructive, marketing driven force that seems more potent in technology circles than it is in the rest of our economy. Why? Because _most people don't know how technology works_ so it is easier to lie about it, create hype, and then grossly dissapoint when the product under-delivers (and then hope that you don't have to give the money back).

I would argue that open-source has been the antidote to much of that. Outside of PHP open-source (which really has attracted some rock-headed programmers), I see some of the most elegant, built-to-last, foundational OS, browser, scripting-language, compiler, database, filesharing, and other software that we have available coming out of open-source. OSS projects first took a lot of fundamental systems and commoditized them away, and now many of them are pushing the envelope.

Why? The OSS stuff was built for someone to _use_ -- not for someone to sell. No one wins when you the OSS stuff is unusable -- not in the short term and certainly not in the long term. There is nothing hidden and nothing to lie about. If it has warts -- no one can tell you they aren't there.

If you are tired of broken electronic promises and feel jilted at the cash register, I would suggest that more OSS is part of the cure, not part of the problem.

Good reply, thanks.You

Good reply, thanks.

You misunderstand me on one point: I was refering to Ajax, XHTML, video, Google Maps, and RSS as cool gadgets that open source porgrammers seem to fixate on, not as examples of open source itself. What I meant was I wish the programmers would concentrate more on some basics like security, scalability and sound architectures instead of banging together some "home-movie" code and throwing it out there.

You make a good point that most of my experience of open source is with PHP, so if you say the programming is of better standards in other environments I can't dispute that. Anyone else comment?

I do agree that the foundation stuff, Linux, MySQL etc is robust and good. (But where are the admin and query toools for MySQL on Linux? PHPMyAdmin is beautifully functional - though a decade behind the MySQL Windows tools, but I find it hilarious that the same peope who wrote and use it are probably slinging mud at Windows and IE security, because PHPMyAdmin security is a joke. And as a user of Apache, without having seen the source, I must say some of it looks a bit klunky, dare i say kludgy, usability-wise. I like to assume that is because all the design is focused on the internal engine, which appears to be rock solid).

No, I'm more concerned at the stuff that fizzes around on top of these: CMSs and application packages and tools and such, user-contrib stuff rather than committee-managed core stuff.

My experience is narrow here, so I look to comments from others such as yours to clarify my understanding. Thanks.

Syndicate content