Worse Is Better

RichardGabriel makes this observation on the survival value of software in the paper Lisp: Good News, Bad News, How to Win Big. See http://www.jwz.org/doc/worse-is-better.html for the section on WorseIsBetter.

For those seeking the first node, see http://web.archive.org/web/19990210084721/http://www.ai.mit.edu/docs/articles/good-news/good-news.html.

For even more context on WorseIsBetter see http://www.dreamsongs.com/WorseIsBetter.html. My favorite part is RichardGabriel arguing with himself.

A story of the triumph of engineering over science. Or put another way, features, availability (delivery), and price appear to weigh heavier than quality in the mind of consumers, both corporate and household.

WikiWeb is a very strong example of WorseIsBetter in the broad sense.

It's an interesting paper. There seem to be two subtexts: one is that WorseIsBetter is a good design pattern, so we should start with simple, predictable systems, and so on; the other is a kind of consolation for people who build good systems and then see them fail commercially. -- MartinPool

One point, in RichardGabriel's book if not in the paper, is that we don't really know what the users want most. So, get your minimal implementation out there as soon as possible and see what happens next. This seems to match YouArentGonnaNeedIt. Also, the smaller the design, the easier it is to port, which makes it more likely to become widespread -- C being an example. Once it's widespread, the rest will follow. -- SteveFreeman

For a similar WabiSabi / WorseIsBetter (depending on the point of view) approach to a chat can be seen at http://www.sange.fi/chat.php?lm_lang=en.

I believe he actually refers to C as a virus, which is certainly an interesting interpretation. An interesting reinterpretation of your last phrase is then,

"Also, the smaller the design, the easier it is to port, which makes it more likely to spread -- C being an example. Once it has spread, the rest will follow". -- AnthonyLander

So smallness is best for every possible reason. SoftwareRenaissance anyone? Can we keep SmallStandards?

Like EnterpriseJavaBeans?

The original paper does not make difference between SimplicityOfImplementation? and SimplicityOfSpecification?. CorrectnessOfImplementation? comes in two forms, CompletenessOfImplementation? and SimplicityOfSpecification?. These are the MIT and CalBerkeley paradigms, not CompletenessOfImplementation? and SimplicityOfImplementation?.

Also, CompletenessOfImplementation? goes against LayeredAbstraction?. -- PanuKalliokoski

From the paper cited above:

"It is important to remember that the initial virus has to be basically good. If so, the viral spread is assured as long as it is portable. Once the virus has spread, there will be pressure to improve it, possibly by increasing its functionality closer to 90%, but users have already been conditioned to accept worse than the right thing. Therefore, the worse-is-better software first will gain acceptance, second will condition its users to expect less, and third will be improved to a point that is almost the right thing. In concrete terms, even though Lisp compilers in 1987 were about as good as C compilers, there are many more compiler experts who want to make C compilers better than want to make Lisp compilers better." (Italics added.)

Take the phrase: "In concrete terms, even though Lisp compilers in 1987 were about as good as C compilers, there are many more compiler experts who want to make C compilers better than want to make Lisp compilers better" and replace Lisp with Java, C with C++, and 1987 with the current year and you've got a lively topic of discussion

I couldn't help but think of Microsoft's Windows victory over IBM's OS/2. Get it out early and make it 90% right later.

And I can't help thinking of current XP philosophy: get it out quickly, simple but basically as expected, and we'll make it better into a future iteration. -- OmCandea

A more recent example of the truth in this paper could be a comparison between SOAP and CORBA. The reason for SOAP's appearent success lies in it's simplicity. The fact that it is so simple that anyone can understand it, is probably the reason for it's widespread use. JanPersson

"Worse" still, in this context, is simply XML-over-HTTP, which many folks will take over SOAP any day of the week.

The cited paper is a recognition of the CommercialReality? that it is important for a system to attract a large number of users, in short, that NumbersCount. The opposite MIT/Stanford desire to do the RightThing fails because it does the right thing for the programmer, not for the user. The same reasoning could be applied to standards and standardization organizations aswell: Neither X.400 e-mail, IPv6 or much of the later output from W3C have succeeded in attracting followers, because of overcomplexity or disregard for the users' real needs.

This seems like an argument for ExtremeProgramming, especially with all the parts about a monolithic system that takes a long time to finish, and that delivering only a certain portion of the functionality, early, will gain wider acceptance for your system.

As MaeWest? once said, "When I'm good, I'm very good - when I'm bad I'm better."

Another notable MaeWest? quote would be: "Too much of a good thing is wonderful!"

My thought on this paper for a long time has been that it is a story of different value systems.

Humans have a natural tendency to think of worse and better as being absolute qualities. They are not. What is worse and better depends on your value system. The mistake of confusing quality according to one set of values with quality to another is very hard to notice, and can lead to deep confusion.

In this case Lisp is Better because it tries to work right. C is Worse because it doesn't. C is Better because it tries to be popular. Lisp is Worse because it doesn't. The "paradox" is that trying to operate correctly is both Worse and Better at the same time. Well, of course! It is worse and better because those two value judgements are being made from different value systems!

For another example, look for the rants of the Multics people on how no current operating system aspires to achieve a level of security which the Multics folks found insufficient for far less dangerous environments than the Internet today. It is the same complaint that RichardGabriel had, only they don't write such fun papers to read because they stay clear on the difference between being popular and being good.

A brief digression. Perl is in many ways a bad language, but CPAN wound up being the largest repository of reusable components in any language. Why? My belief is that a major factor is that Perl is fairly hard to customize (DamianConway has been changing this). This is a bad trait from the point of view of a team being productive but it is a good trait for reuse between teams because my component is likely to integrate naturally into yours and vice versa.

-- BenTilly

Another subtext of this article is that Lisp has somehow failed. There are two different strategies for succeeding -- SimpleImplementation? and SimpleInterface?. The success that is achieved by each strategy has a different flavor. -- AsimJalis

Not subtext, that was an explicitly discussed issue of the article, in fact it was the theme; the title "WorseIsBetter" is about the "why" of that. -- dm

"Worse is worse" by Jim Waldo, December 9, 2003

Summary: The classic essay on "worse is better" is either misunderstood or wrong. And in citing it in our design arguments, we short change both our customers and our craft. I revisit this essay, and reflect...

http://www.artima.com/weblogs/viewpost.jsp?thread=24807

I love the conversation between the UNIX (BillJoy) and the MIT guy (DanWeinreb), especially the bit that goes,

"The New Jersey guy said that the Unix solution was right because the design philosophy of Unix was simplicity [...] The MIT guy pointed out that the implementation was simple but the interface to the functionality was complex ..."

"The MIT guy then muttered that sometimes it takes a tough man to make a tender chicken, but the New Jersey guy didn't understand (I'm not sure I do either)."

Does this mean that 'worst is best'? Judging by the success <insert_your_favourite_here> seems to be having...

I notice that in a commercial system, generally if the software or system fouls up say one in every 50,000 customer orders, the bug will generally be ignored if it's not an easy fix. This is because there's usually a wish-list of other things to do such that losing 1/50000 does not rank up their as important. (Of course, accounting systems must be more robust because losing pennies is not legal.) For example, the wish-list item right above fixing that bug may be to produce a new marketing report that's estimated to bring in 5,000 new customers. Profit-wise, that's considered a better bet than fixing the known bug, so it gets done first. The loss of customers from the bug is judged to be less than gained from the new marketing report. Thus, Bug Stays. The MIT approach would be to fix all known bugs before adding any new functionality. But the idealism of money is often different than idealism of system design. - t

The Economics of WorseIsBetter

There was an article reviewed on Slashdot approximately a year ago (very roughly Jan. 2009) that more or less said the same thing. Unfortunately, I couldn't re-find it. Heavily-engineered tools simply can't economically compete with the more nimble and somewhat hacked-together tools. Customers keep preferring quantity of features and cheap features over reliability and technical polish. I'm not necessarily endorsing one or the other, only saying it's a reality of the marketplace. (Apple is perhaps an exception, but sex-appeal may play a bigger role than productivity for Apple customers.)

Here's a hypothetical illustration that may suggest the reason for this pattern. Unix camp "A" (roughly based around article stereotype) has a bug rate of about 10 per 10,000 items processed, while the MIT camp, known as "B", has a rate of about 3. Assuming the cost for each tool is about the same, version A will have four features while version B has two. This is because B spends more time on each feature to get it "right" and thus has fewer resources available to work on other or more features.

The customer of product B has to perform features 3 and 4 themselves. They may use a spreadsheet or their limited knowledge of Microsoft Access to hack a result out themselves. Generally this approach is bug-ridden, and the mistake rate is about 40 per 10,000 items processed. (Even if they are careful, it is not as road-tested as the same thing sold to many users/customers.)

From the customer's perspective, Product B is better because it results in a lower number of total bugs for the themselves, the customer.

Product A from consumer's perspective: The Unix Camp:

Product B from consumer's perspective: The MIT camp:

From the customer's perspective, the lack of features is a bigger "quality" problem from their own perspective because they have to fill in the missing features with their own efforts.

Keep in mind that this illustration held the tool-maker's effort constant. Suppose "B" had a bigger team and thus charged more for their product such that they finished features 3 and 4. Would the consumer then pay the higher price? I may address this issue later.

- top

Re: "triumph of engineering over science" -- I'd argue it's "economics over science", not "engineering". Given a choice, many if not most engineers are perfectionist by nature and would prefer to have the time to get every detail right. But, the marketplace puts as much pressure on features, schedule, and price as it does on quality (with the exception of a few niches, such as life-support/safety-related tools). -t

DoesWorseIsBetterRequireOpenSource?

A. OpenSource has shown to be a shining example of WorseIsBetter.

Will be interesting to see if this attitude holds up in light of the many recent security breaches that will end up costing hundreds of millions of dollars.

See also:

CategoryIdealism, CategoryQuality, CategoryEconomics

EditText of this page (last edited October 11, 2014) or FindPage with title or text search
Meatball