Commitment To Quality

Okay, there's a good consensus that bugs are bad. However, there's a slightly distinct question to ask - does the presence of known, simple, reproducible bugs in software indicate a lack of commitment to software quality? Or is it possible to reconcile the two?

The QxF principle...

We might view things in terms of "total value delivered". How much functionality is shipped, times its measure of quality, yields in some appropriate arbitrary real-valued units the amount of work done for the customer. Prioritizing issues is then just a matter of comparing the "total value" any given task (bugfix, new feature implementation) will bring to the bottom line. This may leave a residue of bugs, and under this theory that's not such a bad thing.

This might be true under special circumstances, like when you inherit the maintenance of an important but buggy product. However, when you are developing new functionality, the "let some bugs slide, we have more important features to implement" is a disease. Using priorities as a rationale for this is bogus.

...vs lexical ordering of quality and functionality

We would insist (and I think XP does so) on a "lexical ordering" of quality and functionality. Functionality exists only if it is delivered totally perfectly, and "bug fixes" are always priority tasks.

Adjust your priority system to a finer granularity of decision-making so that only really clean stuff gets released. If there wasn't time to do it well (without lots of annoying bugs), then you bit off too much. This is a really key principle to understand, and it's the only way I know to scale up with control.

Software quality means no bugs in what you do from today on.

The quality of the development process is largely the defining factor in how "buggy" the software will be. However, the quality of the software is dependent upon how well it supports the user in his tasks, a much broader requirement. I can't tell you how much time I've seen wasted debating whether something is a bug report or a new feature request. If it is important to the customer, get it done. Also, quality is not an absolute, you need to speak of "more or less quality" not "quality or no quality." --WayneMack

This is in line with the "quality times functionality" view above.

In my opinion, this view doesn't hold up.

Not because the evaluation of a real-valued measure of quality ''or' functionality might be tricky; that would be the least of my objections. Mainly, I would argue that a "buggy feature" doesn't help anyone get any work done - at least as soon as the bug is known to exist. Bugs undermine your confidence in existing functionality; at best you learn to "work around" the bug (the phrase suggests extra expenditure of effort), at worst you stop using it altogether.

So (at least in lots of cases) a "buggy feature" might as well never have been implemented. In some situations - server software, which has to service multiple users simultaneously or over time - bugs might even cause unrelated features to become unavailable, or unreliable (and therefore untrustworthy). This suggests that quality is not an "amount" that you could multiply into another amount (functionality). So we prefer the "lexical ordering" principle.

The lexical ordering might seem too demanding, but let's examine it. "Perfect" is defined in a very pragmatic way : "passes all tests". In the extreme case, this is even a vacuuous requirement - if there are no tests, the software is perfect by definition. ("Totally" also means "passes all tests", except that in this latter case there can't be no tests. This yields acceptance tests.)

Assuming honesty in the customer, have them point out a bug, write a test for it, then fix it : you now have a portion of your software where quality in the absolute sense is guaranteed. Iterate until no bugs left. You can then start coding new functionality.

This might be a hard pill to swallow for a legacy system, but you'd have to admit that it's perfectly feasible for a greenfield project; you start with zero bugs and zero functionality, so you're free to add functionality as fast as you want. Right until the moment when you get a bug report, where you stop and fix it. Then you go on.

To the extent that this strict ordering is respected - without exceptional reasons, you can never invoke efforts on functionality to excuse not fixing known defects - we can speak of a commitment to quality.

[DeleteMe - responses to the "quality times functionality" view by WaldenMathews and LaurentBossavit, refactoring in progress.]

Armed with the foregoing theory, we can turn to the case of Ward's alleged negligence "deval[uing] anything that is written here about software quality". Just because a bug exists doesn't mean it isn't Ward's top priority... as far as dealing with Wiki innards is concerned. As long as we don't see tons of new functionality piled on top of a known bug, Ward's credibility should not be endangered. (For some reason I'm not worried too much.)

I think we would agree that a label that says "passwor" instead of "password" is a bug, although a pretty unimportant one. So what's wrong with leaving a few of these in, while we make progress on eagerly awaited new functionality? (Of course, if it said "assword", you'd fix it right away!) I believe this is the kind of marginal case where holding to the "lexical ordering" might seem absurd. But would it be?

-- WaldenMathews

Does a mislabeled button affect the program's behaviour ? I'd call it a "typo", not a code defect or a bug. You're right though, there might be borderline cases.

The mislabeled button is the program's behavior, part of it. I'm glad I overshot with my example. What I wanted to do was to clearly establish the borderline cases, and find out how our policies hold there. Does "borderline" mean a breakdown in the lexical ordering?

Not necessarily. We might counter the suggestion that it does with this observation : even if we do classify a mislabeled button as a bug, fixing it would require so little effort that allowing new features to take priority would definitely imply a lack of commitment to quality. So "small bugs" as a borderline category does no harm to the lexical ordering.

Okay, and what about a gazillion of these small bugs, to the tune of a month's worth of fixing? And then, a harder case, what about bugs whose behavioral simplicity belies their causal complexity, like when several acres of fungal root mass blossom in a tiny, white little button in your front lawn?

Gazillion of small bugs will not accumulate if you do have, and uphold, a commitment to quality. This is an assertion, not a deduction - if it can be proven wrong, the sooner the better, as I'll need to reassess my entire worldview with respect to software engineering. In such a situation, commitment to quality and commitment to delivering value are at odds, which they shouldn't be - use risk-driven scheduling, in agreement with the customer, to bring them back in line.

Small bugs which result from complex emergent properties of the design are another matter. Also, yet another category that would be a good candidate for disturbing the lexical ordering is "debatable" bugs - unexpected behaviours which it might not be possible to unambiguously label as bringing positive or negative value. An example I have in mind is when JohnCarmack "fixed" a bug in Quake III which permitted players to move at unrealistic speeds by "bunny hopping" - performing sequential jumps. The number of users who protested was such that John put the "bug" back in.

I hope to justify the digression by suggesting that "emergent bugs" and "debatable bugs" stem from the same root causes - the desired behaviour is too fuzzily specified, or not specified at all. This is obvious of the debatable bugs, less so of the emergent ones - the reasoning is that if the expected behaviour were clearly specified and known to the developers, the software's design would probably reflect these expectations, and the "emergent" behaviour would be easy to relate to the design.

Another way of stating the same idea is that "quality" can only be assessed objectively to the extent that you have a formal specification, in the form of acceptance tests, or use cases, or requirements documentation.

Does the delivered system meet the requirements specified by the customer? Quality is only quality if it is what the customer specified. If I order apples and I receive a load of bright red apples, do I have quality? NO, I ordered green apples. Unless I receive exactly what I ordered, the delivered product does not have quality in my eyes, the paying customer. Who gives a whit about what the manufacturer calls quality, does it meet my requirments. IF I buy an off the shelf product, then the quality of the product depends on the % of attributes that meet MY criteria for quality. - Monty Moore

See: DoesSoftwareQualityMeanNoBugs

CategoryQuality