Sources Of Bugs

If this topic were ever turned into a truly detailed typology, it might lead to some important new ways to minimize defects. (Apparently I'm wrong, since that's already been done to some extent.)

See IEEE 1044 Standard for classification of software errors, faults and failures.

BorisBeizer regarded the IEEE 1044 Standard as too coarse, and created a very detailed and widely-cited bug taxonomy, which appeared as the appendix in Software Testing Techniques (ISBN 0442245920 ), also in Word format at http://inet.uni2.dk/~vinter/bugtaxst.doc.

http://www.kaner.com/pdfs/BugTaxonomies.pdf

Top levels of BorisBeizer's bug taxonomy (original includes several sublevels beyond that shown here):

Sometimes I informally try to monitor myself as I program, to see where I go wrong. A lot of bugs I catch by eye as I'm editing. Many more are caught by the compiler, especially in statically type-checked languages. More are discovered by my own testing. By the time code is put under version control, over 99% of the bugs have been found and fixed. I want to categorise them here.

It's not my intent to duplicate the AntiPattern topic, or to list inverses of SevenPrinciplesOfSoftwareDevelopment. Bugs are lower-level than that. On the other hand, this isn't the place for WarStories. The aim is to make general observations. Bugs which are not language specific are the most widely appreciated.

-- DaveHarris

I went through a 6-month period when I kept a log of all faults I found in my programming (it was a compile-link language, so I had time to). Almost all of my faults came from (a) a usage failure of a subroutine or function from a documented library or API, i.e., not reading the manual, or (b) misunderstanding how something worked in the operating system (which wouldn't have been fixed by reading the manual, because I wouldn't have looked in the manual in the right place).

I used the DijkstraGriesProgramDerivationTechnique, which meant that I had almost no logic errors and no OffByOne errors.

I found the results of my fault-logging to be most disconcerting, because I really dislike spending time reading the API documentation closely and would much rather have found logic or OffByOne errors.

-- AlistairCockburn

This is what can make Eiffel-style pre- and post-conditions such a big win. They get some of the assumptions documented (and preferably tested). -- DaveHarris

Alistair's experience with APIs makes me think about several of the problems with BlackBoxComponentry and SoftwareReuse: (1) adequately specifying things that are to be reused, and (2) the inevitable bugs and sheer time loss associated with building familiarity with what is to be reused. -- MichaelFeathers

Donald Knuth had a long, excellent article in Software Practice And Experience that listed hundreds of bugs found in TeX and METAFONT, by him or by users, from the earliest development until the article was written. The paper has been incorporated into Knuth's book, LiterateProgramming:

-- PaulChisholm & KatyMulvey

How much does the type of bug affect when the bug is discovered? Is there a deep difference between the bugs which get caught before source is put under VCS, and those which survive until Beta testing or even final release?

A good Beta test, by definition, should catch all the same kinds of bugs that an actual release would expose. A lot of these are due to interaction with various other hardware and software products. It's not economical for a solo programmer to test each day's new code against every printer and video card that's out there.

I understand some Smalltalk people to claim that the combination of static type-checking and manual testing does not catch more bugs then the combination of dynamic type-checking and manual testing. This is partly because a dynamicly-checked environment enables more thorough testing, and without manifest types there is less to check. Obviously this is controversial, and full of hidden assumptions.

-- DaveHarris

Certainly the thing that lets XP people go really fast is our focus on UnitTests and to a lesser degree AcceptanceTests. The tests contribute in at least these ways:

Defect injection rates are generally proportional to code size. Less code means fewer defects. -- RonJeffries

Closer to the topic, I saw a taxonomy of bugs once in an appendix of a book. I looked around last and couldn't find it. Since I cannot reuse that list and I'd have to take time to learn it, I guess I'll have to make my own bugs. -- MichaelFeathers

Testing Computer Software, Second Edition, by Kaner, Falk, Nguyen, (ISBN 0-442-01361-2 ) has an appendix of approx. 60 pages with common software errors. -- VolkerWurst

In "A Discipline Of Software Engineering", WattsHumphrey writes that basic defect causes fall into 5 categories:

  1. Education: You did not understand how to do something.
  2. Communication: You were not properly informed how to do something.
  3. Oversight: You forgot to do something. (e.g. ForgetToFixItLater )
  4. Transcription: You knew what to do, but made a mistake in doing so.
  5. Process: Your process somehow misdirected your actions.

These five can be abstracted into two: ignorance and carelessness. Both are preventable--if not necessarily curable. This is how there can be lots of software-controlled artifacts we use that are for all practical purposes bug-free. Some programmers can do this repeatably.

It's a matter of engineering style--a style that's the antithesis of egoless programming. In my world, programmers own their code forever, and take some pride in it. A code review is an opportunity to demonstrate your superiority by presenting code nobody can improve. You study errors and avoid them. Do something stupid like leaving a buffer open to overflow or lockup and everybody in the room stands up, points at you and laughs. Also important, you never guess at what the client wants or how some resource works--you do the research.

-- MarcThibault

Analyze bugs to discover the (most common) sources of errors and then take steps to prevent them. Or try to detect them as soon as possible.

The Japanese term for mistake-proofing is PokaYoke, for more info see http://www.campbell.berry.edu/faculty/jgrout/pokayoke.shtml

-- KentSchnaith

PokaYoke seems like CandyMachineInterfaces? in WritingSolidCode. -- MartinPool

Knuth categorized the bugs in his log roughly as follows (someone with access to his paper can clean this up):

Mind you, not all of these are really bugs...

In my previous job I was responsible for gathering and classifying all the errors. I think by the 'end' of the project there were about 900 errors documented. For my own interest, I grouped the errors in the following categories:

Some of those categories are overlapping, others may be missing.

The percentages may be off somewhat, I'm typing this in from memory. I had quite fun doing this, but management didn't seem to like the nice colored grafics I produced.

-- LorenzBeyeler

ShootTheMessenger!

Some would say:

Where do bugs come from? Programmers. Which sources of bugs are most significant? Microsoft programmers.

The above was deleted by someone with a microsoft.com address. A little sensitive, are we?

ASCII bugs: 

 ///\oo/\\        Bug


 \\\/++\///       The Last Bug


 ///\--/\\        Bug Eastwood


 ///\`'/\\        Ninja Bug


 ///\></\\        Eric Cartbug

Terminology (paraphrased from "The Computer Bulletin" December 1986 by John Lansdown)

Additions by Dick King (Risks Digest, 14 Dec 1986):

LaurentBossavit, on http://www.ayeconference.com/Articles/Entomology.html wrote

"In the space of one project, many different definitions and uses of "bug" had surfaced:"

"We had accepted these various interpretations tacitly, not really recognizing how they evolved over the course of the project, how different people held different interpretations at the same time, or how these interpretations clashed with each other in various ways."

That list sounds reasonable, but he explains that what people meant was actually the inverse of how it sounds; to paraphrase:

See NotoriousBugs ProofsCantProveTheAbsenceOfBugs TestsCantProveTheAbsenceOfBugs BugTheory

Related: TechnicalFailure

CategoryBug

EditText of this page (last edited November 7, 2005) or FindPage with title or text search