National Software Quality Experiment

http://hometown.aol.com/ONeillDon/nsqe-results.html

PROLOGUE:: "The nation's prosperity is dependent on software. The nation's software industry is slipping, and it is slipping behind other countries. The National Software Quality Experiment is riveting attention on software product quality and revealing the patterns of neglect in the nation's software infrastructure."'
ABSTRACT:: "In 1992 the DOD Software Technology Strategy set the objective to reduce software problem rates by a factor of ten by the year 2000. The National Software Quality Experiment is being conducted to benchmark the state of software product quality and to measure progress towards the national objective.
: "The National Software Quality Experiment is a mechanism for obtaining core samples of software product quality. A micro-level national database of product quality is being populated by a continuous stream of samples from industry, government, and military services. This national database provides the means to benchmark and measure progress towards the national software quality objective and contains data from 1992 through 2001."

(For more information, see the link above.)

Here's where to find the numbers:

  http://hometown.aol.com/ONeillDon/nsqe01-results.html

Conclusions:

http://hometown.aol.com/ONeillDon/nsqe01-conclusion.html

and

http://members.aol.com/ONeillDon/nsqe-assessment.html

From http://groups.yahoo.com/group/extremeprogramming/message/44301

"Here's some interesting results:: The foremost defect types that accounted for 90% of all defects detected are show below: Documentation, specifically lack of traceability, accounts for 40.74% of all defects. Standards accounts for about 22.96%. Both of these are examples of organizational neglect. These are followed by logic, functionality, syntax, data, and maintainability defect types which are examples of programmer neglect."

Documentation 40.74% error in guidance documentation
Standards 22.96% error in compliance with product standards
Logic 7.12% error revealed through informal correctness questions function
Functionality 6.55% error in stating or meeting intended
Syntax 4.82% error in language defined syntax compliance
Data 4.66% error in data definition, initial value setting, or use
Maintainability 4.11% error in good practice impacting the supportability and evolution of the software product

What does this say about the development process?

Near as I can tell, from inspecting his site, this is really all about Don O'Neill's consulting company doing a seminar on software inspections and other "quality" techniques. It appears to have nothing to do with the United States Department of Defense (DOD) or any other U.S. government agency.

Let's see...

40.74% for documentation + 22.96% for standards + 4.11% for maintainability = 67.81% of defects found in inspections are for things that most customers and managers don't really care about -- because they don't directly impact the running system. "2/3 of software inspection results don't directly improve the operation of the program."

So isn't this an argument that software inspections are largely a waste of time? (like 2/3rds waste?) ;->

Reading O'Neill's "conclusions" page (http://hometown.aol.com/ONeillDon/nsqe01-conclusion.html), which says that there's no scientific evidence that we're anywere close to cutting defects to 1/10th (from 1993 to 2000), I think we have a solution: ExtremeProgramming. ;-> -- JeffGrigg

CategoryQuality