Cleanroom Software Engineering
Cleanroom Software Engineeringhttp://en.wikipedia.org/wiki/Cleanroom_Software_Engineering
Also known as CorrectByConstruction (as Prexis uses it http://www.praxis-his.com/) [was CleanRoomMethodology]
The "Clean Room" methodology is a radical departure from traditional development techniques: The developers who write the code never test it. That's right -- the developers who write the code never run their own programs.
Developers do CodeReview of each other's work, with the intent of demonstrating the correctness of the code -- without running it. This is a little like a ProofOfCorrectness, but not as formal, tedious, expensive, time-consuming, etc.
When the developers are satisfied that the code is correct, it is ThrownOverTheWall to another group who runs tests. It is not the responsibility of the QA group to find bugs. It is the responsibility of the QA group to verify that the code has achieved the desired level of reliability.
Teams using this approach have reported defect rates of 1 in 1000 lines of code or lower (equals 99.9% or better of bug-free code). Such teams are also consistently more productive than those using traditional methods.
Related Pages:
"Insight: It may cost less to leave the bugs out of the code entirely instead of paying to put them in and then paying again to take them out" -- TomDeMarco/TimothyLister in editor's note on Chapter 16 of SoftwareStateOfTheArt. (See CleanRoomMethodologyEmpiricalEvaluation in "Related books" list below.)
CleanRoomMethodology isn't ExtremeProgramming. It's the opposite of XP, in some respects.
Aside:
What is known about the cost of CleanRoomMethodology, the cost-effectiveness, and the kinds of apps where it makes sense? And ... is it fun? What organizations are actually doing it? -- AnonymousDonor
In books I have read, people who have successfully applied CleanRoomMethodology say that it reduced costs. That is, they measured productivity improvements, measured in lines of code or function points per man month.
As for fun ... They report that developers using CleanRoomMethodology tend to be more conservative -- using simpler and more direct approaches, as they're easier to "prove correct." This doesn't sound as fun as using bleeding edge techniques. But, on the other hand, in my (limited) experience with PairProgramming, I've found the social programming environment much more engaging and challenging than the traditional every-man-for-himself style. -- JeffGrigg
I read the "Cleanroom Software Development: An Empirical Evaluation" paper again, and posted a (rather long) summary of it in CleanRoomMethodologyEmpiricalEvaluation. I think it successfully presents evidence to the effectiveness of cleanroom in improving quality. Please have a look. -- JeffGrigg
How does cleanroom technology fit in with the SpikeSolution-Pattern? -- CayteLindner
(See also SpikeDescribed and PerformanceSpike.)
It seems that no one here has actually used the CleanRoomMethodology, so I'll describe my interpretation of what the books said...
While clean room is nothing like ExtremeProgramming, it does encourage a type of "SpikeSolution:" If documentation and knowledge is insufficient, programmers can code and run test programs that are NOT delivered as part of the system. Like "what is the maximum string/message size for the win32 function OutputDebugString??" None is documented, but experiments have shown that some strings over "a hundred and something" bytes long get truncated. A quick SpikeSolution test program tells the programmers to keep the buffer size at 100 bytes or lower to avoid the undocumented limitation.
In much the same way that XP allows a person to experiment alone without a partner ("violating" PairProgramming-Pattern), they're not allowed to put such code into production: They must throw it out and rewrite it with a pair partner. Likewise, books on clean room development say that programmers never run or test their code -- except to run short test programs used to clarify inadequately documented aspects of the environment. Such code is thrown away before writing the production code.
Yes, one should NeverUseUndocumentedFeatures?. But EveryRuleHasAnException? -- sometimes you really have no other rational option.
-- JeffGrigg
This sounds more like AskTheComputer than a SpikeSolution. Still, I've worked with libraries/systems that are so poorly documented, poorly understood, and poorly behaved, that I don't think there was any way I could have written software for the machine without actually touching it (actually, never managed to write software for it anyhow, but never mind that detail).
I think this defines one of the boundaries for the cleanroom methodology: When the underlying system sucks so bad you can't understand it, there's no way you're going to be able to write code for it a priori. -- BillTrost
The thing I like about clean room is it has a simple set of assumptions, then pushes them to the limit. There is a certain intellectual honesty to the approach. I can well believe they achieve outstanding results for certain kinds of projects. --KentBeck
Would two applications, one developed under the CleanRoomMethodology and the other using ExtremeProgramming, exhibit radically different internal structures? I know nothing of CleanRoomMethodology other than what I've read here, but it seems to me that given WellFactoredProgramsCannotBeUnderstoodStatically, a CleanRoomMethodology will tend to create procedural/linear code versus the multidimensional code that ExtremeProgramming produces. -- MarkAddleman
I don't understand what you mean by "multidimensional code". Both XP and Cleanroom push hard on the side of simplicity. Cleanroom enforces simplicitly because complicated code is unlikely to survive peer review ("go rewrite that code so we can understand it").
My very limited experience doing Cleanroom in the small is that the source code may end up less heavily factored than what you might get from XP, but that is probably a side effect of the very heavy factoring in the design, combined with my use of NoWeb to turn design more-or-less directly into implementation. --BillTrost
<speculation>
Seems like the results would be significantly different, with XP favored for factoring and ease of change.
With XP, you can change anything you want in as many places as you want, and rely on the tests to protect you from error. In cleanroom, you'd have to convince your peers that every change to every class could not possibly have introduced an error. This could be very difficult and time-consuming, discouraging people from doing it.
Also, I suspect that cleanroom might be "limited" by its "slowest" or "most nervous" or "most conservative" members: Any one member could at any time say, "I just don't feel comfortable that that new way of doing it, with inheritance [or polymorphism, or templates, or generics, or patterns, or whatever...] is entirely so well understood or reliable. It makes me nervous. Why can't you just go back and do it the old tried and true way; the way we've always done it in the past?" -- JeffGrigg
</speculation>
EditText of this page
(last edited May 17, 2012)
or FindPage with title or text search