Fagan Defect Free Process

Michael Fagan Associates: http://www.mfagan.com

The Fagan Defect Free Process was invented by MichaelFagan and is about getting skilled, trained observers to look at the item under inspection. This might be code, design documents, interface specifications, product requirements, or even advertising copy. The core idea is that people who already have product domain knowledge are then trained to be particularly keen on finding potential defects in the inspected item. Without going into a long-winded explanation of the training process let's just say that you get your powers of observation tweaked up a bit. The training also involves:

Reduction of ego and defensiveness for the author/creator. It doesn't do any good to have somebody sitting at the table who is going to deny every questionable issue which is a potential defect.
Elimination of the "us vs them" mindset of different groups. Not that hardware and software - Sparky and Mushhead - ever have that problem <ahem>. We have to work together to find the maximum number of defects regardless of what their source is.
Synthesis of observations among the inspectors that leads to more defects being found (the Phantom Inspector). "Hey, I never thought about using it that way. If that happens then..." I was shocked to discover just how many new problems we found that way.
Training-in-place for newbies trying to get familiar with the product domain. Inspection is an excellent spawning ground for new growth.
Death to preconceived notions. Leave your baggage at the door because all the other inspectors have done this. Now you have room for new observations and can find previously unthought-of defects.

[Note: non-Fagan "reviewing" techniques moved to OtherDefectFreeProcesses.]

It would be impossible for me to sum up the experiences I have had in the last few months with the Fagan process. Let's just say that it has been a real eye opener. I want to repeat something I said before: in all my 25 years of professional experience I have never come across something so simple that has so great a potential for process improvement in all areas of product development. I wish Michael Fagan had come up with all this good stuff 25 years ago and I had been introduced to it then. Oh, well. It took him 15 years to accumulate the data that backs up his claims.

One last point: throughout the 80's and 90's, there were cultural wars among the differing schools of "structured analysis" and "structured design" and "design methodologies" ad nauseum. This is the one process improvement scheme I have seen work, in real time, with real product, right in front of my face. I can't make that claim about anything else I have tried over the last quarter century. -- MartySchrader

Participants in the Inspection Process

All participants in the inspection perform the role of inspectors. All inspectors are considered peers for the purposes of inspection. Rank, age, experience, field of expertise, and other factors are left at the door while an inspection is under way.

A Moderator ensures that the inspection stays on track and focused on the target of finding defects. The Moderator also has other duties before and after the inspection such as distributing the work, booking the room and ensuring that the questions noted during inspection were followed up.

A Reader reads through the work, paraphrasing as appropriate for the context. This is particularly difficult in inspecting source code; it is sometimes a source for defects when one inspector understands a line of code to perform some operation differently than another inspector sees. This is the Phantom Inspector at work.

A Recorder documents all the questions raised for later study and possible correction. Not all questions necessarily lead to correction; sometimes there is no defect, just misunderstanding on the part of one or more inspectors.

The Author of the work (or a representative of the team if there was more than one) is there to "present" the work. Normally the Author will request an inspection when the work has met the proper pre-inspection criteria.

An inspection can have more than these four inspectors or fewer, but the optimum number of inspectors is four.

The Process

[Please note that this description just touches the surface; real training is required to make someone a qualified Fagan inspector.]

When work meets the pre-inspection criteria, the Author asks the Moderator to schedule an inspection. The Author identifies the domain expertise required to effectively inspect this work.

The Moderator sets up the inspection, drafting inspectors and ensuring that they can attend and that a suitable venue is available. The Moderator also assesses whether the whole work can be inspected in one session (maximum time 2 hours) or whether it needs to be split into logical chunks.

If the work is overly complex or new, an overview session can be held to introduce the work to the inspectors.

The moderator distributes the work and any supporting inspection material in good time for the meeting. The work cannot be amended after distribution. The Moderator asks the inspectors to record their preparation time.

Each inspector must read through the work and prepare for the inspection. The inspector writes questions about any potential defects. It is expected that the preparation should take about half the time of the intended inspection meeting and certainly no more. The inspection can only be held if all inspectors have suitably prepared. If any party has not prepared the meeting must be re-scheduled.

The meeting must take place in an environment with no possible interruptions from phones, mobiles, etc.

The Moderator reminds the inspectors that the aim of the process is to find defects and not to correct them.

The Reader reads through the work, stopping whenever questions are raised. If it is unclear whether a question is a defect or not it is logged for later investigation.

The Recorder writes down each defect location, description, and severity. No defect is too small; even spelling and grammatical errors are recorded.

At the end of the inspection the Moderator creates a set of reports detailing the inspection results. (These are mostly boilerplate forms that are created for the needs of the group using them.) The primary result contains a recommendation as to what happens next. ACCEPT - No further action needed. ACCEPT WITH RESERVATION - Certain actions need to be completed (minor changes such as spelling, etc.) but no additional inspection will be needed. REJECT - the work contains defects and must be re-inspected once the changes have been made.

The Moderator is responsible for ensuring that all actions are followed up. The defect checklist can not be closed until all the items on it have been ticked off. The defect report should only be distributed to affected parties and not placed in wider circulation.

[Thanks to SimonMedley for the skeleton of this description.]

The way that Fagan works best is when it is used in a no-blame culture. All participants must accept that the main purpose is to find and correct defects and not to lay blame.

I have used the process a number of times to good effect. If used correctly, it is a great tool to improve quality.

-- SimonMedley

Michael Fagan has to make a living too, I guess. I felt a little sad looking at the site. I understand Fagan innovated formal inspections while with IBM long ago, the work having been published back then and being in the public domain for quite some time. Perhaps Fagan himself would be the number one consultant to call in if you're interested in this discipline. And perhaps he also deserves a little more revenue for this contribution to software engineering. Fair enough. It looks like it's not about inspections any more; it's about new, improved inspections. -- WaldenMathews

Where does this reference to "new, improved" inspections come from? Not from Fagan, surely. His site and materials haven't changed in years.

Marty (or others): in some reading about the Fagan inspection process, I keep running across the mantra "do not discuss potential solutions to the defect", but I can't find any reason for that in primary sources. Is this just a way to keep the inspection meeting short, or is there some deeper reason? It seems to me that if having multiple domain-knowledgeable people helps find problems, it would also help generate higher-quality solutions than the original developer working a day later, from just the list of defects. --TimLesher

The purpose of inspections is to find defects -- to ascertain correctness. Anything outside of that activity is not in scope for an inspection. This is why Fagan is so religious about not coming up with solutions or optimizing or anything like that during an inspection.

This is not to say that participants in an inspection won't say that they have an idea they'd like to discuss afterwards. This happened to me and others plenty of times at Baxter and other places when we were rockin' with an inspection and the Phantom Inspector was doing his job. It occurred during a preliminary meeting I had with a partner when we were doing a mock inspection for a potential new client. The Phantom showed up and my buddy and I had suggestions for improving their product even before we ever got the job! Needless to say, that client was impressed and started Fagan training in-house straight away. -- MartySchrader

I feel I must present the contrarian view. I have experienced reviews for hardware design, software code, and documents and found them to be of no value. Issues I have with this approach are:

The assumption that the reviewers will be more knowledgeable about the work that was done than the person or people who did the work.
This is not, by any stretch of the imagination, a "Defect Free Process." Instead, the underlying assumption is that the process is defect ridden. The approach is based upon finding defect; for this to be cost effective, many defects must be found.
That is no fault of the approach. The approch exactly is to take an existing process that admits defects and turn it into one with less (marketing: none) defects. That is like wrapping an unreliable protocol in a reliable one (like TCP aroung IP). I really like this approach as it separates concerns (procedure from quality) and is incremental: You can start from where you are and don't have to start from scratch (which has its own high risks).
This approach is, by and large, impractical. Few projects have the schedule nor resources to go back and do it again, if the reviewers reject the item. One of the few areas where I have seen this approach applied has been in some proposal development tasks. It was viewed largely as drudgery to spend time and overtime to write a document for a forced deadline only to have it rejected and the cycle repeated. The lessons learned? One, throw together the minimal write up for the first review and wait to hear back what is really wanted. Two, avoid working for this manager in the future.
I am not sure an "ego-less" presenter, as described in the references, is desireable. To me, someone who has put forth his best efforts, has his work rejected by outside observers, and meekly accepts this is burnt out and has given up.

I am interested in defect-free processes. By definition, a fully defect-free process will not have defects to be found. I have not had any positive experiences related to inspection-based approaches. I am willing to learn and would like to hear any sort of underlying logic as to why they should be of value and would welcome any explanation as to why inspections should be useful.

-- WayneMack

Let me tell you about my experience with Fagan, Wayne:

One of the gigs I did at Baxter was on a syringe pump used mostly for post-operative pain management. This was a Class IIb, patient intrusive, high reliability product -- obvious with that rating. I worked on various portions of the code and did a major architectural design (which was never implemented, but that's another story).

When Baxter brought Michael Fagan into the Tech Park facility to provide training I was less than enthusiastic. I read the literature and looked at his site and references, but was not impressed because of all the failures I had seen in the decades before. I had been doing this software development stuff for a good 25 years before I ever heard of Michael Fagan, so dude had a tough row to convince me that he had a better way.

But then I participated in the classes and did an inspection on some of my own "released" code.

Bear in mind that this was code that I was certain met the requirements. I had looked at it, tested it, checked it in, used it in builds along with code from everybody else on the team. I would have staked my professional career on that code being bug free. BZZZT! You are wrong, sir!

I wish somebody had gotten a picture of my face when the inspection revealed bugs (plural!) in my "known good" code. These were not bugs that could possibly have revealed themselves in operation, but they were still incorrect. That's how you define a bug.

There was one phunny incident that happened during our training session, although I wasn't in that one: one author was reading a bit from some of his released code for a product that was in the field. This was a systolic pump that was used in hospitals for all kindza fluid pumping. Anyway, dude was reading along when suddenly he froze, staring at the printout, and turned white as a sheet. He then jumped out of his chair and ran -- ran! -- all the way back to the other building, up the stairs, to his desk, and started slamming away at the keyboard. There was a new ROM package for dealer and field technician distribution that night.

Anyway, after my inspection I fixed the code and set up for the re-inspection of the impacted components. Then the code passed inspection with no defects noted. At that point I would have been willing to stake my life on that code being defect free. That's the kind of certainty you get with inspections done through this process.

-- MartySchrader

While I definitely agree that it would encourage production of minimal documents to avoid the manager, I have a larger problem with this. Documents. I have NEVER found any spec in 21 years of professional programming that was more useful than talking. You can't ask a document a question. When you find uncovered areas you want an immediate response from the knowledgeable party and resolve it NOW, not reject the document, send it back, re-review it and get your 1-line answer 2 weeks later.

I say abandon all documents except code and contracts. On-site customer will be a living document of far more worth and accuracy than any ever produced in the history of software development. IMO, there is no substitute.

Documents are a large part of the failure of CMMI, Waterfall, etc. --BrianG

Your experience differs from mine, Brian. Perhaps your exposure to proper documents has been limited by your working venue? Have you moved around a lot? I am a consultant, and have had prolly 20 different clients over the last 30 years. I have seen good documents and bad documents. The presence of bad documents isn't an excuse for blaming documents, though. Bad documents are the result of bad documentation. The entire CategoryDocumentation talks about this in great detail.

The idea of talking about requirements as opposed to writing them down is amateurish, in my opinion and that of most professionals who analyze requirements standards. The point of having a specification that has been inspected is that you can trust the specification to contain a crystallization of all the requirements in their appropriate context. Therefore, you know what is essential to making the product and what can be massaged later on. The spec identifies this and the spec has passed inspection.

Fagan insures that the result of a known-good process will be correct because the process itself has been inspected and the result has been inspected. Inspection is a mechanism that insures correctness. Not optimization, not elegance, not purity of heart -- correctness. A correct document is what you base your work on, and that is the ultimate arbiter of what the product needs to be.

Talking about the "failure" of CMMI, waterfall, or any other process is useless without talking about the correctness of its implementation. Even your favorite methodology will suffer "failure" if it isn't implemented correctly. So, how do you insure correctness? The only sure way I have seen in 34 years of professional practice is Fagan. Seen a lot of "failures," man. Fagan isn't one of them.

-- MartySchrader

There comes a point, however, when a purely in-your-face process won't work. I can readily think of several systems which are computer controlled, where the customers just don't know everything about the product they're wanting, and where the coders are actually more knowledgable. One area, I would suspect, is such projects as the flight control computer of fighter planes. How would XP or Scrum be applied to this kind of project? When human lives are at stake, unit testing aren't enough. You need formal verification, which involves at least some amount of paperwork and is best when the reviewer is disinterested in the project. Otherwise, the interpretation of the proof's notation can be interpreted in a biased manner.

I love agile development methods. They've worked wonders for me, both on my own projects and in my (admittedly not life-sensitive) commercial projects. But, I recognize that other techniques exist for a reason. --SamuelFalvo?

Since >99% of programming projects don't directly put human lives at stake, it is pointless to raise such an extreme case. I'm concerned with the overall health of the industry. For any assertion you can find a negligible counter-assertion, such as there are people who directly benefit from oppressive dictatorships.

Substitute money for lives, something people care about quite dearly, and suddenly the bulk of the software industry complies with Samuel's point. You, AnonymousCoward, are the one who raised an extreme point involving oppressive dictatorships. Samuel merely provided a single data point to a much more broad reality. Consult RISKS digest some time to see how faulty, non-life-dependent software can impact people in very serious ways. Mars orbiter anyone?

A slightly scary quote... Your development processes already in place do not have to change - they are just made crisper and clearer through this process to ensure defect-free flow from marketing and requirements through development through to customer use.

Hmm. What scares you about this statement? If you already have known-good processes that will produce known-good product then Fagan just ensures that the processes work right and have good material to work with, every step of the way. Anything that can cause variance gets inspected, and the causes of variance are corrected in context. When your processes meet requirements then they are defect free. When your work result meets the requirements of the development process then it is also defect free. This scares you?

CategoryProcess, CategoryQuality