Capability Maturity Model

The SEI CapabilityMaturityModel specifies an increasing series of levels that a software development organization can be at. The higher the level, the better the software development process, in their theory.

Reaching higher levels is (can be) an expensive, time consuming process.

Note: The SW-CMM, often referred to as just the CMM as it is here, is being "sunsetted" by the SoftwareEngineeringInstitute and superseded by the CMMI (CapabilityMaturityModelIntegration). See http://www.sei.cmu.edu/cmmi/adoption/sunset-faq.html.

The SW-CMM framework comprises five levels of process maturity, each with specific characteristics and features.

Level One: Initial

The software process is characterized as ad hoc, inconsistent, and occasionally even chaotic. Defined processes and standard practices to the extent that they exist at all are summarily abandoned during a crisis. Success depends on individual effort, talent, and heroics. The heroes, of course, eventually move on to other projects or organizations, taking their knowledge with them.

Level Two: Repeatable

Basic and consistent project management processes are established to track cost, schedule, and functionality. The process discipline is in place to repeat earlier successes on projects with similar applications. However, the discipline and process while established varies from project to project. Program management is a key characteristic of a level two organization.

Level Three: Defined

The software process for both management and engineering activities is documented, standardized, and integrated into a standard software process for the entire organization. All projects use an approved, tailored version of the organization?s standard software process for developing and maintaining software. Consistent practices across the organization minimize the learning curve for people moving to new teams and projects.

Level Four: Managed

Detailed measurements of software process and product quality collected throughout the organization drive strategic analysis. Both the software process and products are quantitatively understood and controlled.

Level Five: Optimizing

The key characteristic is continuous, pro-active process improvement, enabled by quantitative analysis of the process and by piloting innovative ideas and technologies. In other words, continuous improvement becomes institutionalized into the development process.

An associated process model is at http://www.sei.cmu.edu/products/publications/sw.proc.framewk.html. The Carnegie Mellon associated SoftwareEngineeringInstitute has their CMM model at http://www.sei.cmu.edu/cmm/cmms/cmms.html.

Always remember that there are two ways to use CMM:

- Using it to actually improve your process.

- Using it to get a certificate -- KariHoijarvi?

Someone who believes in it should expound: my belief is that the defined process and its implied choice of what to work on and when make it a good fit only for the government software market, where you will be paid if and only if you implement the externally supplied specs and whether or not the software does "the best/right thing" is fundamentally irrelevant. -- MarkSwanson

Aw, don't be so hard on us government contractors. Believe it or not most of us are trying to deliver useful software to help end users do their job. :-) -- WayneMack

One of the principles expounded by WattsHumphrey, the creator of the CMM, is that high quality software can only be produced by a high quality process. (The truth is that in fact high quality software can be produced without a high quality process, but the chances of having a good quality product seem to be highter is your process is also a high quality process)

All software developers are following a process, but only some software developers are following a documented process. For example, the ExtremeProgramming process is becoming more and more well defined right here on these pages. -- KentSchnaith

One can nitpick about each practice of each key process area (KPA) in the CMM and whether something in XP prevents you from being able to put a check mark in each box. But when you get down to the essence of it, it seems to me that the XP pages give ample evidence that XP is both repeatable and defined. Furthermore, folks like Mark Paulk (one of the CMM authors) and others are repeatedly suggesting how the CMM not only can, but should be tailored and scaled-down for smaller projects, and less militaristic projects. In fact they claim that is what you are supposed to do when using the CMM as a reference model, and that too many rigidly interpret the CMM too literally when trying to implement CMM-based improvements.

Frankly, from what I've seen of XP here on the wiki, I see little reason why XP should be unable to pass a CMM-level 2 or level 3 audit, as long as someone was willing to document the "tailoring" appropriately for the auditors. The dearth of design documentation might be a potential sticking point, but one that I think could be worked out. And as far as design/analysis methods, XP has more of that than shops that I personally know of that have been formally assessed at CMM levels 2 and 3. But I think it would be unproductive of the XPerts to perform such a tailoring unless contractually obligated. -- BradAppleton

In my eternal naivete, reading the industry rag descriptions of CMM, I thought it was a good thing: Programming shops repeat processes discovered to yield repeatable results. But this page contains reports by folks who have actually experienced the Dark Side of CMM - that it is actually a Process itself, and that following the Process is more important to these "CMM Certifiers" than shipping good code.

Hmmm. It looks like using government and other local folk traditions to enforce someone's opinion of "Best Practice" might just not be BestPractice! Who would have thought it? -- PhlIp

Interestingly, I went to a lecture with Mark Paulk at the Pittsburgh Software Process Improvement Network, where he mentioned XP several times. He also said, "We're really interested in what XP may do for extending process improvement to smaller projects and organizations." -- JohnDuncan

CMM related resources

Start of Discussions

A CMM assessment is about having a defined process and having proof that you are following the process sufficient to satisfy the auditors. Unfortunately, the auditors all went to the the WaterFall school of software design, and the CMM reads like the WaterFall process: "Complete Requirements, Complete Design, Complete code, Test code". -- ChrisRule

A CMM assessment is about having lots of supporting documents to satisfy auditors. The auditors are not concerned with software produced nor processes followed, only the existence of documents. The reality is that CMM certification results in the creation of an additional production line building product for auditors in parallel with the production line building software for customers. Finally, this is not the fault of the companies in following CMM, it is a basic flaw in CMM itself. -- WayneMack

For one man's biased comments on XP and CMM, see http://www.xprogramming.com/xpmag/xp_and_cmm.htm. -- RonJeffries

Relatedly, I have no doubt at all that we could easily certify XP to ISO-9001. I rather expect to do that within the next 12 months (starting mid-Nov, 1999). And I'd bet a few months' pay double or nothing that we could certify an XP team to at least SEI Level 3. -- R

The stated "essence" of ISO-9000 is simply "say what you do; do what you say." I don't see any obstacles for XP there. A bit of drudge-work perhaps, but nothing even remotely insurmountable. I would however be interested in the type and amount of effort it requires though. Please keep us posted. -- BradAppleton

I have worked something very close to XP for years, in the Air Force - and we have ISO 9001 certification. I have now started my own company and have started ISO 9001 certification - I am thinking of using Kent's book ExtremeProgrammingExplainedEmbraceChange as the basis of my procedures. I fully agree with Brad that this should be quite simple, after all XP is a well defined methodology, and Kent's book really is a book of procedures for XP. -- NissimHadar

Nissim, please keep us posted on how your effort goes! (Possibly on the IsWikiIsoNineThousandCompatible page.) -- JohnBrewer

But WhatCmmLevelIsTheSei?

Forget that. That's like asking MS to pass one of their own label programs.

What XP level is the SEI??

Well, as far as I understand XP, there are just two levels, On and Off. For the answer to the reversed question, try XpAndTheCmm. -- The WikiGnomes

Ron recently described two XP levels to me. Level 1 is pure XP, no changes to the process, using the majority of the patterns. Level 2 is adapted to suit the situation but still maintains the underlying goals, and some of the patterns. -- BryanDollery

In Chapter 29 of the 2nd Edition of PeopleWare, TomDeMarco and TimothyLister make the following provocative statement:

The projects most worth doing are the ones that will move you DOWN one full level on your process scale

That is, a process is a process for doing something. The way to up your CMM level is to keep doing very similar, low-risk things, while you focus on improving your process. Meanwhile, a paradigm shift in the world may mean that what your company really needs is a brand new something that you've never even attempted before.

For instance, you're a CMM level 3 shop that does client-server database work. Management dictates that you go for level 4, and ties your compensation to your successfully being certified at level 4. Now, the rest of the world is moving to web-based applications, which you have no expertise in. Your company needs to get into this new area, but it's almost certainly going to screw up your process until you get acquainted with it. Do you risk your level 4 bonus, or do you do what your company needs? -- JohnBrewer

If you're level one, the advice above would say that you should choose a project that renders you level zero. I'm not being glib. There are times when I overwhelm myself with the unfamiliarity of too many new things. I don't produce good value in that mode.

If a group is CMM level three and wants to introduce a little chaos in order to expand market share, I don't think the CMM or SEI have anything negative to say about that. It makes sense as a considered tradeoff. What doesn't make as much sense is to ignore issues of process capability altogether. The CMM provides a framework for thinking (and acting) about that.

Remember, it's not like God wants us all to be level five or something. -- WaldenMathews

This prompts me to the question: Is CapabilityMaturityModelForLiving? -- GunnarZarncke

I think the point is that if a process is repeatable (a precondition for improving it) we should automate it. The only tasks humans should be involved with are not repeatable. Build robots to do the repetitive work. -- EricHodges

Many of us have worked at places whose maturity level can be found in the CapabilityImMaturityModel.

And the only places which are currently CMM5 are places where the programmers get paid peanuts - they're the only places which can afford the OVERHEAD.

My impression of the CMM is that it has nothing to do with software quality. It is about process quality with a focus on the ability of an organization to meet budget and schedule commitments. Having documented, repeatable, measurable processes helps with estimation. Software quality is not really the focus, except that an organization must be able to predict when the product will have a sufficiently high level of quality to be considered "finished", and an organization that makes good estimates won't find itself in a position where it has to change plans and trade quality for time or money. -- KrisJohnson

Well, the primary purpose of the quest for process quality is product quality, on the theory that the quality of a product is dependent upon the quality of the process used to produce it. WattsHumphrey has stated this many times in his talks on the CMMs, and on PSP/TSP which are definitely oriented to achieving high product quality. Using CMM assessments for purposes other than guiding an organization's efforts to actually improve constitutes misuse (IMHO). Almost any tool can be used as a blunt instrument, but that doesn't make it a bad tool.

Having said that, there can be many different ways to achieve a high-quality process, which is why the CMMs (correctly applied) acknowledge 'alternate practices'. Certainly, arguments can be made that properly used, various agile methods constitute high quality processes.

The SEI has been collecting data for a long time on correlating [process quality as measured versus the CMM] to improved [product quality, productivity, costs, ...] and the resulting ROI. For a recent report, see http://www.sei.cmu.edu/publications/documents/03.reports/03sr009.html. To date, the body of comparable data on agile methods is much smaller; for XP at least, LaurieWilliams and BillKrebs are working on addressing that. See XpEvaluationFramework.

-- KarenSmiley

I am only half tongue-in-cheek when I suggest:

  1. You make mistakes
  2. You make the same mistake over and over again
  3. You know you're going to make a mistake before you make it
  4. You make the mistake you intend to make
  5. You can choose the mistake you make

-- SkipSailors

Actually, I believe the pursuit of Product quality and process quality are not necessarily linked. Process quality gives you a better way of developing software, where as product quality just gives you better software. (Maybe we should quantify better?) Basically if a project team had a crystal ball and could look at what their clients wanted in 6 months and then just write THAT software, they would likely be fired at the 3 month check in point. (Well they weren?t doing what the client asked for, though they were writing better software.) Their process wasn't acceptable to their client.

So I believe that good process makes acceptable software, if and only if the client has the willingness and ability to communicate what they need, and then the ability to realize when they have gotten there. XP I believe makes this more possible simply because people are less likely to forget in 2 weeks than in 6 months. I know that this is what requirements documentation is for, but 6 months from the start point those documents are all but irrelevant if the software doesn't meet the needs.

So really, acceptable software is what I believe XP can help produce. And I think it raises the chances that a client will find the software acceptable. However, I don't believe that it will make all projects successful nor do I believe its a solution to entropy. Systems will break down with XP or without it. And yes, lots of projects using XP are canceled. So what?

To close, if you can't look at the things that XP gives you, and say, "Yes, that would be valuable to me." Then XP isn't right for you. A lot of clients don't want to be involved with the process of there software development. A lot of developers don't want to tell their clients how far along they are. You can keep those clients and those developers. I'll keep XP. -- Todd Edman

I've just helped put together the CMMI User Group at: http://www.capability-users.com

We have been struggling though with the relationship with ISO 15504. The SEI have always made positive noices about alignment and such, but we struggle to find anything concrete. If anyone has anything on this I'd love to see it posted. -- Jayne Marshall

As a RPG player, I have only one question: How much XP does it take to go from one CMM level to the next? But seriously, since the stated goal of CMM is to avoid heroics and downplay/impede all GrandMasterProgrammers, you can almost mathematically prove that it's a bad philosophy.

The essence of CMM is to be able to repeat your successes and to avoid repeating your failures, across the organization and down through time. This means that the guy you hire for Team A next year learns from the successes and failures of the Team B guy who died last year. This takes not just great documentation but also the merciless kind of introspection that's hard on self-esteem. This is possible with XP but, given the culture, unlikely. -- MarcThibault

The problem is that you can't factory-produce software. Developing software is more like team surgery, where competency, experience, group chemistry and knowledge of the patient go a lot further than a set of processes for how the surgery should be performed.

Perhaps one *can* factory-produce software. However, a big factor software creation and maintenance effeciency within a domain/shop is experimenting with and improving software-based abstractions for that shop. One must deviate from prior approaches if they want to pioneer new domain abstractions. But managers are risk-aversive and repeating a known but inefficient process is sometimes to their liking. This is where CMM shines. Whether this is good or bad probably belongs in a trade-off weighing topic such as [will insert when re-found].

Perhaps you've not seen that surgeons are starting to use processes now? Because it improves their work. http://www.npr.org/templates/story/story.php?storyId=122226184

However, processes don't make up for competence, experience and knowledge.

See: InstinctAsIntelligence, CargoCultSoftwareEngineering, LevelsOfSoftwareSuccess, CmmIsEvil, MetaSignal

CategoryMethodology

EditText of this page (last edited May 20, 2012) or FindPage with title or text search