Xp And The Cmm

NOTE: Since CMM has been superseded by CMMI, a new page has been created for that topic: XpAndTheCmmi.

XP Magazine article asking what SEI CapabilityMaturityModel level ExtremeProgramming might be:

http://www.xprogramming.com/xpmag/xp_and_cmm.htm

On CMM level 5: Adding unit tests to address bugs found is a very good practice, but I don't think it addresses the issues of CMM level 5. In CMM, the "causes" of problems are assumed to be processes that encourage error rather than making it easy to not make errors.

I think you have a very good argument for ExtremeProgramming being at least CMM level 3, which is a lot better than most organizations in the industry. -- JeffGrigg

What if every time you had a defect reported from the field you had a meeting to discuss what kind of unit or acceptance test you would have needed to write to prevent the defect from escaping, and you are careful to write that kind of test in the future? -- KentBeck

Yes; that's it. -- JeffGrigg

That could well hinge on how much critical project and process information is actually documented in XP. Is there any standardized view of what that is?

As I read the CMM, no particular documents are required to be produced. And XP produces a lot of documentation, most of which is machine-verifiable. Doubtless there are auditors who would require more paper, however. I've found a CMM auditor who is interested in XP and its certifiability - actually he found me - so there may be more on this in the future. -- RonJeffries

Yes, the auditors I've seen seem overly obsessed with paper - even when the paper really has very little to do with the way the actual project is being done. On the other hand, if your auditor can only recognize stains on dead tree carcasses, you can always print relevant source code for them. ;-> -- JeffGrigg

Is there anything special about ExtremeProgramming that it allows you to pass CMM Level 3? It seems from the description of CMM level 3 that any well-defined process which is being adhered to will allow you to pass CMM level 3. The questions are then: Is it easier to use XP than any J. Random Process (especially if the purpose of the process is to pass CMM certification)? Does XP help you more than J. Random Process at CMM level 4 and 5? -- JohannesBrodwall

I haven't found many developers who really like to write a page of documentation for every attribute and method in every class. But this is what CMM/QA auditors have required on several projects I've been on. So, XP, having practices much closer to what developers would normally want to do (once you get over PairProgramming issues and such), would be easier to implement consistently; your people wouldn't be constantly whining at you and trying to avoid the official process. -- JeffGrigg

The problem with the comparison is that it uses the general description of the CMMLevels as a basis. For an organization to reach a certain level however, it should implement each of the key process areas for that level (for level 2 that means Requirements Management, Software Project Planning, Software Tracking and Oversight, Subcontract Management, Configuration Management and Software Quality Assurance, assuming we're talking about version 1.1 of the CMM here). Implementing a key process area means, again according to the CMM, that the goals of that key process area are met by the organization (NB: the complete organization, not just one team). So a better comparison would be to discuss what goals are met by an organization that uses XP as its (standard) software process. -- FrankNiessink

I've written a paper on Process Discipline vs. Development Agility that's published in the November 2001 issue of Cross Talk: The Journal of Defense Software Engineering. Starting with the assumption that there are plenty of people who think process discipline and development agility are mutually exclusive, I use CMM and XP as examples from each camp and to make my points. I walk through the origin of that misconception and conclude that CMM and XP can live together, and be mutually productive, therefore process discipline and development agility can co-exist on the same development effort.

As far as CMM assessors who are dead-tree-centric, I agree that there's a vast preponderance of those sorts of assessors. It's hard to blame them or the CMM though; these assessors are a product of their pasts and have just as hard a time growing out of their old ways of thinking as any one else does. It's too bad, OTOH, that such people are in positions to make life miserable for developers. Happily, I work with one of the more "progressive" CMM assessors who's much more interested in good processes than dead trees proving good processes. -- HillelGlazer

You can see the above paper at:

Excerpted from the paper:

In other words, most XP projects that truly follow the XP Rules and Practices could easily and quickly be assessed at CMM Level 2 if they could demonstrate having a process for the following:

See also Mark Paulk's write-up Extreme Programming from a CMM Perspective. Mark Paulk is one of the authors of the CMM, and so has an extremely good understanding of that side. He thinks that XP does a lot of good stuff. He also points out that CMM level 3 and beyond focus on the entire organization, not just the development team. That means that a process that only deals with the team level, be it XP or something else, is not going to be sufficient by itself to meet the requirements for the higher CMM levels.

http://www.sei.cmu.edu/cmm/papers/xp-cmm-paper.pdf (has moved to http://www.sei.cmu.edu/publications/articles/paulk/xp-from-a-cmm-perspective.html and ftp://ftp.sei.cmu.edu/pub/documents/articles/pdf/xp-from-a-cmm-perspective.pdf)

I find the practice of tracking estimated vs. actual task duration variances quite common among the XP community in Oregon. Does this not qualify as QuantitativeProcessManagement? at CMM level 4?

It becomes clear very early when story and task estimates are consistently > 100% off and the PlanningGame is in need of tuning. -- MichaelLeach

Tracking estimated vs. actual is an important part of CMM level 4, but they're talking about at an organizational level, not at a team level. That is, the question is, "Given that we've standardized on process X, (back at level 3) what can we say about our capability to complete project Y with team Z?" This is a silly question for the typical XP team, but a very important question if you're the federal government, and you want to sink billions into Aegis cruiser software, or the like. -- JohnBrewer

It also depends on what you do with the data. The key idea of process management is that you manage (the changes to) the process. QuantitativeProcessManagement? requires a measurement basis for those decisions. Just tracking deviation from estimated to actual effort is not management, let alone process management.

I just read HillelGlazer's paper on the Xp and the Cmm referenced above, and wanted to note exactly those lines here on wiki (then saw that he had done exactly that). It's a good paper, by the way, you should go read it.

I feel these lines capture the proximity, and the distance, between XP and CMM oriented organizations.

"All you have to do" is those things... Most companies interested in the productivity and maneuverability of XP won't be interested in spending the money for those things. Most companies needing CMM 2 certification won't be interested in that level of maneuverability. I suspect the two will never collide, and I also expect to be wrong in this expectation.

One day, in the not-too-distant future, maybe there will be an organization that is committed to CMM 2. And actually wants productivity, maneuverability, quality and CMM Level 2 all in the same breath. And at that instant, they will care to introduce an external auditor to check on the Activities of XP, not just the deliverables. Do the people really do their daily stand-up meeting? Do they really program in pairs? Do they really refactor? Do they really write tests before writing code? Some of these things can be detected automagically from the check-in logs, so the team can write scripts to demonstrate. Others need a spy-cam, which will unnerve many XP proponents.

The two are close. Can they meet? Dunno. But that list of To-Do's is a really good measure. -- AlistairCockburn

The overhead of proving things is always a burden, especially to those who already believe. I think that's the crux of what's presented on this page. To pass a CMM audit, you gotta have the proof. To stabilize a software development effort, you need some attention to requirements, project tasks, commitments, error correction, etc. The CMM does a good job of presenting that stuff in a tiered fashion. You can apply bits of CMM wisdom and get improvement. I've done it. Proving it, assuring that it happens - those are a different game. You don't go around proving everything in daily life unless you have extraordinary reason to do so.

-- WaldenMathews

Well, from my experience, most teams that say they're doing XP don't actually do the practices. Having the auditor, while a pain in the x, would at least assure the management that the team is doing what it is saying it is doing. -- AlistairCockburn

I wish that were true, but how does the presence of an auditor actually accomplish that? Doesn't it depend on the reasons why the team isn't doing the practices? And if the presence of an auditor became the sole reason for doing them, then what does that say about the practices?

No, I have come to reject the notions of auditor for the same reason I oppose nuclear power: it's so powerful that it's only good for stuff you never want to do. Moreover, in this particular case, let the management move amongst the project and form their own opinions of what is being done and what isn't. How come they don't do that, anyway?

-- WaldenMathews

Thus illustrating "the proximity, and the distance, between XP and CMM oriented organizations." Cheers, -- AlistairCockburn

Also, there are ways to prove you do something just by the way in which you do it, without adding overhead. Yes, it requires some up front process planning and design, but once it's built out the process runs and gets the auditor what they want without having to burden the developers. And yes, if you only do a process because an auditor may come around, then the process is the problem. I'd like to suggest that what many people accept as the notion of the auditor's role and the audit process is unfortunately very narrow. There are ways of auditing that are neither "Big Brother-esque" nor intrusive. Maybe the problem is that there are too many organizations as well as auditors setting themselves up for audits using very non-progressive approaches. Approaches that no longer match the technology. What if processes were designed and followed because they made sense and value doing so? What if auditing was reduced to ensuring that the macro-processes were in place and not the micro-management that so many audits have become? -- HillelGlazer

I've only seen a few CMM audits, but the ones I've seen have been uniformly an exercise in deception: No way was the development team doing what team leaders told the auditors! When a more through audit was expected, team leads coached the developers as to what to tell the auditors. This, is real life.

As I see it, CMM audits have generally been something "top management" subjected teams to, to further some apparently unrelated political objective.

On the bright side, I hope to see people adopt XP because they want to get lots of work done. -- AnonymousCoward

Yes, it's easy to "pull the wool over" a process auditor's eyes, whether it's for CMM, ISO, or many other audited processes. But it's safe to say that any such organization feeling the need to and/or finding themselves doing so has completely missed the point to having, using, or pursuing the processes in question. The point of the processes ought to be to add value to what you do, not to win contracts.

Has anyone ever been in an organization that was doing something "for political purposes" that didn't result in dissent, disassociation, resentment, and ultimately an AntiPattern? Organizations that deceive the process auditors may succeed in the short term achievement of the parchment they seek, but they also succeed in the long term disservice to the very same credibility that they tried to gain by getting the parchment. By conducting such deceptions, companies are exasperating the problem in every possible way from adding effort to risking failure. (Not to forget mentioning the fact that if they really are operating without processes they are asking for serious business problems.)

This deceitful approach contributes directly to the problem of "proof." If organizations weren't wasting so much time trying to prove they followed a process, and instead they actually followed processes that generated their own proof, then the added burden, overhead, and resentment of proving a process was followed as well as the auditor's need to seek this "objective proof" would simply go away.

The reasons so many of these process audits came about and the manner in which they are audited, and the reasons so many organizations find the need to deceive auditors are because of several factors, including (but not limited to):

(not a complete list, I know)

Overall, formalized process discipline ought to be implemented only if it makes sense in your organizations. The extent to which and detail of the process formalization are, again, a function of the value it brings your organization. Organizations that have and use processes that they like, and that are good for them, and therefore they don't want to change, should not have to revamp their processes to get CMM "certified." A creative process consultant with understanding of CMM and how it's audited should be able to help your organization find ways to "prove" you follow CMM processes without having to add a layer of burden, and without doing anything stupid like torqueing off your developers and screwing up your rhythm. And without having to deceive the auditors.

-- HillelGlazer

Sounds like a sales pitch for a "creative process consultant". Are you one of those, by any chance, Hillel?

-- WaldenMathews

Making a good case for taking action is what a sales pitch is supposed to do. But not every case for taking action is necessarily a sales pitch. A visit to my Wiki presence will answer anyone's questions. -- HillelGlazer

Given the degree to which Indian software companies dominate the CMM rankings of the world, this page got me to thinking about OffshoreXp. -- BillBarnett

I once discussed the CMM with a 65-year-old aerospace engineer who had worked on software for the Apollo missions. He told me that in his opinion, the CMM was backwards.

How about...

  1. Initial
  2. Optimizing
  3. Managed
  4. Repeatable
  5. Defined

I can't remember his exact explanation, but I'm guessing it went something like this:

The first thing you need is an ad hoc process. The second thing you need is a way to improve your process. Then you need a way to judge your improvements. Once you have these, your process will get better. When your process is good enough, it will start being used by others. When enough people use it, you can observe what they are all doing and write it down on paper.

If you don't use continuous improvement to get to your process, you have to do big design up front to get to your process. If you do this, chances are the process won't be usable, and will just sit on the shelf in a notebook.

In other words, you have to figure out a good process first, by trial and error, before you can write it down.

Those who have been on Wiki for a while can remember Kent and Ron struggling for over a year to describe their XP process, after they already had it working.

We are much beholden to Machiavel and others, that write what men do, and not what they ought to do. FrancisBacon, The Advancement of Learning, bk. 2, ch. 21, sct. 9 (1605).

KeithBraithwaite would like to use this anecdote in a talk, could whomever posted this drop me a line, please. TIA

I often think that giving your development process a cool-sounding name is the first sign of a solution looking for a problem. Define the vision for your new software development process (cheaper solutions, more business value, quicker, whatever you think's important), set realistic goals, define measures and targets, describe fully (and realistically) how you currently do things and then redesign your processes/practices in a way that helps you meet your targets. Automate parts of the process that make sense to automate. Buy tools only if they will pay for themselves. Call it "XP", call it "PSP", call it "The Extremely Unified Development Method" if you like.

If at the end of that you end up with XP (and undoubtedly parts will look similar), then that's nice, but then again, who cares - just as long as you meet your goals? No?

-- JasonGorman

The following was posted to the extremeprogramming Yahoo! group, http://groups.yahoo.com/group/extremeprogramming

The XP/CMM subject comes up periodically on the list.... I reprint my post in its entirety here (minus the tagline): 

 * * *
I've been lurking on this list for a while. Well... a while in Internet terms.

The debate over CMM (or CMMI) and XP comes up over and over and there are several points that I have not seen made so I'll jump in the water and make them myself. (In no particular order of importance.)

(Warning: this got long... due to the many misconceptions and unspoken issues generated by this topic.)

  1. There are situations appropriate for XP and situations that aren't.
  2. There are situations appropriate for CMM and situations that aren't.
  3. These situations in #1 and #2 are neither mutually exclusive nor collectively exhaustive.
  4. CMM is about managing software projects, not about developing software products.
  5. XP is a little of both, a little about managing software projects and a lot about developing software products.
  6. Where the XM and CMM vin diagrams intersect can be managed -- by CMM allowing the XP management processes to be the "defined and controlled" process CMM wants you to have.
  7. BOTH XP and CMM will fail if followed by or managed by goobers. This leads to a corollary: if you don't acknowledge #1-#3, you're doomed.
  8. BOTH XP and CMM will fail if not done in the way the framers meant them to be done (that includes those parts that say "don't do this piece if it doesn't add value to what you're doing").
  9. As far as CMM assessments go, as a frequent CMM assessment team member and professional CMM consultant, I'm the first to admit that assessors are the reason why so few XP'ers (attempt to) attain CMM(I) maturity ratings, or vice versa.

To edify #9: The problem has to do with legacy processes that won't let go. Many of the assessors (in the US) come from backgrounds of legacy software development life cycles whose scope criss-cross between development methodology and management methodology. And many people can't see the distinction between the two.

Furthermore, there is a parallel legacy problem with government standards. Until the CMM, government standards for software told practitioners and auditors _both_ WHAT to do as well as HOW to do it. The CMM doesn't say how to do anything. OTOH, CMM does suggest how companies typically address many of the requirements. In the absence of a defined how, CMM assessors, consultants, and many practitioners take these typical examples and make them into requirements for achieving CMM. This is flat-out wrong-headed and counter-productive.

Speaking of the CMM as an entity, the CMM only wants:

as determined by whether and as well as That's it, in a nutshell. Nothing else.

As far as I'm concerned, XP is a life cycle that works well when it is applied properly, and CMM is a way to manage people doing software which works best when it is applied cost-effectively. Deviations from being focused on doing a good job for clients and doing right by the company are a complete waste of time and energy. Whether it's XP or CMM doesn't prevent deviation.

Here's an analogy I've used before: XP and CMM are as distinct from one another in the realm of software, as blue prints and inventory control are distinct in the realm of widgets. Although you want the two to complement one another, neither is necessarily dictated by the other.

Cheers!

-- HillelGlazer, http://www.entinex.com

Anyone interested in this line of discussion - or CMM and <anything> - should take a look at the CMMI (CapabilityMaturityModelIntegration) which is superceding the SW-CMM. Also, check out the XpEvaluationFramework ... it is not an audit (but then, properly used, the CMMs aren't either :), but it is intended to provide a way to measure a team's adherence to the XP practices.

-- KarenSmiley

Has anyone actually taken XP through a CMM audit? If so, what level did it come out? Given my experience as a subcontractor to a CMM Level 4 house, I don't think it would pass a CMM audit. I think XP is one of the better approaches to development, I just don't think CMM and CMM audits will see it that way.

Yes, see http://www.xpuniverse.com/schedule/papers for "An Agile CMM". We got our certification in December 2003. Only thing we needed to add really was an independent QA officer and a QA group (consisting of all QA officers). We recruited the QA officers from the engineers working on other teams. This worked very well, since they are also technical trained people. -- ErikBos

EditText of this page (last edited June 27, 2006) or FindPage with title or text search