Critique Of Xpxec

[moving from CritiqueOfXp]

One major problem XP has is that ExtremeProgrammingExplainedEmbraceChange (XPXEC) doesn't do much to dissuade people from extending XP into places where it doesn't belong. Maybe (probably) this is because XPXEC was the first book in the series. It was written before a lot of case history had piled up. It's meant to be a guide to new practioners, sure, but it has more akin to a manifesto than a reference. It is colloquial, anecdotal, loose, non-substantive. Simply put, it is an appeal, but it isn't convincing.

Unfortunately, one of the weakest parts of the book is Chapter 25. Chapter 25 attempts to identify places where XP doesn't belong. Any good definition is inclusive and exclusive. Certainly XPXEC stakes out the inclusive part of the definition in detail. (In grave detail too; it often goes too far. e.g. defining story cards as part of the process. p.88) But it does a really bad job of explaining the exclusive part of the definition.

Chapter 25 uses many tautologies and implicative denials in its poor attempt to explain the outer bounds of XP's scope. For instance, "It is as important not to use XP where it is bound to fail as it is important to use it where it provides real advantages," is an obvious tautology. It means nothing, of course, as it's always important to do good things and avoid bad things. Beck unfortunately doesn't explain the circumstances where XP could fail, the whole point of the chapter.

Another example, "If you have the wrong physical environment, XP can't work." Naturally, if I have the wrong anything, I can't do whatever. Beck would have done better by describing the physical conditions necessary for XP rather than making this empty statement. It is definitely an empty statement because the reader is left asking, "Ok, but what is the wrong physical environment?"

Another example. Beck writes, "I won't be telling you 'Don't use XP to build missile nosecones.' I haven't ever built missile nosecone software, so I don't know what is like. So I can't tell you that XP will work." This is where he should have stopped. But, perhaps a little eagre he continues, "But I also can't tell you that it won't work." This last sentence has an information efficiency of 0 because the previous statement clearly demonstrated that Beck doesn't know. The rhetorial purpose of this sentence was to leave the door open a little bit wider, thus weakening Beck's own position that he can't speak on this issue.

He even resorts to truisms. "Any business that runs projects by trying to point the car in the right direction is going to have a rough time with a team that insists on steering." Actually, any (business) strategy (software or otherwise) that is irresponsibly ignoring environmental factors affecting its success will fail. This is completely independent of XP.

And sarcasm. "If a customer or manager insists on a complete specification or analysis or design before they begin the small matter of programming, then there is bound to be friction..." (emphasis mine) Beck's use of sarcasm reveals his contempt for this viewpoint, suggesting that he isn't honestly assessing failure conditions for XP. Of course, XP by definition is anathema to BigDesignUpFront because it doesn't fit into its structure.

None of this is very appropriate. It's clear that Beck really didn't know where XP doesn't apply at the time of writing. He should've just admitted that without throwing breath at it. That would have been respectable. However, because Beck went on to attempt to guess where XP doesn't apply, a sufficienctly critical reader might reject XPXEC out of hand as being non-substantive. Who wants to trust their jobs to guesswork? And given the import of this book to XP, the worst outcome would be that reader reject XP assuming that XP was handwaving.

However, we do know that XP can work. It isn't handwaving, but we need to demonstrate this. In the future, when enough case history has been accumulated, I think a new introductory book should be written. XPXEC 2nd Edition perhaps. That edition can usefully exclude ExtremeProgrammingInEnemyTerritory, ExtremeProgrammingForOne, and ExtremeProgrammingMayScaleUp as well as other breaches of XP's scope (e.g. XpAndOfficePolitics) that are classical symptoms of "RunawayReligion"--more like runaway ideas.

Besides, it's a mark of an honest man, he who can admit his own failings. If XP can transparently and clearly outline how to make XP succeed and where not to use it, not only will its success rates improve (seriously), but I think its credibility in the industry will increase dramatically. -- SunirShah

SunirShah has definitely read the same book I had, yet comes away with extremely different conclusions. He is already on record here as being anti-methodology, which (it appears to me) to be blinding him to that XPXEC describes a family of practices with a common core, not a single methodology like the many others he (rightly, in my opinion) derides for their rigidity.

Based on how I understand his comment in AreYouDoingXp?, he seems to be generalizing in part from his own experiences without taking into account some of what is discussed in XPXEC, then pointing to XPXEC as the root cause for this. For instance, his dismissal of index-cards as unsuitable "quality assurance for the domains I work in" appears as part of his support for dismissal of XP for ANY domain. This dismissal is unfounded, given its clear success in creating a large payroll system. Likewise, he dismisses the observation from KentBeck and others that marketing is a best-effort substitute for OnsiteCustomer in a product environment, because "in my opinion marketing is not a good substitute". His opinions must be weighed against that of Kent here, which echoes that of general business (not just software-industry) observations: that marketing can be a reasonable substitute when continuous direct customer contact is not available.

Answering each of the points in his critique of Chapter 25 above.

I only take exception to "non-substantive" in the first paragraph, and that XPXEC is merely an unconvincing appeal. The remaining points should amply show why I disagree with Sunir's characterization. This comes back to showing that the exclusive portion of the "definition" of XP is indeed present in XPXEC, contrary to Sunir's reading. (I definitely find XPXEC convincing, especially Chapter 25.)

Much of what Sunir objects to is individual statements, taken out of context, which he claims are "obvious tautologies" and thus meaningless. [As support, he asks questions that he claims are unanswered, which are indeed answered in this chapter.] However, given the current state of practice within software development and software engineering, these "obvious tautologies" are honored as much in the breach as in the observance. I have observed each of Sunir's so-called "tautologies" violated (subtly and grossly) in dozens of organizations in my years of development experience.

For instance, "it is as important not to use <N> where it is bound to fail as it is important to use it where it provides real advantages". And, "Any business that runs projects by trying to point the car in the right direction is going to have a rough time with a team that insists on steering." These are, as Sunir points out, independent of XP. So why hold this against XP? By the way, these violations are also violations of the pure economic self-interest assumptions, because companies are much more than purely economic entities in the real world. (The difference between theory and practice is that in theory there's no difference between theory and practice.)

In addition, it is direct admission that there will be instances where using XP is detrimental to project success. Chapter 25 cites at least two (the overtime as "commitment to company", and where realistic testing is impossible). MartySchrader [below] cites a few more, based on his experience.

I have had companies commit suicide because they refused to do good things and actively embraced bad things -- imposing big specifications and rigid methodologies; ignoring customer requests and feedback; refusing to develop new products (!); taking away equipment from tech support and maintenance teams to cannibalize for executives who refused to touch keyboards; chewing out programmers in public for failures elsewhere in the organizations (and getting promoted for it); splitting up active teams into separate buildings or nations and expecting communication to improve because of the move; taking away people and space on business mission-critical projects to put onto pet projects without any justifiable business case.

Beck spends much of Chapter 25, and several places elsewhere in XPXEC, explaining several circumstances where XP is likely to (or will) fail. Page 156, paragraphs 2-3: Projects run by "pointing the car" and not allowing for small adjustments. Page 157, paragraph 2: Projects with lots and lots of people; paragraph 4: Projects where the technology prevents flattening the cost curve; paragraph 5: Projects where the link cycle takes 24 hours; paragraph 6: Projects where testing time is unavailable (precluding any Functional Testing). Page 158, paragraphs 1-3: Projects where the team cannot communicate richly, because of physical distance or other barriers such as excessive noise. Tom DeMarco's PeopleWare has several good stories of more such barriers, and how some of them have been solved.

"Okay, what is the wrong physical environment?" Perhaps Sunir is expecting too refined a definition.Beck shows that the "wrong physical environment" is one in which the team cannot overcome inefficiencies imposed on them by that physical environment. For instance, where a baby or other uncontrollable noisemaker have invaded the team space, and the team cannot be rid of it. Cube farms tend to be poor physical environments for XP, because of the lack of control: over noise-attenuation, over ventilation, over where the desks and computers can go, and other factors which make subtle creative work and communication more difficult or impossible.

Nosecone software passage. What is Sunir's objection? That Kent has said that he cannot claim that XP will not work for nosecone software. Far from having "an information efficiency of zero", this is useful new information. Having eliminated "Kent says XP will work for nosecone software so try it freely" (first statement) and "Kent says XP will not work for nosecone software so don't even bother" (second statement), we are left with -- not nothing, as Sunir claims, but -- "Kent says nothing either way, so decide for yourself". Which is indeed the last sentence of page 155: "If you write missile nosecone software, you can decide for yourself whether XP might or might not work." Sunir attempts to read in implicative denial, which Kent forestalls specifically by providing both statements. Sunir suggests an additional line ("cannot recommend") which mangles Kent's meaning here, especially if "implicative denial" gets read in.

"Small matter of programming" is an ironic phrase in the software community, playing on the (often-fulfilled) stereotype of management assuming that actually doing the work is straight-forward and easily dealt with on a schedule. The irony comes from knowing the matter of programming is anything but small. This is actually jargon now, not sarcasm. Also note the word Kent used: "friction". This is not a failure guarantee; this notes where there's a process clash, which dissipates energy better used for achieving the ultimate objectives of the project. It might well dissipate enough energy to cause the project to fail; it might not.

If Kent didn't know where XP doesn't apply, why would he write this chapter? After all, it is titled "When You Shouldn't Try XP", and goes on to talk about just that, with general rules of thumb which programmers and other professionals can easily turn into falsifiable experiments in their particular situations. This is in very remarkable contrast to several methodologies with books in current circulation, which dance around the issue or imply very strongly that the methodologies work in all circumstances.

"Who wants to trust their jobs to guesswork?" Apparently, most of the software developers I've come across. Rather, they do NOT want to trust their jobs to guesswork, but their management does (by uninformed imposition of practices), and they have to suffer for it. Again, this is a flaw independent of methodologies and practices, so dinging XP for it is flawed reasoning.

Yes, we know XP can work. Kent describes how to make it more likely to succeed, based on experience with several teams. Several teams have demonstrated repeatable success with XP. I'm seeing XP's credibility improve in what I see of the software industry, as its benefits become clearer and more well-documented and well-proven in early-adopter organizations. -- JosephBeckenbach

[duplicated]

Okay, is it not our duty to analyze the environment in which we are forced to work to determine if XP can be applied or not? How about some of this risk analysis stuff? And I'm not talking about CriticalChain or any of that blather. When I started this assignment in July of '00 I was told that we were going to be developing in an XP environment. Sure thing. All of the XP stuff except for pair programming, short iterations, customer involvement, testing, and a few other minor priciples of XP. Needless to say, of course XP is a huge failure in such an environment.

As a result of that I always take time to point out to the site boss that we don't actually do any of the XP stuff on this project. XP hasn't really failed us here because we haven't tried it. But then I point out that the book says XP is not to be applied to the development of a product which is any of the following:

 - a platform for other products
 - fulfils a critical mission
 - has a long V & V cycle

Yah. What are we developing here? A platform-based crtical drug dispensing medical product. Oy!

When I get to the next gig I will evaluate the risk involved in attempting to employ XP on the product under development. My analysis will have to include such things as resistance from corporate culture, physical availability of space, direct customer involvement, etc. This seems to me to be an AND function. If any of the environmental conditions mitigate against it XP is bound to come up short. XP is kinda fragile in that regard and needs to be treated just like an intolerant religion based on ignorance over truth. Oops, sorry -- not exactly the analogy I wanted, but the closest thing I could come up with. -- MartySchrader

I have many objections to your response, including that you are introducing your own experiences to enhance the material in the book--and it is the material of the book in question. But I don't have time to respond, nor do I really care. XP is off on its own; anything we discuss here will be wasted time.

I'm not really taking a DevilsAdvocate position. I don't particular like the way XP has progressed. Personally, I would recommend reading about it in a fair way, learning as much as you can from it (maybe even trying it once), and then move on. Adapt. Of course, I think XP is inefficient because it works with lower bounds and not averages. While this increases the probability of success, I'm fairly sure it has a lower economic expectation. -- SunirShah

I think XP is inefficient because it works with lower bounds and not averages.

I find this statement extremely intriguing, and I also have no idea what you mean by it. Can you clarify, please? -- FrancisHwang

It assumes the worst possible outcomes, so you over do it on the rigour. While this means you are always rigorous, and therefore solid, it may cost more in the long run because it may be more efficient to accept some deficiencies which are fixed later on. So, unit testing everything may be too much of a stretch. (And in practice, we often find it is too much of a stretch.) -- SunirShah

I get the sense that our positions are at different stages of acceptance of process changes. In terms of CrossingTheChasm, I think you're aiming at audiences/clients who are at least a half-phase later (me no earlier than the early Early-Adopters, you the late Early-Adopters). Which is fine. XP is at a stage in its life where it needs to be tried out more, to find out more finely where it should not be extended to. A first cut has already been made, to provide initial guidance; it peaks in particular in Chapter 25 ("When You Shouldn't Try XP"), to which you raise specific objections, and to which I make specific response.

I'm somewhat surprised you object to my using my professional experience to correlate with what had been said in XPXEC, especially after you had done exactly that (talking about index-cards being inappropriate for your domains in terms of quality assurance [I assume regarding planning]). I could go forward ignoring my experiences and what I've learned from them, but I have already found out that that's unwise. (And it certainly doesn't deliver value to my customers....)

Why do you feel XPXEC fails to make use of averages? What are Velocity and Load Factor for teams or over periods of time? These are averages, measured and used as inputs to materially improve the speed and quality of decision-making, compared to most others sets of practices.

XP, in my experience, actively works to rachet bounds in desired directions, much more quickly than most other practices, and allows the teams quicker response in making adjustments and adaptations for local use. For instance, the lower bound on how quickly a piece of functionality can get into production is the iteration, on the order of a week. Most places (that I have data for) which practice BigBangIntegration? find their lower bound is on the order of a month. Lower bounds on customer feedback have a similar half-order-of-magnitude differential, favoring XP. And the lower bound of "time on project" appears to be at least incrementally but significantly higher for XP compared to others. I agree, it will be great to have a full-sized study on this; we don't have the breadth of projects to pull this info from yet -- but the early indicators suggest (at least to the Early-Adopters) to try this out and work its key pieces in.

I have no clue in what sense you are using "economic expectation" above. If XP is more likely to produce a successful project for a given amount of effort, the value expected from the project is higher, not lower. Do you have any data to support your presumption? Beck, and Jeffries, and Cockburn, and several dozen others have data to show higher actual (not just expected) economic returns -- even NOT factoring in the value of time, which improves the situation even more dramatically in XP's favor. Their observations of why this might be, backed by experiments, some of which show up in XPXEC, mesh with my own professional expertise and experimentation, and of those with whom I talk and ask questions.

I disagree that discussion of the points will be wasted time. Others will have the same disagreements (again, based on observing software professionals in their everyday practices). I also intentionally refused to edit any of your revised words, specifically so as make sure readers understand that you think that the whole book is flawed. However, you make answerable objections only to specific statements made in Chapter 25. Which way do you want it: that I answer what you say your objections are regarding the material of XPXEC, or that I answer what I think your objections are regarding the material of XPXEC?

And when you confuse the practices with the book and with the person who started writing about it, as you do in your last paragraph of the first section, how am I supposed to know when you are talking about the practices, the book, or the person? -- JosephBeckenbach

See: GreatFailureOfXp

CategoryAdoptingXp, CategoryXpCritique