What Is Extreme Programming

What is XP?

I've read many things about XP; almost all of them tell you how to do it. Even this topic, WhatIsExtremeProgramming, which sounds so basic, didn't exist until just now.

Robert Martin has this as his signature:

  "One of the great commandments of science is:
      'Mistrust arguments from authority.'" -- Carl Sagan

So I decided to listen to Carl, and to the guru’s of XP and came up with this. (Before anyone starts on me, I know that Sagan was really discussing argumentum ad verecundiam here, but it's fun to deliberately misinterpret things when the context is removed).

So, what is XP?

Is it the collection of (excellent) practices that spurt from the fingers of Beck, Cunningham, Jeffries, and others? A process, even a lightweight process, is a set of practices from which the practitioner can choose a subset that is applicable to their particular project. The majority of modern processes encourage the practitioner to use these practices when necessary, and to drop them again if they cease to be necessary. These processes also often encourage the practitioner to add other, novel, practices if required.

So, according to the XP gurus it should be possible to do XP, but to drop, or change, any of the practices of XP. So, the practices by themselves do not constitute XP.

If the practices aren’t XP, what is?

Chet Hendrickson (in http://www.xprogramming.com/xpmag/NotXP.htm When is it not XP?) said that the four principles of XP are communication, feedback, simplicity, and courage.

Well, I’m not too sure about this. On my last project customer feedback was in high demand, but low availability. It was a $250,000,000 project, and the bit we were working on was a very small bit (about 1/100th), but we were still doing XP?

It felt like XP, it looked like XP, it even smelt like XP, but we didn’t have that feedback loop. We did have the other three principles though, and most of the practices fitted the project, so we used those (metaphore, yagni, o&oo, refactor, evolutionary architecture, spikes, unit-tests, integrate, etc).

We didn’t use extreme planning as such, but an alternative that was equally novel. We didn’t really solve the feedback problem, but the client had provided us with a large set of fairly well defined requirements, which we converted into use-cases (because stories require significant user interaction to write). We also used a variation on pair programming, because the idea of it was too outré for the management to swallow.

The project was a huge success, and the maintenance mode will be considerably less expensive because of our efforts.

Some practices didn’t fit, so we didn’t use them. One of them was user interaction with the project. This is the key to achieving the feedback principle, and as the practices are flexible then this particular principle must also be flexible. If the principle is flexible enough to disappear (as it did on our project) then it can’t be a principle (perhaps its just an emergent feature).

I believe that it was still XP. I would argue that all of Chet’s principles are really just emergent features of XP. Rather than XP existing because a project determines that these are the principles it will be built on I believe that a project is based upon XP and it exhibits these features.

So, if the practices aren’t XP, and the principles aren’t XP, what is?

To answer this question properly one has to examine the practices of XP. Rather than examine an actual project lets examine the practices themselves and how they evolved.

Software Engineering is in deep trouble. Once we called it “The Software Crisis”, but Grady Booch pointed out that a crisis is a state of change, and we have witnessed no change, so we now call it Software Engineering. We acknowledge that software engineering is in a bad way, and we try to fix it by improving the way we do it. The XP’ers noticed that our improvements generally weren’t. So, they tried doing things differently on the basis that it would be really difficult to make things worse.

They started from first principles, in an engineering manner, by stating the goals, and then trying to discover the most effective way of achieving them. For example, when investigating design they hit upon the idea that the design documents we all know and love existed to communicate the structure of the solution thought up by one person into the mind of another.

They understood the problems associated with classical design (some of which had been mitigated by the advent of Bohm’s spiral model). The design is very difficult to do; nobody really knows what level it’s necessary to go down to; the design rapidly falls out of sync with the code; the design is difficult to change. They knew too that software design was originally conceived back when code was complex. They were also aware that, in the majority of cases, developers progressed rapidly into areas that weren’t designed.

They thought, perhaps subconsciously, “these problems are insurmountable, so lets just write the whole thing off as a bad idea and progress onto something more useful”. We want a technique that is good for communicating concepts, that is amenable to change, that is sufficiently sophisticated to represent complex designs and which automatically stays in sync with the code.

They realised of course that the code was the only tool available that currently met all of these criteria. Hence, the code is the design.

This process was followed with every bit of software engineering practice, including the act of coding itself, and the code produced. No comments, pair programming, you aint gonna need it, constantly refactor; they all developed from questioning first principles, perhaps from mistrusting authorities who told them to do more design, more process, more documentation, more of everything except coding.

So, does this lead us to a better understanding of what XP is? Does it lead us to understand how a project can seem like XP even if it ignores some of the, so called, principles of XP?

Yes, it does.

XP is characterised by questioning assumptions and reasserting goals. XP is never loosing site of the ultimate goal, to produce software that provides a business benefit to ones customer.

XP is saying “Why am I doing this, what am I trying to achieve, and how does it benefit my customer?”

A project doing XP is one which constantly asks that question.

The XP’ers have provided us with an excellent set of tools which are a pattern language containing abstracted solutions to this question as it was answered for them at various points in their projects.

A quick definition: Intelligent Goal-Oriented Evolving Process – A process that evolves, as a project progresses, through the intelligent choice and development of goal-oriented practices.

An abbreviation: XP – An intelligent Goal-Oriented Evolving Process.

XP then is two things, an intelligent goal-oriented evolving process, and a pattern language for such a process.

Just as I can write my own singleton without following any of the suggestions in GOF so too can I run an XP project without following any of the suggestions of the XP gurus.

--BryanDollery

Bryan suggests that any process that is based on intelligent goal-oriented practices is XP.

But humans are intelligent, and have at least the goal to get the project done. And no one sets any practice into place without having some goal for it.

It follows from Bryan's definition that you can do anything you want to and call it XP.

No, sorry, I don't think so.

In my opinion,

XP is about balancing authority and responsibility, especially the balance between business needs and technical needs. Therefore probably you can't have an XP project without a visible way to separate business and technical responsibility. I'd be very doubtful if the answer was "we all understand the domain." I'd be very doubtful about a project that didn't have explicit acceptance tests monitored by someone other than the developers.

XP is about short cycles and rapid feedback. Therefore a process that doesn't use close-in unit testing when it's possible doesn't look XP to me.

XP relies on simplicity and communication. Therefore I'd be very doubtful that a process filled with documents and milestones and hand-offs and conventional CMM Level 5 trappings, when conversation would do, was XP. I'd wonder hard about a project with people in two offices separated by a few paces, communicating mostly by email.

One of XP's principles is courage (and not foolhardiness). A project where people live in fear that the project may not work - that it may not be working now - is probably not an XP project.

I could be wrong.

Certainly the things Bryan talks about can, in the hands of an experienced, reflective person, yield good results, perhaps better results than any canned ideas, XP or not.

But XP begins with specific practices. Later, one would modify the practices in accord with need. Bryan seems to be saying that XP is a synthesis of whatever practices you may thoughtfully choose.

I'm not at all comfortable with that, but maybe I just don't know what XP is. --RonJeffries

I was just analysing XP and its invention to see if we can shed light on the core of XP; so that we can see what can be dropped and what is required. I'm attempting to derive a formula by which the common question "am I doing XP?" can be answered.

This analysis was just the first step. It was never my intention to imply that one can do anything and call it XP, it was my intention to discuss this proposition.

Also, given my premise, and rules for developing a process, I don't see how any of the scenario's Ron discusses are possible. For example, the thing about a process filled with documents, milestones and hand-offs; given the stated premise that conventional wisdom is bad, and that things must change, the methodologist must, by definition, produce something which isn't full of documents, milestones and hand-offs. Therefore I see this suggestion as meaningless. Unfortunatly all of Ron's suggestions along this vein are similarly flawed (I think).

I think I'm saying that XP is a synthesis of whatever practices one may choose, by following the rules of XP. Those rules are

  Conventional wisdom is wrong
  Things must change
  Design a goal-oriented replacement

I think that when these rules are applied they'll produce a lightweight process that has courage, communication, feedback and simplicity as emergent behavior.

I've just read this, and think that I should share it with you all:

'Do not rely upon what you have heard proclaimed, or upon custom, or upon rumour, or upon scripture, or inference, or established principles, or clever reasoning, or favouring a pet theory. Don't be convinced by someone else's apparent intelligence, nor out of respect for a teacher.... When you yourself know what is wrong, foolish and unworthy, and what leads to harm and discontent, abandon it....And when you yourself know what is right...[and so forth] develop it.' - SiddhartthaGotama c.500 B.C.

-- BryanDollery (Ron, you know what light is, right? Do you really understand it, do you know the underlying principles of photonic propagation? Have you ever heard of a photon cloud? Do you need to? [this is where Ron tells us that he actually does have a background in atomic physics specializing in optics])

With regard to this page's query, and ExtremeProgrammingIsTheExtremePractices vs ExtremeProgrammingIsTheExtremeValues, I have been pondering the XpAsStory analogy.

So, according to the XP gurus it should be possible to do XP, but to drop, or change, any of the practices of XP. So, the practices by themselves do not constitute XP.

I would argue that the conclusion does not necessarily follow the assumption. Like most useful definitions, I believe XP is fuzzy (see FuzzyLogic) in nature and does not map to a binary yes/no checklist. One may use any of the practices all of the time, some of the time, or none of the time, though very few would fall into the end categories of all or none. Yes, the categories define XP and a project's level of adherence to the practices indicates the project's level of adherence to XP.