Xp Ina Strange Land

Does XP work when the problem domain isn't well understood?

Some people certainly think so. But there are reasons to think otherwise:

Bulleted italics and [bracketed phrases] in this section begun by RonJeffries. Please interleave further, perhaps we can converge by iteration. This is helping me, if no one else. What is here is all explicit somewhere in the XP pantheon, but it seems to be hanging together nicely here. I'm starting to like it. I hope that doesn't mean Peter is hating it.

XP emphasises a clean division between customers and developers. The customers know the business, so they can deliver concrete UserStories without too much thought. But in a poorly understood problem domain, customers don't know the business and can't give you concrete UserStories.
If in fact no one has a clue what is wanted, you're not doing software development, you're doing something else. Some principles of XP might apply, but if you're not trying to build something for someone, you're outside the scope of XP.
XP emphasizes WorstThingsFirst. How can you tell what's "worst" when you don't know much about the problem domain? Perhaps there's a process for exploring a new problem domain via XP ... but that process is something of a mystery.
The not-so-mysterious process is for the team to decide (guess, as Alistair points out) what's difficult in the domain. Select an experiment. Do it, learn from it, rinse, repeat. Each time you experiment, the team increases its ability to guess well.
XP [, it seems,] rushes to act. Learn by doing. Don't think, experiment.
However, the impression that XP "rushes to act" is a mistaken reading. It's not doing without thinking. It's about learning, not blindly going ahead. XP *explicitly* calls for team judgment to be applied to determining worst things.
But plainly, many problem domains contain cul de sacs, dead ends and local maxima. In a poorly understood problem domain XP seems more likely to get stuck in one of these than something that does a fair amount of up-front research and analysis.
Again, this seems to be based on a misreading. What is WorstThingsFirst but up-front research and analysis? What other up-front process is proposed that improves over picking the most troublesome areas as best one can and trying to crack them?
In such troublesome problem domains, since the WorstThingsFirst discpline is about ignorance reduction, investment in any given local maximum is quite limited. You assess as best you can where risk lies; probe; reduce ignorance quickly; reassess. WorstThingsFirst isn't about fully exploring a local area: it's about roughly mapping the entire space, informed by what you have already learned.
XP projects begin ...
by sitting teamed with "customers" developing stories about benefit. These stories are intended to be things that could actually be done. Then high risks are identified and reduced by experimentation. Then the system building begins.
[With risks reduced, XP projects then proceed] with an architectural spike, a skeleton to which stories will be iteratively added. If problem domain is [still] not well understood, then this skeleton will be inappropriate and inefficient, resulting in gross, likely interminable, and possibly project-fatal, refactoring.
For refactoring to be interminable, there would have to be no learning. By selecting the areas where learning is most needed, and then doing the learning, you reduce the chance of massive rework. By keeping the entire system well-factored, you reduce the cost of rework that does occur. If the domain is actively trying to confuse you, or if the team lacks the ability to correctly identify risk in the domain, you'll do less well
With stories written, risk assessed and reduced, and experiments done, the domain is no longer that poorly understood. And the skeleton is just a spike. A danger of any project is that initial work will be wasted. XP directs that the entire system be refactorable, and refactored as needed. It directs that unknowns be reduced until the principals are satisfied with the reduction, before going ahead. But XP isn't perfect. What would be better than taking your best shot at risk, reducing it, then going ahead with clear tests and repeated reports of progress against business value?

So maybe XP isn't the best way to go about exploring a "Strange Land". --PeterMerel

Maybe it isn't. Please describe what would be better. Don't feel it necessary to go as relentlessly far as certain XP people have ... ;->

If nobody has ever done something before, the first people to do it are doing research. Most research projects don't accomplish what they set out. That is why research is risky. (And exciting.) --RalphJohnson

The question is whether XP is a good way to go about such research. I'd certainly be very interested in hearing how you would adapt XP to overcome the 4 objections above. --PeterMerel

XP is not about research, it is about building software. XP tries to solve the problem that the programmers do not know the problem domain and the users don't know how to design software nor how to explain the problem domain to programmers. Suppose that I want to invent a new kind of file system. Perhaps I assume that large, cheap main memory will mean that we can cache most of the active part of the disk, and that writes will become more common than reads. So, it will be faster to treat a file system like a tape, and to write changes at the end in a sequence, i.e. to make a log of changes. Naturally, I do some statistical analysis based on the file systems I've benchmarked in the past. It looks good. Now it is time to build a prototype.

At this point, I could use XP. We'd write user stories (which are just the usual file system user stories, except that we expect the frequency of them to be different and so performance tradeoffs are different), and implement them one at a time. Open a file. Read a file that is on disk. Read a file that has been cached (i.e. read the same file thirty times in the same hour). Write a file. Create a file. Delete a file. Rearrange the disk. Reboot the system. Initialize a new disk.

XP will help us make software that is easy to change. This is good because when I actually try out my file system, I'll probably find out that I made some problems in the specifications. This is a lot like what happens in a payroll system when there is a new union contract. Moreover, XP (supposedly) helps me build the software quickly and inexpensively, so I will have time and money to try out other ideas.

You might object that file systems are a pretty well understood domain. Suppose I wanted to build an air traffic control system using virtual reality. (I've got a bunch of ideas about how to do this). Air traffic control has been around for some time, and there are people who know how to do this. VR is pretty cutting edge, though, and nobody knows whether the combination will work. So, I'd get together with some air traffic control experts, build some mockups (without using any computers at all, just actors!), argue my case, and hopefully get a good plan. Once I had an idea what I wanted to build, I'd get a team of developers and do it. What is wrong with them using XP? Sure, maybe I am a crackpot and my ideas will never work, but that is not the fault of the programmers. As long as they build software quickly for me, and as long as they can keep changing the software as fast as I change my mind, I don't care what programming method they use. Why won't XP work? -RalphJohnson

This page is correct. If your domain causes you this much fear, you should not use ExtremeProgramming. --RonJeffries

This page is entirely wrong. The only way to get past your fear is to face it. You should use ExtremeProgramming. --RonJeffries

Grin, but actually...

You should use ExtremeProgramming ONLY if your domain causes you fear. A key risk that XP mitigates is that developers are not good at prediction (cf. YAGNI). This will be very true when the domain is new to the developers, probably not when the domain is familiar. -- DaveCleal

So when there's no fear, we should use what?

You should do more design up front, because your experience of the domain will shift the balance of the YAGNI forces in that direction (ie the probability of you guessing correctly will increase).

Building from ExtremeProgrammingBoundaryConditions, here are what I feel are the boundary conditions:

number of people and their geographic dispersion (based on communication and alignment difficulties),
ability to change interfaces freely (affected by number of people and implementation technology),
ability to alter code and retest rapidly (affected by implementation environment), and
ability to create regression tests (affected by willingness of the people, and the technology and problem at hand).

As far as I can tell, these are the only boundary conditions. Knowledge of the problem domain does not affect the matter, as far as I can tell. --AlistairCockburn

Alistair, while I have no qualm with ExtremeProgrammingBoundaryConditions, I'm not certain how this addresses the objections raised here. --PeterMerel

Peter, what it means is that I disagree with your analysis, above. I am a die-hard fan of feedback informing both the proces and the design. As XP provides rapid feedback, I expect it to deal well with the mid-course surprises that a new domain will offer. XP really is extreme to me - it seems to kill all the sacred cows at one swoop. The sacred cow of mine it kills is the "thinking" one - I like to think and like to think that thinking is useful. However, when I try XP on an interesting project, I'll get to find out more about that. In the meantime, it seems to have so many feedback mechanisms that I don't think a project will get too far off track.

I said I disagree with your analysis. Here's my reading:

XP emphasises a clean division between customers and developers... The customers deliver concrete UserStories without too much thought. I don't get to these assertions from my reading of XP. The customers and developers work together. That is the opposite of a clean division, it is the closest thing to a mind-meld I could ask for. I don't know where you get the bit about delivering concrete stories without much thought, I don't recall getting that impression. I would say that, in a project where the domain is not understood, I would want a user or domain person sitting right with me, so we could make it up as we go along - with the user or domain person making the business calls, and me making the design calls. Good teamwork. I call it HolisticDiversity in my book, others have other names for it.
On WorstThingsFirst. I dislike the tautological argumentation that has shown up on wiki assuring success by doing WorstThingsFirst, and particularly that in competition or confluence with BusinessCaseFirst?. However, I agree with the idea - you stake out what you believe will be the worst thing to hit you, and you address that (known as risk reduction, in the literature). What is worst, and how that competes for resources with what business story is needed first, that is a matter for deliberation and resolution. It is called guessing.Given that we know we are guessing, I would want feedback as soon as possible. That means doing a spike and a quick series of increments, both XP practices. In fact spike and short increments are exactly the process for exploring a new problem domain, in XP or any other process I would stand with.
XP rushes to act. Learn by doing. Don't think, experiment. I, personally, like to think. In a new domain, I would include library research, interviewing experts, using my imagination - and then experimenting like a madman (or scientist, take your pick). As I said, thinking is my scared cow, the one that XP kills. However, given the spikes and short increments, I think they'll find the dead ends faster than most project teams will.
skeleton will be inappropriate and inefficient, resulting in project-fatal, refactoring. I can't tell. At least three XP projects say not. However, the refactoring issue gets back to the boundary conditions I laid out. If the team can refactor and test, they can be excused an astonishing number of other mistakes. Hence, I feel that lack of knowledge of the domain does not put an XP team further behind other teams. Hence I said that having an unknown domain should not be a failure situation for XP.

Sorry for the long comments. I felt your sentences merited a fairly full reply. --AlistairCockburn

You are right on on how you select worst: you guess. I would suppose that if you think your estimates of risk are likely to be poor, you'd just rank more things high, and do more spikes. Perfect? No. But what would be better? --RonJeffries