Why They Stopped Xp

As a brand new process model, XP has some champions, and a lot of detractors. While it is getting more and more attention, controverse and speculation based arguments prosper on the net (and in some Quality Assurance Depts).

There's always something to learn from a failure. Maybe it would be interesting to read about reasons why projects using Xp as a process model failed or have been cancelled, or just changed for another approach.

(Maybe this should be merged into WeTriedXpAndItFailed or WeTriedXpAndWeFailed, but these don't properly convey the meaning : we tried Xp but some of the parties involved did stop it).

---

Here is why and how some XP practices have been hindered, then stopped in a recent attempt to put a project back on its track.

Context :

The project was a migration of a DOS Clipper application into Delphi on Windows. It was an outsourced project, with some difficulties :

• high level of obfuscation of the legacy code
• presence of an intricate, spreadsheet-like, table based subsystem with a lot of obscure spaghetti routines working with some overused global variables
• the customer wanted an exact replication of the DOS UI, under Windows
• very poor functionnal documentation and asking the customer to participate to a specification stage was out of the question.

First approach (9 months) :

1 project manager, 3 developers

• general business domain related specification, elaborated mainly from analysis of the legacy application.
• application detailed specification : the project manager, beeing told he would have only beginners for that project, directly jumped into writing the whole pseudocode of the new application, failing to deliver functionnal specification and to clearly expose business rules.
• construction, from the pseudocode, with very low quality standards and zero unit tests.
• first delivery, leading to 480 bug reports from the customer.
• 2 months spent in vain trying to debug this 50 KLOC thing. One project manager is added to the team in order to produce project reporting, the first p.m. being too much involved in technical coaching and debug.

Second approach (4 months) :

Initially : the added project manager, the coach, 2 programmers, 1 tester, then a lot of people more.

A global refactoring is studied, proposed to the customer and adopted. The goal is to fix the bugs while completing missing or vague feature specifications. The approach include some XP practices :

• OnSiteCustomer
• ContinuousTesting
• evolution towards TestFirstDesign
• MercilessRefactoring (and progressively migrating to Object)
• AcceptanceTests written by the customer
• FrequentReleases
• ContinuousIntegration
• PairProgramming
• CollectiveCodeOwnership
• progressive plan refinment
• FortyHourWeek

3 first weeks : Team in a state of grace. Productivity increases, so does quality. After 3 weeks of work, the first refactored module wipe off 70 bugs. The customer feel rather confident and willing to continue this way.

1st crisis : Project manager asks the team to stop PairProgramming, arguing that we are deliberately halving our productivity by 2; after some days of SoloProgramming and a flawed integration, the team spontaneously comes back to PairProgramming.

2nd crisis (month 3) : the project manager's manager forces everyone back to SoloProgramming "or else..". The "agile" coach is ordered to not take any more decision of any kind at the process level, and to stay at a technical level. The project manager is asked to produce a full detailed plan of the next release. Although being refuted by the team, the plan is exposed to the customer.

3rd crisis : the developers, quite demotivated, ask the coach if he could take back the control of the process. The coach sends to his management (and Q.A dept) a long memo where he explains how and why an agile approach has been taken on the project and why it should continue. The project manager approves the memo, and prepares another plan based on an more accurate measure of the team performance during the past two months. But people are added to the project, while the team is asked to parallel the refactoring (from 1 module to 3 modules at the same time).

4th crisis : after keeping silent for 3 weeks, the management pauses the project in order to make an internal quality audit. The auditor make separate interviews with each member of the team. The questions are

1. why pair programming ?
2. why are you testing at such a fine level ?
3. why refactoring the code ?

The auditor collects grosso modo the same answers from each team members

1. because we are more productive this way
2. because if we stop doing it, integration turn into nightmare
3. because improving the internal level just delete many bugs, relieving us from fixing them

nonetheless, the auditor concludes : pair programming has caused much waste of time; the granularity of the unit testing is imposing too much modularity on the code; the initial code delivered 4 months ago had a sufficient quality level. The refactoring is stopped. The management tries to convince the customer that the coach should be ejected from the project (the customer demands to have the coach always present in the team).

The coach, seeing that he's bound to be involved in a death march, and exhausted from having to struggle against his hierarchy, resigns. Although his resigning notice could last 3 months, he is asked to leave the project immediately.

Epilogue

2 months later, the project, which has been given another project manager, plus 2 developers, seems to be completely stuck, (and not a cent of euro will be paid for it).

WhyTheyStoppedXp ? Although I clearly explained in a written doc where I would go with the team when starting the refactoring stage, my management, stressed with the financial loss of the project didn't pay too much attention to the details. Then when all the rather spectacular changes in the process occurred, they immediately tried to stop it. My mistake was probably not insisting enough to get their buy-in. "We can try this way, BUT you should be prepared for some counter intuitive results and also take patience...". I've learned my lesson : don't change any process (even if you're asked to) without a full exploration and a full commitment of all parties involved, including QA and upper management level.

Maybe the second and important reason is a global lack of project measurement, so they couldn't compare the (measured) productivity of the refactoring process to anything relevant in the previous stage of the project (except maybe numbers of bug reports).

From the customer's point of view, the refactoring stage was promising. They mandated an external audit in order to prove the low quality level of the code that was to be refactored. I was told they wanted to get rid of my ex-company on the project, but this could take time.

I choose to adopt some XP practices in that project refactoring for the following reasons :

• most of XP views on the development process were consistent with my experience doing other quality project refactorings.
• business domain specification ofthe application were not documented; any attempt in a more classical process would have required a complete documentation of the features before going to the code and the test; I thought that XP would help us on both "quick" specification and unit testing.
• PairProgramming and CollectiveCodeOwnership would obviously facilitate knowledge transit : from the coach to the developers (SoftwareQuality?, ObjectOrientation) and vice versa (legacy features and business rules).

--ChristopheThibaut

---

Project manager asks the team to stop PairProgramming, arguing that we are deliberately halving our productivity by 2

This (the first crisis) was a very important sign. The project manager was being pressured (either explicitly or implicitly) by upper management. XP was dead from the get go. No amount of reasoning, auditing, or proof could dissuade the PM from pushing against XP. An audit amounts merely to a formal pointing of fingers. Nobody likes to get fingers pointing at them. Without upper management support, well ... you know the story.