Extreme Programming Research

Sounds like an oxymoron, doesn't it?

However, I contend that careful academic research could prove out some of Extreme Programming's tenets through statistics. I began thinking so when I saw the article GrowingSystemsInEmergentOrganizations. -- KyleBrown

I very much agree. See http://www.xprogramming.com/xpmag/proposed_experiments.htm. -- RonJeffries

Please note that GrowingSystemsInEmergentOrganizations seems to take an anti-XP approach. Most obvious in this is their feeling that user-satisfaction is a rat-hole in information system design since the user often does not know what they want. XP on the other hand centers everything on user-satisfaction. I don't know that I agree, but they had some interesting arguments for their case. -- RobertDiFalco

Important point. I agree that the users of information systems often do not know what they want. Until they see it. Or, more accurately, something that kinda reminds them what they wanted all along. Defining and delivering this intermediate delivery that "switches on the lights" cannot in my view be reduced to science but takes considerable analytical, social and indeed psychological skills. I'm all in favor of research on what can be verified in XP. But at the same time we have to take care not to constrain real world customers to fit our favorite experiment. -- RichardDrake

Here are some more proposed topics:

Field Observation

Project measurement - defect rates, productivity, profitability, cost of change
SoftwareMetrics - measure the code produced by extreme teams according to conventional measures of 'goodness'
Measuring the cost of change. The idea that the cost of change does not go up exponentially as software ages is fundamental to XP. Could this be measured?

Business

Management economics - are there numerical justifications for XP? (PairProgrammingCostsBenefits)
Extreme business strategy - are there new businesses or new ways of doing old businesses that benefit dramatically from having an extreme team develop the software?

Tools

Refactoring tools for Java, C++, relational databases, object database, concurrent systems (preserving liveness and safety over refactorings) (See CategoryRefactoringBrowser)
CM tools for fast builds of big systems and distributed builds (preserving the good features of sequential ContinuousIntegration)

Sociology

Team dynamics created by floating pairs
Connection to 5th Discipline
I'd like a folklorist to look at storytelling in extreme teams
Anti-Taylorism

Philosophy

Connection between XP and various schools of post-modern philosophy- XpAndPostModernism
Metaphor and reality construction

Complex systems

The extreme team as a self-organizing system
Evolutionary design of complicated algorithms and systems (could you do Linpack extreme?)

The academic research has started. I expect somewhere between 50 and 100 publications and presentations in 2000 tackling various small parts of XP. For example: Laurie Williams has data on the impact of PairProgramming on error rates and attitudes in a software engineering course

http://collaboration.csc.ncsu.edu/laurie/

And then between 200 and 600 papers in 2001. The growth phase is usually exponential as a new idea comes in.

-- DickBotting

Is anybody interested in master thesis on ExtremeProgramming? The topic would be something like introducing XP in a large organization, transition from old processes, fixed price contracts etc. A key area would be the "soft facts", social and organizational issues. I'm open to discuss specific interests. The thesis research could either be done in Stuttgart, Germany, or in Auburn Hills, Michigan (Ann Arbor maybe?). If you are not studying there consider an exchange program ;-) It's always worth it. I will try my best to help with an exchange program. We have good relationships with a number of universities and are always interested in extending them. Maybe any researchers are interested in a cooperation with DaimlerChrysler? -- FrankGerhardt

I think proving (or disproving) various aspects of XP in an academic setting is great. How else can you get a statistically-signficant sample size to all perform the same task? The experiment I did on pair-programming could be done and respected as an academic study because it looked at the interaction of two programmers. (However, I would *love* to try to validate my findings in industry . . . let me know if I can work with you on this!) Research to study the affect of "unit test first" would be appropriate in academia also. But, I think studies to compare XP design practices vs "traditional" design practices really can't be done in academia. Many disregard academic studies that try to deal issues of complexity or scale found "in the real world". I think the best we could do in academia is try to find a school with a year-long project class and make some comparisions . . . I'd be interested in how conclusive *you* would think this kind of study would be. How conclusive to you think XP-skeptics would find it? -- LaurieWilliams

Most software engineering studies are tainted or flawed or invalid. Usually, the observer is directly involved with the subject, usually pushing the subjects along some political agenda. Other common failures are selecting a biased sample and misreporting and fudging data. Computer scientists aren't trained in sociology/anthropology, you know. Be careful. Especially if you're already an XP fanatic. Then your findings will be categorically rejected as more methodology marketing.

Most software engineering studies are tainted or flawed or invalid

ThreeExamples please.

I've come to these pages as a dedicated skeptic. What I have found is a description of what ExtremeProgramming is, but not very much about why adherents believe it's not only a good method for developing software, but also better than alternatives. To answer LaurieWilliams question, the kind of study I'd find convincing is actually really very simple to describe but maybe not so simple to implement.

First, find several groups who are willing to experiment and, ideally, who have some kind of identifiable method already in place. From historical data they've got, one could derive some measure of their effectiveness (quality, time to completion, whatever measure seems appropriate). Now choose some teams randomly and change them to XP. Choose other teams randomly and change them to something else. Leave the third group as they are. Come back in a year and reanalyze the data to see what effect the changes have had and see if there is a correlation which shows that transitioning from methodology A, B, or C to XP is superior to the alternatives. I would find such a study interesting, and probably very convincing. -- EdBeroset

But one of the tenets of XP is that you can't change a team, the team can only change itself. If this is true, how would you design the experiment?

The same way. Remember that the first step is "find several groups who are willing to experiment." All of the groups would change themselves. -- EdBeroset

CategoryResearch