Large Extreme Programming

ExtremeProgramming is a methodology which tries to do only those things which really help get the product written. (The product, in many cases, is just the program. Often it's a program with a manual, with some documents, etc.)

We focus on minimizing the process because this maximizes the resources you can apply to the things the customer really wants.

As a "philosophy", I'd hope that few people would have difficulty with the idea that you should do only the necessary process to ensure that you deliver what is needed, on time, at the right quality level, within budget. It's not easy to agree on just what can be safely left out.

People who actually do XP monitor how the project is going, using a few key measures: progress on functional tests (scope growth and quality); resources; stories completed (scope and time). We measure the rate at which we produce functionality, and when the ratios change we examine what is going wrong and fix it. (This usually amounts to getting back on track with doing the process as we know how to do it. Sometimes it involves coming up with a little bit of new process.)

Since the XP meta-process is introspective and self-adjusting, we believe that we could morally undertake a project of any size while applying the philosophy. We wouldn't run the project into the ground blindly, because just like you, we don't do anything blindly, we try to keep our eyes open just like you do.

We believe that the "philosophy" of minimal methodology with self-measurement can be effectively applied to all sizes of projects. Someday we'll have more experience with different sizes - and we'll have more to say about it.

On this page, and related ones, we speculate about how XP might apply in the large. We freely state that we are speculating. --RonJeffries

I am all for minimal methodology and Listening, Coding, Testing, Refactoring, but I just can't see how ExtremeProgramming scales up to larger projects, in other words, the systems that are so large and complex that you can't simply throw them away and start from scratch when business conditions change. Systems that have to involve hundreds of software professionals to get them finished in any reasonable timeframe, where many of the software professionals are not very talented because of the short supply and huge demand these days.

ExtremeProgramming and minimal methodology do not meet the enormous communication and coordination requirements to keep 100+ people driving down the same road. And they certainly don't provide the concepts of good documentation and training that are vital to maintaining and properly using a large, distributed software product. Of course, neither do the mainstream methodologies of the day, which introduce road blocks and arbitrary productivity killers in the sake of implementing a "best practice" or 10. So, how do we scale the real best practices to large projects? Or maybe we just don't do large projects? -- ToddLauinger

Inside every HundredPersonProject, is a 10 person project struggling to get out. RonJeffries and I just did a back of the envelope calculation of the LoadFactor of a 60+ person project we are familiar with. I worked out to about 1000. Lets say our calculation was an order of magnitude off, they still have a 6 person project with 54 people in their way. -- ChetHendrickson

WardCunningham once speculated with me about having four or five ExtremeProgramming teams working together on "the same" project. I would have to be able to apply DivideAndConquer in some reasonable way. I would start small and let the division take place as it wants to, both the functionality of the system and the team. I think with a group like this you could tackle about any project I know about, which doesn't include phone switches. -- KentBeck

It's the edges, always the edges. Could we do PairProgramming across the edges, an engineer from the (e.g.) GUI team and one from the model team, etc? Perhaps we'd have code ownership by team, instead of none at all? Could we still do ContinuousIntegration? How much paperwork could we still not do? Above all, could it still be fun? -- RonJeffries (Can't remove this w/o loss of continuity, sorry.)

GUI and model teams would be a recipe for disaster. Each team would have to be responsible to a different set of clients and work against a common set of interfaces, file formats, shared database, or socket protocol. Each interface definition has strengths and weaknesses for this kind of DivideAndConquer.

One way of combatting this might be IntegrationTeams?, which would form the internal nodes of a scaled XP project, with regular XP teams forming the leaf nodes. One integration team could play customer to 3-5 XP teams, say. And integration teams might play customers to other integration teams in a tree hierarchy. The practices might be different from those of a traditional XP team. For example, testing would play a very large part of the integration team's responsibilities. They would generate AcceptanceTests for their sub-teams and UnitTests (of sorts) for their own integration efforts. I've never worked on a project bigger than 4 programmers, so all of this is naive speculation, but perhaps others could elaborate with their own ideas? -- RobHarwood

If you had a problem that really couldn't be tackled with less resources, and the client was willing to pay 50 or 100 times as much for 5 teams as they did for one, yeah, sure, it could be fun. There would be more constraints, which would be less fun, but there could be more impact, which would be more fun. -- KentBeck

An often overlooked but still key part of XP is the revised relationship with management. By the time you have teams of XP teams you will also have invented ExtremeManagement (XM) which will have its own versions of familiar extreme principles. I set out once to construct exactly this (WardAtIbm) but found personality types and reward structures at both company and clients, well, unfavorable. Still, XM will happen in time. It will happen first among a small group of managers who are delivery oriented, have the confidence and knowhow to work from first principles, and find themselves working with teams that are already practicing XP. And it will be glorious. -- WardCunningham

ExtremeProgramming advocates refactoring. Part of refactoring is that if a method gets too long, you break it apart. Ideally each part implements some discrete functionality.

How does refactoring map to organization for large projects?

I have a large (over say 30 people) team working on a very large project. I refactor the project into 6 smaller projects and assign 5 people to each. It seems to match the pattern.

There is some aspect of inertia in organizations that seems to cause problems, though. The interfaces between the 6 smaller projects become static or at least less changeable. This seems to break ExtremeProgramming.

How do you provide organizational volatility that matches the software volatility? Or is it even necessary? --WayneCarson

If people can pair across teams, then you could "refactor people from one team to another" with ease. It would also dramatically increase the bandwidth inter-team interfaces, and make changes to inter-team interfaces easier. These would be good things. -- JeffGrigg

I am on record as separating the issue of "large teams" from "large problems". Large problems can sometimes be handled by small teams. XP addresses this matter - make everyone much more productive, and your small team can solve a larger problem than you thought possible.

XP doesn't address large teams - those are necessarily communication-burdened.

XP answers two questions:

How can I get 6-10 people to work as fluidly as 2?
How can I get 6-10 people to deliver what I'd have thought would take 25?

-- AlistairCockburn

Could the software for the space shuttle have been developed with XP? It has over 25 million lines of code and cost about 1.2 billion dollars to develop.

The XP folks have said over and over again that XP is not a universally applicable methodology. Systems with draconian safety constraints are almost invariably the first category mentioned as NOT being XP candidates. That being said, there's no reason why some XP practices couldn't help improve the productivity for ANY software project. UnitTests, FunctionalTests, PairProgramming, DoTheSimplestThingThatCouldPossiblyWork and RefactorMercilessly should be almost universally applicable. -- DonaldMcLean

Maybe you could do XP in large by dividing the system into layers of subsystems and assigning separate XP teams to each subsystem and each layer so that the XP teams of the higher layers are "customers" of the subsystems that the layer needs. Thus the XP team for each layer is responsible for applying XP principles at that level and so the whole system will be developed using XP... What do you think about that?

As seen on the XpMailingList:

What's the risk of having (too) fine-grained teams from the start?

Too slow evolution of interfaces
Too large chance of hidden duplication
Too little team feeling
Too little unity of vision
Too much dispersion of customer attention

-- KentBeck

In a small battle, with swords and spears and shields, the rule for a soldier in a squad is you keep peripheral eye contact with other squad members. Don't let the enemy draw you away from them with a feint, or let them best your comrades when you don't notice and isolate you.

In a large battle, each squad follows the same rules regarding the squads to its left and right. They all form a unified front.

-- PhlIp

On a large project developing a new product for eventual sale through a channel, WhoIsTheCustomer?

Inside every HundredPersonProject, is a 10 person project struggling to get out.

I think this may summarize one of the key areas of disagreement. XP seems to take the approach that a software project is infinitely divisible. A large project can be divided down to be handled by a single team. This reduced project can then be further subdivided to have useful deliveries in 3 week increments.

I am not sure how to bridge this gap. It appears you either accept the infinitely divisible concept or not.

-- WayneMack

It seems to me that the problem of ScalingExtremeProgramming is the same as the problem of scaling any methodology. The unit of work is a person and the problem of scaling is a communications problem not unlike the advocacy of high cohesion and weak coupling as a mandate for good programming, it is also a mandate for good partitioning of any large project. The key to scaling is finding a way to DivideAndConquer such that communications requirements are minimized. Then in each of the resulting weakly coupled cells one can run that cell on XP. So XP will not scale by itself, it has to be combined with some kind of appropriate partitioning methodology that finds the right clumps of cohesion and defines interfaces which are weakly coupled in the right fashion.

That is a whole lot easier to say than to do. Many integration failures are due to the fact that people thought they understood but really didn't so when it comes time for integration things don't work. This can happen at any scale, but at small scale it's only a small annoyance while at larger scale it kills the program. Hence the evolutionary emphasis of some.

I am fond of saying that we need the software equivalent of a wire. This is not because a wire is a particularly good interface, but because it imposes acknowledged and universally understood constraints. UNIX streams, pipes etc. provided a little bit of that flavor and are a large part of the utility of UNIX. A set of simple, standardized, easily understood "wires" would do a lot for providing inherent scalability. Software is entirely too free and unconstrained to make that achievement easy or likely.

-- RaySchneider

My colleagues and I have been doing PairCoaching and have found that it works extremely well. -- JoshuaKerievsky

Most of the proposals I've seen for scaling up XP go something like this: "With more people on the team, whole-team collective code ownership won't work anymore, so we need to reintroduce code ownership (in the form of sub-teams owning code)." Usually, this is followed by the original part of the proposal: how exactly to split up the team and carve up the code by owners.

This is all fine and good, and may even work. But I think there's another, more extreme way to scale up.

Let's review individual code ownership and why we don't like it for small XP teams:

Bob knows everything about the code he owns; the rest of the team knows nothing about it. Bob becomes a scarce resource, a bottleneck, and the one to get hit by the proverbial TruckNumber truck.
Bob's code has an interface somewhere; Alice's code uses that interface. Bob can't improve the interface without waiting for Alice to change her client code; Alice can't improve Bob's interface without waiting for Bob.

As Ron says above, "It's the edges, always the edges." If you have a chunk of code owned by entity A (whether an individual or a team), it's got an edge somewhere -- an interface, a line with A's code on one side, and B's code on the other. Crossing that edge is a pain in the ass; writing the code at that edge is a pain in the ass; improving the code at that edge is a pain in the ass.

Why would you want to enshrine that edge in the structure of your people?

Instead, you'd want to intentionally blur any edges you come across. Rather than one interface always marking the boundary between two different teams, you'd want the teams to ebb and flow over that interface throughout the life of the project.

Here's a redescription of traditional collective code ownership that will hopefully make some things clear: When Alice is working on a feature, she touches a bunch of classes. In some sense, she and she alone "owns" those classes while she's working on them: she changes them any way she needs to, and changes their tests however she needs to, too. If Alice changes some important interface in class X, she also makes the necessary changes in X's clients (classes Y and Z). So, however fleetingly, Alice owns the interface -- everything on both sides of it -- and can change the interface however she needs to. By the time the day is done, of course, she surrenders "ownership" of everything to the rest of the team.

Note that we can't avoid having "ownership" in this weak sense: somebody has to be making changes to piece Q of the code, and at some level, however fine, nobody else should be able to simultaneously change Q. But we don't try to make this weak "ownership" persist over time: Alice has no special formal relationship to the X, Y, and Z code once she's integrated it back into the main line. Informally, she may be the expert on XYZ, and it might be a good idea to pair with her if you come along later to change it, but it's not mandatory, and probably not necessary.

So you can view collective code ownership as just traditional individual code ownership, with extremely short durations of ownership.

Back to large scales: due to coordination, communication, synchronization, and whatever other costs, say we can't have all 100 programmers plunking away at all 10,000 classes all the time. So 10 teams of 10 programmers each grab, for the duration of the iteration, the classes they need to implement their features. Maybe Team A "owning" class A for the iteration makes cruft pile up in client class B, which Team B grabbed for the iteration. No sweat: either there's time after Team A finishes with class A in this iteration, and can transfer it to Team B, or refactoring the cruft out becomes a task for the next iteration -- one which will require whoever takes it on to grab both class A and class B.

So team collective code ownership is just traditional team code ownership, with relatively short durations of ownership (one iteration or so).

There are probably many other ways to scale up collective code ownership, but this one seems pretty straightforward, and pretty extreme. Smooth the edges.

-- GeorgePaci

I'm interested in this idea, but it does not match up with my experience of even rather closely integrated teams.

I have never worked with teams which share exactly the same tools for coding, building and testing. Teams and products, often from different countries, merge, split and reform. Even in the highly standardized area of Network Management there is enough diversity in tools and platforms that some kind of competence synchronization across teams is all but impossible.

What are other peoples experiences? Is is usual that teams are so well integrated that they can start to meddle with one anothers classes directly? And without a lengthy discussion around such things as development tools, servers, test beds and configuration management? -- ChrisSteinbach

Perhaps there are lessons to be learned from the OpenSource community on this, especially regarding tools to be used across teams. -- EelcoRommes

CategoryExtremeProgramming