Xp Critique Discussion

Comments on critique of XP from a systems perspective

XP leaves us with some of the problems identified and dealt with in the earliest methodologies. Systems analysis was already a part of AnalysingTheProblemDomain in the SystemsDevelopmentLifeCycle. Perhaps all methodologies to date can be criticized for concentrating too much on either the systems side or the software side. This brings me to the question: can (and should) approaches such as SoftSystemsMethodology be reconciled with XP? -- ChrisSteinbach

Comments following critique of large scale XP

Right: ExtremeProgramming is designed for small teams. People talk about using it for larger teams, but I haven't heard any success stories yet. See HundredPersonProject.

However, a large program does not always require a large team. One argument in favor of ExtremeProgramming is that the team becomes so much more productive that it can do what would otherwise require a much larger team. Also, a good design can reduce the size of a system by an order of magnitude, and the claim is that ExtremeProgramming will lead to better designs. ChryslerComprehensiveCompensation comprises almost 2,000 classes, almost 25,000 methods, and over 140,000 statements.

Refactoring has always been important for Smalltalk, with or without ExtremeProgramming. If your requirements change (and unless your customer is dead they do), or if you learn something new about software (and unless you are dead you will), your design is will be flawed. The question is just whether it is worthwhile to change your design. If changing your design is expensive then you tend to stick with it until you are forced to change. If changing your design is cheap then you will improve it much earlier. One of the big advantages of Smalltalk is that it lowers the cost of improving your design, so you do it sooner, so your system's design tends to be better. Smalltalk developers who do not refactor (and I admit there are a lot of them) lose this important benefit of Smalltalk. I would not hire a Smalltalk developer who would not refactor.

CRC cards are not supposed to be a complete design. The design is in the heads of the team and in the code. The purpose of CRC cards is to get everybody's heads in synch. CRC cards in fact do a great job of explaining the interactions between objects. They don't capture the interactions; they don't capture anything. A pile of cards on a table is useless. However, watching a CRC card session and interacting with an expert using CRC cards is a great way to learn the dynamic interaction between the objects. Then you can go read the code and it will all make sense.

This only works where everybody can talk to an expert. Like other ExtremeProgramming techniques, this works best when everybody is in the same room. Jim Coplien, after studying lots of development projects, says that the only way to have a high-productive project is to keep the team small. In "A Generative Development-Process Pattern Language" he says that a suitably selected and nurtured small team can develop a 1,500 KSLOC project in thirty-one months, a 200 KSLOC project in fifteen months, or a 60KSLOC project in eight months. (Note that he is assuming team size is a constant, so code production goes up with time as the groups learns to work better together.) Assuming this is true, and that ExtremeProgramming is a suitable way to nurture a small team, you can develop large systems this way.

The phrase "large programs must be designed" is clearly wrong. Lots of large programs were not designed in the sense you claim. Maybe he means "large programs should be designed". ExtremeProgramming would not argue with this, the question is whether the design must occur before the code is written or whether the design and code evolve together. As Grady Booch (and probably lots of other people) says, "Every large system that works started as a small system that works". Or was that FredBrooks? (FredBrooks also had the corollary that big systems that don't work are often made from small systems that don't work.)

: ''Inside a large, cumbersome system, there lies a lean, efficient system begging to be let out...."

Nobody says you can understand a large system with one piece of paper. There are lots of small systems that take more pages than that.

-- RalphJohnson

Nicely put, Ralph. Makes me glad I waited. You da man. -- RonJeffries

Comments moved to WhatProblemDoesXpSolve

There's no need to keep large pieces of design in your head. The part you're working on only affects a few classes at a time. See also TheSourceCodeIsTheDesign. With a well-designed, well-factored system, with browsing capabilities like Smalltalk's, we've found that you can rediscover what you need to know more quickly from the system than you can from the documentation. -- RonJeffries

How large is large? Is VisualWorks large? If so, I know lots of people who know its design. At least, they are able to quickly answer nearly any question about the system by reading source code. If you want something bigger, look at FieldStudyOfTheSoftwareDesignProcessForLargeSystems. It claims that many of the successful systems they studied had a single person who could answer questions about it and who acted as the hub of the developers. This is definitely not ExtremeProgramming, but it answers the questions about whether people can keep large pieces of design in their heads. The answer is that they certainly can, and a large project will not succeed unless there is somebody on it who does. This person is usually called "the architect".

-- RalphJohnson

I wonder how well this correlates with the LanguageLawyer role. Some people go out of their way to absorb the minutiae of systems so that larger patterns can all jell from all the details. Regarding the ability to do this... I suspect that people who are very good at grasping systems quickly acquire the ability through a lot of exposure and reflection. I suppose that the more systems you see, the more likely you are to be able to extrapolate from portions to get a sense of what else is there. The more patterns you've read and applied in the practice, the better as well. Example. As a young'un I used to sleep with the C++ ARM under my pillow at night and I could answer incredibly arcane questions. Although I've slipped from that level of connection, I can still answer many questions correctly based on a general feel for the language. I think that the same thing happens with architects... they memorize a lot of details, but also develop a sense of the internal logic of a system so that they can recreate details mentally. -- MichaelFeathers

This sounds a bit like how babies acquire language, as described in TheLanguageInstinct. Babies are very good at guessing. A good designer will be good at guessing the designs of strange systems. In a sense, the baby is born knowing quite a lot about language and merely needs to fine-tune this knowledge by listening to its parents. -- DaveHarris

I can get that XP is a lot to swallow. I fronted up the money and flew to Detroit just because I couldn't believe what I was reading here and wanted to ask some questions myself. What I found was that the stuff here is true as advertised. I couldn't break it, and I tried. I think if you or your colleague went there, you would end up saying, "Oh, well it works because XXX or YYY, of course it works..." Personally, I find the whole thing remarkable, and Ron Jeffries has said that it went against all his reflexes at the beginning.

I also get that your colleague won't be swayed by anything we write here. TomDeMarco has certainly been involved in software for a good long time, and his considered opinion is that people ought to sit and stare at the design until they just get it right, and then type it in (or something very close to that). I have been involved with software for over 20 years, and I disagree with Tom. Who is right? Or rather, when and under what circumstances? I had a long argument with a waterfall-advocating senior designer, also with 20 years experience, and he told me that (the usual) incremental development simply can't work (for the same reasons your colleague said XP can't work!!)

(Sorry, my point is only that experts don't agree. I have met senior designers who advocate waterfall, as well as those who do or don't like RAD, or do ro don't like XP, etc. I do not intend to imply that people who disagree with XP must be advocating waterfall. -- Alistair) Please see InterTeamCommunication.

I can say that the C3 team sits in a single large room, all 11 programmers. I can say that they could whip out the cards and start showing me the system to whatever level of detail I could tolerate at the time. The scenarios live in the speaking with the cards, is lost into the air, and is retained in the ears of the listeners. That they rotate partnering and dig around in the code to find out what is happening. That they have been shipping product every 3 weeks for 2 years. That they have astonishing levels of testing.

I have heard the argument: "If you can't design it right the first time, I can hire someone who will say they can." And heard that line used to argue that iterative development is not necessary. I can also say that all the designs I have seen in my life have changed over time, never mind what the people promised up front.

I still get that you don't buy it. I also get that notwithstanding Ron's boldfaced text on several pages asserting that We Really Actually Do Design Here, that a residual impression is that there is no thinking or designing done. Finally, I get that XP is really frightening - it frightens me, and I have even seen, heard and smelled a success story in action. The amount of trust in people it requires is astonishing - you really have to believe that the people you hired can think as they go, communicate and work amicably with other, and have good intentions backed by attentiveness. That is a lot of trust to put on the line. All you get back is a delivery every 3 weeks, which is not enough reassurance for many people. -- AlistairCockburn

Ah - a delivery every three weeks. As opposed to a delivery of software in eight (or eighteen) months. You hope... But you're right, some people are so hooked on a project's virtual deliverables (usually paper describing an imaginary future system) that they don't know what to do with the system itself. I also see a lot of "we use a case tool/design methodology, so what we're producing must be good". -- DavidHarvey

See ExtremeProgrammingWithTypes.

Frankly, I find XP very exciting. It meshes well with everything else I know to be true in software. Regarding trust... after living with the ideas of XP a while (although I can't claim to be *doing* it consistently) it seems remarkable to me that leaders place their trust outside their teams. It is only easier if you are scared. I suspect that when teams are empowered to develop their own processes, they end up looking like XP, but the teams themselves may feel twinges of guilt.

Much of traditional development processes are founded on mistrust. They develop FalseCustomers. GeraldWeinberg was right. All issues are people issues.

-- MichaelFeathers

A CRC session, done correctly, takes a half-hour to an hour. It involves more than one developer. The probability of a bus hitting everyone in the session approaches zero, and if it does the impact is loss of an hour of time. It will take more than an hour to draw the diagram, and it will immediately be incorrect as soon as anyone starts to implement to it. Therefore drawing the diagram is a net loss of productivity. Don't do it.

I don't think I buy this argument. While I agree that you might need to be able to present an architecture (or "SystemMetaphor", as RJ puts it) to a group of suits, I can't see them getting anything out of a 2000-class UML diagram. It's not going to tell them anything. Then again, it probably won't tell me anything, either... (-; -- BillTrost
You're right, of course, Bill. CRC is a far more powerful technique for work with non-technical people. Our customers actually get involved with it, pick up the objects, move them, find defects in the design. -- rj

C3 has about 2,000 classes and 25,000 methods, done all with CRC. The issue isn't how many classes you have, it is how many you think about at one time. The right answer to that is probably "seven plus or minus two".

While agreeing that you can have packages with Rose, I don't see how this relates to whether CRC can be used effectively on large projects. Have you had Ward or Kent show you how to use CRC?

We make no claims for large team size. If you aren't in the same room, go to the same room. If you can't, you can't expect the same performance as XP, because you will have to substitute paper for speech, with a loss of some 80% of effectiveness. You can still do just fine, but (in Ron's opinion), you won't be doing XP. That's OK, you can still do whatever's best in that environment.

XP would suggest that thinking about the design a little and implementing it gives an even better chance of getting it right sooner, by providing feedback about how the code really turns out, not just how the design looks on paper.

No one is suggesting not thinking. Just not thinking for more than a few minutes. CRC is a fine methodology for designing scenarios. You pick up the cards, you move them, you show what really happens.

Carry on, you're doing fine the way you're going. If XP ideas help you out a little, we'll be just as pleased as can be. Best of fortune to you. -- rj

I agree, let's try to keep the tone soft in this dialog.

Picking up some points (sorry for the long append, but there are just too many dangling statements for me to leave it alone...)

"This can't have been applied to large projects (100,000 lines and up) because..." Not true. It has been applied to a large project.

Please separate "large problem" from "large project" from "lots of lines of code". I don't really know what a large problem is (i.e., don't have a metric for problem size), because a large problem can be solved with fewer lines of code depending on who's thinking up the solution. A "large project" is one with lots of people, so communication is the dominant topic. That is not necessarily connected with large problem. What C3 demonstrated, that is also clear upon introspection, is that a few good people working well together can solve a large problem. One needn't necessarily have lots of people.

"CRC cards are not a good way to design..." Not true. In my 7 years of interviewing designers, CRC cards have repeatedly been named as one of the best ways to design.
"...or capture a design...." Partially true. CRC cards are used to create scenarios. But the design and scenarios are then in the memory of the people in the room. There is no static record of the scenarios, which is why some people document the design with interaction diagrams. However, if you happen to have a small, stable project staff, then the static record might not be needed. YMMV, but that is indeed what XP is all about.
" The CRC mechanism begins to break down after about 30-40 classes." Partially true. I vote for around 20 classes at any one time. But as you move around the system (just as you would flip pages with a UML diagram) you have new conversations, into which you can bring new cards.
"...refactoring being questionable at best. If you have to constantly refactor your code, then you didn't get the design right." Refactoring is a given, and I'll happily bet you a dinner that you and he do that regularly on your projects, that you simply hadn't noticed or named it. RefactorMercilessly is a peculiarly XP habit used to keep making the design and implementation simpler and easier to maintain. Since it involves changing interfaces, having a small team may be a necessary prerequisite (see above about small teams).
"ExtremeProgramming to him and me seem to just be a way of avoiding design and just jumping into coding." I agree this is the perception, and can't help that perception. There is the radical part about XP that says to implement a small design with just one test case before tackling the second test case. However, I don't know that that is essential or central to XP. There is also the possibility of doing CRC cards with a couple of people before sitting down to the keyboards. And don't forget the week-long system design ("metaphor" design) session at the start of C3, after one year of designing and typing wildly with 26 people.

Personally, I think what we are seeing is that different people are working differently. I think KentBeck has gotten to the point, after rooming with WardCunningham and eating sleeping drinking Smalltalk for 15 years, that the code really speaks to him, and he can feel the force lines of the design when he scans across the code base. So I'll be he designs on the fly with Smalltalk guiding his actions. I think other people will happily sit and do whiteboard sketches and CRC and whatever, to get design issues stable in their head before they sit down. I don't get that XP mandates one or precludes either.

"A large program must be designed." I read an implicit "... up front". That is a whole big topic of how much - when. See BigDesignUpFront, XpDoesDesign. XP is definitely a vote for less up front and more distributed across the project. XP starts from the thought: "What if you could make it so design changes weren't to be feared?" That is where the real experiment starts, imho.
"ExtremeProgramming would only work on a small project with all the people in the same room." Mostly true. Ward says he has used it with small teams in private offices along the same corridor. However, the key is "small project", i.e., few people. A small team gets to work differently, and more efficiently, even when working on a large problem.
"We are working on a large system with teams. We have found the use of UML and RationalRose a necessity to communicate amongst the teams."

The key here is small team vs large team. XP is a vote for letting a small team work in the most productive conceivable way, eliminating all barriers to productivity, relying on the people developing excellent communication and programming skills. The idea, in part, is that if you hear, "... there are troubles to coordinating a large team...", then don't use a large team. If you hear, "... XP won't work for distributed teams...", then don't use a distributed team. etc. If it slows you down, don't do it.

Now, from where I sit, that means that XP has limits to its applicability - and that's where my discussions with Ward, Kent and Ron center. If, even with XP, you can't get 10 people to produce the needed system in 3 years (I am thinking Space Shuttle, here), I think you go get 300 people and work differently. If you must have assembler and COBOL and relational databases and Java, then XP won't transfer directly (but the core values of XP might still drive the design of the new process). Of course, the Ward Kent or Ron might question your basic assumptions... they know better than most that people box themselves into tough situations by overconstraining the ground rules. Change the ground rules and things get much simpler, all of a sudden.

"What if the project is 50 people, NOT in the same room, using different languages." Then you question why you are using 50 people (who then can't sit in the same room). If you insist on 50 people, you lose XP. If you want XP, you drop to 10 people and you put them in the same or adjacent rooms. On the languages part, I can't offer a suggestion. Smalltalk is particularly malleable, but the XP values might find a home.

-- AlistairCockburn, having really written too much this time

I have worked both in a XP environment and non-XP environments. I would say what differentiated the two was not team size, but instead was a combination of team experience and full-lifecycle considerations for the project.

The XP environment consisted of highly experienced developers producing a shrink wrapped product. The product was large (i.e., ~300 KLOC). It flexed to changing requirements easily (major upgrades were shipped every 3 months). As a result of being a long-lived, shrink-wrapped product, the design of the system was passed from developer to developer (both one-on-one and in CRC sessions). No one else cared. There was no need to permanently document anything. Life was good.

The non-XP environments I have experience with include large and small teams on large and small projects. However, they all needed to be "turned over" into a maintenance group that was never part of the original development team. Basically, it was a get-in, develop a product, turn over, get out process. Also, they consisted largely of average developers. The need for a structured process (and all of the resulting artifacts) was necessary both for the group that was going to maintain the system after the core team left and the core team. The core team needed it because it provided a structured, documented framework in which to develop.

The only projects I have been on that have failed have been ones with average developers trying to do XP. The ones I have been on that succeeded were either highly-skilled developers doing XP or so-so developers using a structured (albeit iterative) process (keeping in mind lifecycle issues).

I would have to say that the opportunities to do XP are rare. Because of this, it seems it would be better to decided to do XP if/when the opportunity presents itself. Using XP without real knowledge of the lifecycle issues and skills of the developers is a serious mistake.

-- HankRoark

Hank, in the above you seem to be assuming a different definition of XP from ours.

We seem to be differing on team quality, and on structure. For example, while the C3 team has good people, we are not overall anything special. At the same time, ExtremeProgramming (tm) as defined here is actually a very rigorous self-managed process of planning, communicating, designing/coding, and testing. XP as we define and practice it leaves very little room for surprises, and very little room for failure. In the context of your second-last paragraph, I would characterize ExtremeProgramming as iterative (clearly) and very highly structured (apparently not obvious to the reader).

As for needing knowledge of the lifecycle, definitely yes. I believe that we may differ on the skills required. At any point in time, including the hand-off to a maintenance team, an XP system is well-factored, well-designed, and comes with a huge number of tests, all running at 100%. This is a basis for ongoing maintenance that's pretty hard to beat. -- RonJeffries

Ron, I think we agree more than we disagree.

Like you said above, the C3 team was good people. As much as I dislike this, I would say that is rare. Or at least not as common as you think. Most teams I have worked on have had one or two outstanding developers and everyone else was average (or worse). Plus, most of those teams were far from what I would describe as self-managing. To top that, I would say that it is becoming worse as the demand for IT professionals is causing organizations to overlook someone's less than decent design/practices in the name of functionality. If functionality is all we are concerned about, use ExtremeProgramming every time. I hope we are aiming for functionality plus several other qualities (maintainability, reusability, testability, etc.).

I am not trying to say ExtremeProgramming is bad. Like I said, I have worked on XP projects and life was good. It is just that the pragmatic side of me says that it is rare to have a situation (self-managing, above average software engineers) in which to use XP.

(Maybe my experience is distorted because of my current job role. I tend to get called into projects that are in trouble to help turn them around. XP would have failed every time because the developers were less than average and needed a highly manageable framework/methodology (Objectory) to work within.)

-- HankRoark

Hank, you seem to be implying XP doesn't bother with testability. That is not the kind of XP I read about here. This XP places enormous emphasis on UnitTests; they drive the method. They are used to specify new code and they help document old code. I'd say XP generates a great deal of documentation, but it's in an executable form. So I really think you're talking about a different system. -- DaveHarris

XP is excellent for less-than-average programmers, primarily because of the emphasis on testing and PairProgramming. These two practices together will quickly turn anyone who can program into a competent programmer, or drive them out of the team.

Re: self-managing. Everybody is self-managing. The only question is whether we try to go with that or go against it. At first, there must generally be loads of structure within which to self-manage. Over time, that structure can and should fade away. Ron will tell you- I am one fascist bastard at first. I am trying to focus people on making clear commitments to the team, relying on the rest of the team to meet their commitments, relying on the team for help in meeting individual commitments (PairProgramming, for instance), clearly estimating level of effort, seeing and responding to feedback about those estimates, notifying the team when there is a problem, and clearly delivering on the commitments (via UnitTests and FunctionalTests, in part).

Re: functionality vs. all the other good stuff. I'm too stupid to deliver functionality without having a simple, shared design, lots of tests, and a clear understanding of what I don't have to implement. This adds up to the good stuff. For me it isn't a versus, it's an implication- the good stuff implies functionality. This is what leads me to believe that you are talking about a different kind of ExtremeProgramming- the XP I do just won't work if "functionality is all we are concerned about".

Re: "needed a highly manageable framework/methodology (Objectory) to work within". Oh baby. The ability to make two guesses in a row reduces your chance of success by the product of the probability of the guesses being wrong times the cost of a mistake in each decision. Put a hundred or a thousand guesses in a row, and get someone to pay for it because you can print tall stacks of paper, and the best possible result is that you will leave before they stop paying you.

My solution- don't. Make one guess. Test it. Repeat.

-- KentBeck

A while back I said to myself: "this XP stuff is really a management methodology." But, I kept it under my hat. It looks like Kent and the XPers have taken just about every risk in development and introduced a safety net for it or a counteracting force. -- MichaelFeathers

Just a thought, but large teams are often large for a reason. There's often a group the works with hardware. This is a skill you can't just rotate into. There's a group working on BSPs or other low level items. This is a skill you can't just rotate into. There's often a group with very specialized PHD level domain knowledge in areas like switching/routing/whatever. This is a skill you can't just rotate into. There are often account teams that work with customers and have relationships with customers. This is a relationship you can't just rotate into. And of course unless you are working on a completely new project you rarely get to work just on the current project which requires a lot of task switching and it does require meetings to keep everything straight. And so on. I have never met anyone at all the can realistically perform all the roles in a complex project. If you are just building software application, it could work, but that's not representative of large portions of the world.

In response to the (immediately) preceding point and other points: I notice earlier in this page that Michael Feathers said "this XP stuff is really a management methodology". I agree, and I think this is a fundamental point. We all know that many of the worst problems with projects relate to organizational and political issues rather than technical ones. In software construction our challenge is to translate volatile fuzzy ideas into a solid executable reality. We want to produce systems that meet customer's needs and we would like the customer to be satisfied and express their satisfaction. In reality this means that we must be negotiators and mediators. We must understand how to see it from the other guy's point of view. We must understand how to take all of those fuzzy ideas and package them together into a good deal for everyone. For so long as we are building the system we are still forming the deal. (Sorry for this epic reply - it's just too big to communicate briefly)

Seen from this perspective, XP has some dramatic advantages:

XP has a much stronger and more effective focus on cooperation than non-agile methods (e.g. RUP). Every project contains a problem of communication and perception. This problem is not a side issue, it's central and it's a major source of trouble in many projects. By pairing developers and marrying the customer to the team we are continuously working on solving this problem.
XP is intensely focused on results: "nothing is sacred" if it works use it, if it doesn't change it. Of course, other methods are aimed at getting results too but XP is "results first" rather than "method first". This is a key philosophical difference.
XP ensures that all technical staff (and the customer too!) receive continuous training
XP totally integrates testing. This improves the quality of the system, educates people about testing, and translates requirements into tests thus improving the quality of the requirements and increasing the chances of delivering a satisfactory system
XP tries to embrace change rather than fighting it. In XP you still control change, however, you focus on cooperation rather than domination. I think that's a good idea.

As for the question of scalability, I think scalability is a definitely problem, but it's a problem in every method. There is the point about differences in teams raised above (customer service, hardware, roll-out, strategic architecture, business teams etc.) There is also the question of program management and portfolio management. We can see XP works well for developing within one team. It can also work well for big systems (for example, I ran into an Australian CEO who claimed to have a 2 Million line system that was now entirely maintained under XP. He had photos of the developers pair programming and having stand up meetings etc. to prove it). But how do we do this multi-team/program management stuff? I've put my 2 cents worth below. I would very much like to know what you other guys think.

As Robert Butrick points out in his brilliant book "Project Workout" (ISBN 027364436X ) two central reasons for project failure (in any kind of project, not just software) are firstly that "organizations don't know how to control change - there's no 'company-wide' way of undertaking business change initiatives". Secondly, "organizations don't know what they should be doing. There is no clear strategy for decision making." I think that the question of scaling XP is really a question of how to include XP in an overall management process that works. In other words, I think it's part of a more general problem. However, let's suppose that you canXpCritiqueXpCritique see how to introduce a simple staged framework for all of your projects, and you can see how to collect the data needed to make informed (strategic) decisions about cross-department/cross-company projects and programs. How do you include XP in that?

I think that the suggestion of having a kind of "mission control board" made up of coaches (and representatives) from the different projects is a good start. It would be like a project board but XP oriented and not as stagnant. You could also have pairs of people that act as on-site "interfaces" between teams with interface members rotated at certain intervals. The sponsor would chair the mission control board and the strategic requirements would be introduced there by stake-holder representatives (you could even still have planning games). Release and iteration planning would be done in detail with specific teams, but strategic planning would be done at mission control board sessions (perhaps once weekly). It would also double as a risk control board. -- AlanRoberts

CategoryXpCritique