Xp Is For Bad Planners

A belief that ExtremeProgramming is for those who are poor planners or poor designers. If you know how to plan and design, then you don't need XP because XP has high costs associated with PairProgramming, UnitTest management, constant ReFactoring, etc. (I am not against a certain amount of some of these techniques, but XP over does it.)

XP is a result of the large percentage of software project failures. But some of us have NOT seen our projects fail for the most part. Thus, we don't need the "safe mode" of XP to achieve success.

XP is a "brute force" approach to reaching the desired goal in this view. If planning/design fails for an organization, then you can hire an XP army to reshuffle and clean code to provide what is needed and only what is needed. However, that is expensive. (Well, maybe not if things move to low-wage countries. But some say this makes understanding the business tougher.)

Perhaps XP is the right choice if quality is more important than budget. It is a technique that is pretty sure to reach a satisfactory result I will agree, but at great expense due to its brute-force nature. Companies really want "cheap but reasonable". It takes good planning and design skills to pull that off, but is possible. GodIsQuickerThanEvolution. I suppose the problem is that it is hard for a manager to know if a designer is good in today's world because people shuffle around often. From the manager's perspective, XP may look like the way to go because the outcome is fairly certain, even though expensive. But a designer that knows they are good may not see it that way. It is an odd political conflict resembling NobodyEverGotFiredForBuyingMicrosoft.

It is a classic case of IncompatibleGoals. XP sacrifices cheap because of the army of duplicates and refactorers it needs.

And there are no good planners.

Well, I think I can do pretty well without XP. It is not just a matter of good planning, but also good future-proofing. There are patterns (AntiPatterns perhaps) to things that grow into messes, and I make mental notes of these.

XP is a collection of those notes.

What use is such notes if you do not take advantage of them? XP does not learn from history, but rather makes reinventing it faster. Learning from history and using those lessons is called planning and design-up-front.

[Perhaps that is why older developers tend to evolve into architects? Since I find that my engineering experience outweighs most of the rest of the development team on a lot of my projects these days I tend towards architecture instead of coding. Sure, I can code with the rest of them, but why? Why not put all this expensive empirical knowledge to use where it pays off the best? That's design up front, avoiding the pitfalls of refactoring before they happen.]

Where is it said that XP does not learn from history? Where in planning and design-up-front does learning from history occur? If anything, XP would seem better in this regard, as it puts the planning in the hands of those who would actually have a history of development.

I just realized the other day that - although I have been fired >10 times in as many years - I have not once been fired from an XP job. Both went over 2 years.

Now, what were you saying? --PhlIp

XP isn't just the refutation of BDUF; it is also pair programming and unit testing. I think many of the comments here (above and below) on the pro-XP side are more properly agile programming arguments. The anti-XP side can argue that, if you can't plan well, then you need pair programming, unit testing, frequent iterations, and constant refactoring. The pro-XP side argues that no-one can plan well. I consider myself an agile programmer (tho I don't use XP), and I would counter that constant refactoring and inspection provide most of the antidote for the inability to plan perfectly.

"XP is for bad planners"? I will agree with the above that there are no good planners. But I think absolutes such as "there are no good planners" and "XP is a collection of those notes" are purely polemics. BigDesignUpFront captures XP's stance on planning: planning is important not for the material result (the design document) but rather the process. Planning results in a better understanding of the problem domain and possible solutions. This can be viewed as mental results, and my team is trying to codify the results of planning efforts in a private, project-specific Wiki.

Actually, there are FourVariables in software development, not three.

Depends on how you want to divide things. There are probably thousands of variables, but we humans simplify stuff to avoid going insane.

It's the choosing of the simplification that counts. I could say there's only one variable, which is Caffeine, but I wouldn't get much more than a chuckle for it. Caffeine is a constant.

And XP teams are never armies; 10 is the maximum recommended, and 4-6 is optimal. But hey, if you never use XP because you've done enough homework to not need it, you're far ahead of most of us. Myself, I find that XP helps me learn to be a better developer ... and a better planner.

The teams might be small, but for large projects there are probably going to be lots of teams, unless time is not a factor.

Is this based on experience or an educated guess?

Okay, I have not taken a formal sample. Do you have such numbers?

I would suggest the issue is not good planners versus bad planners, but whether the cost of planning exceeds its value. XP asserts that it is more cost effective to do minimal planning and perform multiple attempts to accomplish something than to plan everything out. The standard waterfall approach asserts it is expensive to perform more than one attempt, so everything needs to planned in detail to allow success on a single try. Neither approach seems to have much beyond anecdotal evidence (but XP seems to be anecdoting better for me).

But the premise is that the more "organic" approach of XP is needed because so many projects otherwise fail. My non-XP ways (more or less) don't fail for the most part. Further, the advanced planning can save a lot of steps that otherwise would have to be reinvented from scratch under the XP approach. If you are a good shooter, then you can toss your garbage into the can without getting out of your desk. If you are not, then you have to walk over to the can. Sure, even good shooters will miss sometimes, but they hit often enough to not justify walking every time. XP'ers would keep walking all the time. True, us "planners" sometimes get things wrong, but it is a cost-benefit analysis. There is a point where you guess right often enough that you can rely on it and simply eat the cost of occasional mistakes. Plus, most companies are cheapskates who want and like shortcuts. I can use my past experience to satisfy the owner's desires for quick, inexpensive turnaround. Pairing and bothering the customer all the time is costly. (Note: VeryMildXpOkay)

I don't agree with the premise that "XP is needed because so many projects otherwise fail" The statements I have read are merely that it is more efficient. So, how does "advanced planning ... save a lot of steps that otherwise would have to be reinvented from scratch under the XP approach?" There is nothing about XP that makes people less knowledgeable about what must be done, nor anything about advanced planning to make them more knowledgeable. In both cases, to achieve the same result, the same information must be gathered, or are you suggesting that guessing should be used instead?

If you don't mind me asking, what types of projects do you work on? Team size? Schedule length? Customer type/domain? Language/environment? Just trying to get a better picture of your point of view. I read a good book recently that seems to take a similar view, called TheSoftwareDevelopersGuide? ISBN 1-930919-00-X .

The project boundaries are often fuzzy. But I would generally say roughly "medium-sized". See SystemSizeMetrics.

In XP, the time it takes to answer questions is shrunk by having a customer always available and engaged. This is a key strategy for keeping things agile.

Here's one way of putting this to someone who believes in BigDesignUpFront: If your customer was (almost) always available to clarify what you need to work on next, would you do the same amount of design that you do now?

{I don't think it is practical to have the most knowledgeable users always available. If they are, that is great, but if they have enough time to always be available, then they probably don't need automation in the first place.}

But in many projects, particularly coupled projects, it takes a lot longer than the XP ideal to reach clarity in the requirements. One way to deal with this is to kick in XP development practices once the plans are sufficiently in-focus -- but not before. The plans don't have to be quite as in-focus as with a waterfall or big design up front methodology. View this as reducing the risk at entry to the development phase by deferring low level planning to right when needed. Isn't some higher level meta-planning useful to understand when various plan elements are needed for the project to avoid being held up by late-breaking long-lead item surprises? XP practices can be used to generate these early plans, but instead of the product being a deliverable executable, it is a sellable plan. Frankly, I like the idea of unit testing a plan.

One of the strengths of XP and other agile methodologies is the recognition that, whatever you deliver to the customer, it isn't going to be exactly what the customer wants. It won't be what they expected, or seeing a working system will give them new ideas about what they really want. No amount of good planning can eliminate this effect.

Yes, but in my opinion good planning reduces the problems caused by this effect. As a designer, if I don't have a pretty good feel for what the customer generally wants and the general "shape" of the business, then more study is needed. I generally know where that point is after years of experience. Quicky mock-ups or dummy report examples can often be used to flesh out the fuzzier or riskier aspects of the system. Good planners often know where the potential soft spots are. Perhaps use XP for the fuzzier parts of the project rather than the entire thing.

The BDUF proponents' response to this result is often "The customers signed off on the requirements and on the design, so if they don't like what we delivered, that's just too bad for them." In contrast, the XP response is "OK, this feedback is great. Tell us what you want for the next iteration." The cost of change associated with XP can be expected to be smaller because the initial delivery comes much earlier and feedback is available before too much inertia sets in.

I am not against feedback. I just don't think it realistic to have a full-time on-site customer, on both their part and the developers'.

What do you mean by "unrealistic?" A full-time on-site customer exists in some situations, so it's certainly a real phenomenon. Even so, a "full-time" customer is usually not a large cost. Customers must be available at all times for face-to-face communications, but they will generally have lots of time to do whatever their normal work is.

In my observation, the points in the timeline where something is available for inspection is feast or famine. No use having a customer on-site when there is nothing ready to test. I suppose this topic belongs under OnSiteCustomerDebate? or something.

As Kent Beck writes in ExtremeProgrammingExplainedEmbraceChange, "If having the system doesn't bring more value to the business than having one more person working, perhaps the system shouldn't be built."

{If the customer wants a system built but doesn't want to have a full-timer in the IT shop, that is their decision. Some of us are capable of delivering such such systems for the most part. If XP'ers want to pass up such projects and give them to me, that is fine by me. Experience with cheapskate companies has given me techniques for living with less-than-perfect resources.}

Besides, someone must iron out the requirements, whether it's all done up-front or in a continuous manner. Someone must spend a significant amount of time on this task. That person had better be the most qualified to do it, or you won't get the system you want. Now the questions are: Do you want that person's time sucked up wholesale at the beginning of the project, or bit by bit as the project progresses? Will that time be most effectively spent if spent up front with no feedback, or bit by bit with continuous feedback?

Sometimes fairly detailed analysis needs to be done to know if the project should be undertaken in the first place and how much funding and resources it will get. Thus, we might as well kill two birds with one stone.

The problem is that the "detailed analysis" must also be funded and man-loaded. At least with the projects I have been involved in, we need to produce the schedule and staffing levels for the planning, implementation, and deployment before any work starts. Once we have made those predictions, it doesn't much matter whether we do BigDesignUpFront or incremental development; with the exception that incremental development gives us earlier feedback on whether the project is in trouble.

I disagree with the "earlier" claim because planned designs move faster because GodIsFasterThanEvolution?. One will get to a working version that the customer can see faster because there is no (or less) constant refactoring and double labor with pairs going on.

XP is for bad planners because even they can write a good program as if it had been well-planned.

{I generally agreed with this above in my "brute force" comment. However, it takes more resources.}

Can someone explain this statement, please?

I think XP is simply an acknowledgement of the truth, that planning isn't possible because change is inevitable and unavoidable. XP isn't for bad planners, it's for realists! In the face of change, planning is a bad strategy, accepting change and dealing with the now is the only way to succeed.

I think XP is for bad planners. And we all are bad planners. It's completely beyond our possibilities to predict everything related to business and development in 3 months. So we need to adjust our plans all the time. So we need to think in the short term. That's why planning which component will be programmed by which developer in 4 months is *completely* useless. Agile methods, including ExtremeProgramming, try to deal with the uncertainty (RefactorMe) by constantly looking at reality and adapting. If we were able to plan everything, I would agree than a BigDesignUpFront of the hole thing would be better. But we can't. Then, things are much more fun.

Well, I am not a bad planner. The "70% failure rate" might be an industry average, but it did not come from me. If I don't know enough to make a pretty good BigDesignUpFront, then usually I know that I need to study the needs more. My projects generally don't fail, assuming I am given enough time and resources to study the needs. Experience and observation have taught me a lot. Another thing is to manage and explain expectations. Many people still don't get GiGo. I think many projects fail because expectations are manipulated for political or greed reasons. XP won't help there.

Burned

I recently I've seen a big failure of DoTheSimplestThing. It was a web app where people could "publish" pages with a few paragraphs and a handful of pictures. Originally the customer said that it was "write once" in that once the pages were "published", there is no real need to re-edit them. In the (allegedly) rare case that a typo was found, the user could simply delete the bad one and re-create a new one via copy-and-paste and picture re-uploading.

So, the programmer kept the source code simple by not keeping the page text and attributes in a database or repository; rather just generated a static web page after Submit was pressed on the spot.

Originally I was only a minor player on the project, and when I heard the "write-once" declaration, my developer horse sense kicked in and I stated an objection that the requirement for re-editing is likely to pop up in the future. But it was dismissed. "You are over-complicating things", was the charge against me. I saw nothing special about this app that made the desire to re-editing significantly smaller than normal.

Sure enough, after some trials, the key user requested, "I know I didn't originally ask for it, but how hard would it be to add the ability to edit an existing page?" This, along with some other poor decisions, has derailed the project. Not only does the entire "publish" process need to be reworked, but the URL references all need to be changed in other parts of the systems from a static convention into a dynamic one. And the existing static pages need to be re-parsed or re-typed to separate the attributes and text to be loaded into a database. (Fortunately the mistake was caught early in the process, otherwise mass re-parsing and loading would be needed.)

I believe many of the actual problems encountered are caused by "the suits" (PointyHairedBoss). Most techies like to do proper analysis and study first and address complicated realities rather than hide from them, but political issues often trump an engineering approach, creating pre-doomed trains. XP perhaps may be a way to get success despite bad political decisions using a kind of brute-force approach and processes that reduce a tinkering-happy big-wig's influence. Thus, it's more of a political solution than technical one.

See: ExtremeProgrammingAintPrototyping, RelationalVsXp, PrematureComplexity

CategoryXpCritique