When Are We Gonna Need It

When I see people use YouArentGonnaNeedIt (Yagni) and DoTheSimplestThingThatCouldPossiblyWork (Dtsttcpw) on this site, I get the impression they really mean it. So, if someone said "code this report" they would have some routines that hard code all the logic for that particular report. That is the simplest thing that could work and it doesn't look forward at all into future needs. If someone says "oh, we need this other report", well, isn't the most simple thing just to repeat the idea behind the first report and hard code the second? etc. etc. [You are forgetting refactoring here - PeteHardie] At some point you must realize hey, we ARE going to need it and design a more generic solution. There are a lot of verbal gymnastics on this page to basically blur the moment when someone realizes "yes, we will now need it" and we need to make it more generic. Let's face it, unless you are working on a very limited app, you KNOW you are going to need a robust reporting engine. Might as well implement it the right way from the start. And that doesn't mean BigDesignUpFront. You can still do it in iterations. I think Pete and I are agreeing here in some respects although we are saying it from two different starting points.

I think this is a valid complaint against the XP literature, and hopefully we can fill in the gap here. If XP tells us we aren't going to need it right now, how will we know when right now happens? How do we know when we're gonna need it? Sometimes people think these things go without saying, but lemme tell ya, when you're talking to managers and customers, they're going to want good solid answers, with no fluff and no hemming and hawing.

It seems to me that XP says there isn't a "right now". It's not a matter of hemming and hawing; the claim that is made is that if you follow the process - build by functional increments, writing tests first and refactoring at every opportunity - you will end up with a generic solution; that is, one that is precisely generic enough for your needs; and there will never have been a time when you sat down and wrote a generic whatever-it-is. There will be times when you sit down and make this little bit slightly more generic, or this other bit slightly more robust; but these will happen regularly.

That there is no exact answer to the "when" question isn't a logical flaw. There is likewise no exact hour, day, or year when a young person becomes an old person - but it happens.

Maybe the answer is, ListenToTheCode. If you try to follow the other XP practices such as OnceAndOnlyOnce and ReFactoring, your code will tell you when you need the thing. In other words, if your question is "When do we need generic feature X to replace our hard-coded features A, B, ..." then the answer is, when you are refactoring A and B and notice that you can improve the code and satisfy OnceAndOnlyOnce by merging parts of A and B into some part of X.

Also, maybe if your generic X is a "large" feature (a report meta-generator or whatever), perhaps the reason you don't know when you need it is because its granularity is too large. Perhaps a subset X_ab of the functionality of X is totally sufficient to refactor A and B. In that case, you need this subset X_ab when refactoring A and B but you don't yet need the rest of X. And you may never need it--you only thought you did because you weren't able to separate the X_ab subset out from the rest of X in your head. So rather than guess into the wind, ListenToTheCode.

In a sense, we always need it, as developers with reputations for being able to deliver new things promptly. Whenever I'm tempted to build a framework with only one or two examples to base it on, the same thing is always going on in my mind: Make it real easy to do this the next time. I love capability. I love tools. I love readiness. I need it all the time.

-- WaldenMathews

It's common knowledge that they need it for yesterday. ;)

You need it when the OnceAndOnlyOnce rule tells you that you do. If you write that second report and it duplicates some logic from the first, you should RefactorMercilessly until there is no duplication of logic. Code duplication is not the simplest thing that could possibly work; it's just the easiest thing. Simplicity is more elusive. --JamesDennett

Isn't there are rule that goes something like OneTwoThreeRefactor, or ThreeStrikesAndYouRefactor?

Refactoring doesn't get you the new reporting architecture that you knew you were going to need at the start but had to pretend like you didn't because of XP religion. It just makes you have to get there the long way and consume many more dev and QA resources on the way. To me refactoring is different than a ground up re-write. It is naive to say that refactoring your way to a fundamentally different architecture for a subsystem is always the best course of action. Hey, I like XP, but there are some problems with it, and I want the XP leaders to acknowledge and account for them in XP philosophy. There are YagniExceptions, many of them in my opinion. The question XP needs to answer is how can you tell if you are dealing with YagniExceptions.

It honestly sounds like you've never really tried XP. That doesn't sound at all like the refactoring I know. Do you have any first hand accounts of how XP has failed you?

From ReFactoring...

: Practically, refactoring means making code clearer and cleaner and simpler and elegant. Or, in other words, clean up after yourself when you code. Examples would run the range from renaming a variable to introducing a method into a third-party class that you don't have source for.
: Refactoring is not rewriting, although many people think they are the same.

So what refactoring is it that you know? I think of the above when someone talks about refactoring. It is both intelligent and beneficial to build in generality in some cases when you know YouAreGonnaNeedIt even though a user story doesn't say so. That is all I am saying. Now getting XPers to admit this is the difficult trick. The value of Yagni, in my view, is to keep people from gold-plating their code. That it has been turned into a rule with no exceptions seems awfully naive to me. (Even still after being on XP projects.) Perhaps in place of Yagni XP should have a rule called NoGoldPlating?.

[I split the above exchange up to make to back-and-forth clearer. I'm 99% certain I got it right. If not, feel free to fix it. Sometimes, signatures are helpful.]

So you're adding a feature (possibly an internal one, such as making a class more general) that's not used by any other code right now, and won't be used by any code in this iteration?

I don't see how that counts as refactoring. If anything, a concern for refactoring would normally lead you to remove that as-yet-unused code.

If the generality you've added is in fact being used by other code (in a chain all the way to a CustomerTest), then Yagni explicitly allows it: either you've got a new CustomerTest you used it to satisfy, or you added the generality to make the existing code that satisfied the CustomerTest cleaner, clearer, simpler, or more elegant. --GeorgePaci

I didn't have to look far to find an example of a noted XP project where Yagni failed them:

http://extremeprogramming.org/stories/functionaltests2.html

If XP gave better guidelines for WhenAreWeGonnaNeedIt (i.e., not the overly simplistic Yagni), then perhaps they would have that easily updatable test suite.

YouArentGonnaNeedIt is not the same as YoureNeverGonnaNeedIt?. When you call Yagni overly simplistic, you're really talking about Yagni. Yagni says unless you need it right now, don't do it. But if you do need it right now, then do it. And Yagni is always coupled with Oaoo. You keep talking about Yagni on its own. Stop that! :-)

From the above page:

: I had noticed that it was difficult. But continued on anyway. That was my mistake, we needed to create a tool to maintain our tests. A custom made, domain specific tool that would aid us and our customers in creating acceptance tests. My excuse was that there was never enough time to create it.

Don admits that he wasn't following Yagni. He did need it right then, but he figured he didn't have time, and this ended up backfiring (no surprise). Yagni is about avoiding gold plating and AnalysisParalysis, and it is balanced by Oaoo. It's not simplistic or naive. The people following it may be (by following it incompletely or incorrectly), but the maxim as it stands is robust.

It seems to me there are two situations. One is I don't have a robust reporting infrastructure from a previous project and I have to build one. XP says the most appropriate way to build one is through the principles of YouArentGonnaNeedIt and OnceAndOnlyOnce and the practice of ReFactoring. The other situation is that I already have a robust reporting infrastructure from a previous project or I bought one or whatever. In that case, if putting that in is the SimplestThingThatCouldPossiblyWork, XP says, go for it.

At our roundtable, one person from an XP project I won't name, but which had run well and is occasionally cited as a success story, told of one place where they said, "we don't really need it... we don't really need it yet..." and put off refactoring too long. Then, one day, they saw the kludge they had, and it was suddenly too big for a reasonable-sized refactoring - it was easier to keep changing 20 places than to refactor into a tidy design.

The point I want to make (before you all leap in and criticize them for not having following the true path of XP, etc.) is that this raised in our mind the notion of refactoring having a range: don't need it yet -- good time to refactor -- too messy to refactor.

So this page is good. How does one detect those two boundaries? cheers --AlistairCockburn

Always remove duplicate code? YAGNI should not be used to avoid writing clean code: YAGNI is about not putting in features when you don't have a card for them. TheSimplestCode is a daily technical chore like cleaning up your room. When the chore is put off too long, it becomes difficult, even impossible, to clean it up. C3 reported a similar experience. They detected the problem through a measurable decrease in velocity. In their case they begged for, and got, a full iteration to clean up the code to bring their velocity back up.

We need it when the code looks ugly without it. Unfortunately, many programmers can say to a manager, "We need a report engine because we will need to code more reports in 2005", but can not say things like "We need a report engine because two reports coded straight on C++ look ugly!". The only way out I see is to say again and again that YouNeedCleanCode. --AlexJouravlev

(See YouNeedCleanCode for a rejoinder to this.)