Xp Vision Scope

UserStory(s) might communicate your stakeholders' Vision, but this is delegating the Scoping activity to someone else. Stakeholders are often folk who don't know how to do Scoping well, who don't have a good feeling for what's really useful to users, and who happily toss around inconsistent requirements.

Scoping's result is a concrete problem statement. The stakeholder may say, "I want you to climb the tallest mountain in the world", but Scoping tells you how high that is. The stakeholder may say, "I want you to sail to China", but scoping tells you how many ships and where the best ports are.

Scoping is taking the UserStory(s) and filling in the blanks to build a consistent, bounded Vision, taking that Vision back to the stakeholders, accommodating their feedback, and iterating until the Vision obtains buy-in enough to Spike. Or it may be done after a Spike; only by climbing a nearby hill do we catch sight of Everest. Like the XP activities, Scoping is something you want to keep doing throughout the project.

So is this activity replaced by XP, called by another name in XP, done a different way in XP, or perhaps still needing to be formalized in XP?

-- PeterMerel

RonJeffries replied:

XP is about seeing the far side of the lake, how high the mountain is, as clearly as actually possible, not by introspection, but by production of actual artifacts.

I referenced briefly: UserStory cards, SpikeSolution (ArchitecturalPrototype?), CommitmentSchedule, WorstThingsFirst.

Here's a little more about how those items address the vision and scope issues PeterMerel raised:

Let's remember the CommitmentSchedule, which is done by estimating all the UserStory cards for the whole system. This is done at the beginning, and every (9) weeks thereafter. C3 did it right after our BootCamp? (not described here) and SpikeSolution. At that time, we had something like 42 weeks' worth of stories for ten developers!

Let's remember the SpikeSolution. C3 did a "complete" end-to-end payroll system before we even started. It only paid one guy, it didn't pay him any overtime, it took only a (fixed) percentage as taxes. But it tested our entire architecture. (This specific SpikeSolution can also be called ArchitecturalPrototype?.) [PragmaticProgrammer calls it TracerBullets.]

Let's remember WorstThingsFirst, the process by which we consider first, and do first, the highest-risk problems. By this practice, we encounter as quickly as possible the things (we know of) that might kill us, and we subdue them. (When something suddenly comes to mind that might kill us, we shout, We're in big trouble, guys, in honor of a former team member, and kick it around. If it does seem to be trouble, we attack it right away to be sure we can crack it. Sometimes we just discuss till we see it isn't hard; sometimes we card it; sometimes we Spike it to see what the code is usually like. This far in, discussion is usually enough.)

We arrange the CommitmentSchedule largely in WorstThingsFirst order.

In sum, UserStory, CommitmentSchedule, and WorstThingsFirst are all about Vision and Scoping. Other rules too, but these should remind us how XP addresses V and S. -- RonJeffries

This came very close to answering what I was asking, but today we've gone off tangentially. I'm hoping a slight reorganization will get us back to this train of thought. -- PeterMerel

Well, there we go. Are most of the story cards done at the beginning of the work? Or do you get a few, work, add some more, work, add some more... If you do stories in advance of the work that will be happening for the next few weeks? Why? You really are doing some analysis... the design coalesces a bit, perhaps not very consciously by what will be expected a little further off, unless you just lock all those cards up in a drawer and pull out two at a time. That is, make a conscious effort to not mull over the how and use the advance stories only for estimating.

Hmm.. and by doing WorstThingsFirst you minimize the cost of rework.

-- MichaelFeathers (probably answered his own question)

We measure the project by Resources, Scope, Time, Quality. We report each of those explicitly, as described elsewhere.

That's fine. Maybe my words are poor choices; if you're already using "Scope" for something that isn't refining the problem statement, then feel free to pick another word for the activity. "Analyzing" would be more traditional.

Our first report is before the project starts: Quality assumptions (no tests yet exist), Resource Plan, Scope (User Stories), Time (the CommitmentSchedule). How do these practices relate to your Vision/Scope activity? -- R

In what I do, the UserStory(s) are used to model classes and collaborations, and these are plugged together and reviewed iteratively by stakeholders until we all get a warm fuzzy. The Time and Quality assumptions (the non-functional requirements) are used to make these models as concrete as possible. As to where, in general I see this Scoping activity as something you do between Listening and C/T/R. -- P

I am backing up to this point in the discussion: "Stakeholders are often folk who don't know how to do Scoping well, who don't have a good feeling for what's really useful to users, and who happily toss around inconsistent requirements. "

OK, let's assume that is the case. What should you do now - write a requirements document or write some code? That is the start of the XP story. If you vote for document, you are off XP, because XP is based around feedback from running code. But it sounds to me like we have agreement that running code will work better than documents - it gives the sponsors something to look at to set their ideas.

I'm all for running code. I'm all for CRC and no document - where I can get away with it. I'm happy to do it the simplest way that gets it done, even by actively not doing it but I still think it needs to get done somehow. I feel confident that XP does get it done. I'm just asking how.

Your next question sounds like - But how do we get Sponsor to shell out the money for code development when we don't have Scope? That is perhaps where the tricky bit is, because one has to do some selling one way or the other, and one has to take a decision and bet on it. My line would go something to the tune of: we don't know exactly what it is, we don't know very well how big it is, so we are going to build a little piece of it to get the size and cost of it, and to get an idea of what is really needed; we could spend the same time to write it all down on paper, but we still wouldn't know the size, cost or accuracy of our guess. At this point there is a gamble, and one either wins or loses, goes XP with SpikeSolution etc, or sits and writes paper.

Say you Spike. Examining your Spike tells you something. You need to figure out whether you've actually now got a fair idea of the scope of the whole problem or not, and you need to bounce that understanding off your Sponsor and iterate until you get buy-in. How XP does this is my question.

One thing to recall about story cards. A story card is so informal that it does not contain a spec. It contains a promise for a future discussion. The spec, the interface definitions, come out of those future discussions.

Also, "make a conscious effort to not mull over the how and use the advance stories only for estimating." Yep, that is exactly what XP says to not do and to do.

-- AlistairCockburn (testing his XP knowledge)

I'm afraid you lost me on that last bend. -- anon

Not relevant to scoping, in the XP practices. They say (or are supposed to) specifically not to look several use cases ahead when designing. That's the do the simplest thing that can possibly work philosophy. Only look at a use case when you get there. Was a big surprise to me, and quite interesting to watch in action. -- Alistair

Mostly good, Alistair (as if I were somehow suddenly the arbiter of goodness). However, experience is that the more comprehensive (though brief) a story card is, the better things go. There is always an implicit promise of future discussion, but if the card includes the file format or the table of lookup values, time can be saved.

As for the "how", when estimating, one does discuss, for a very few minutes, how the thing might be done. -- RonJeffries

OK, Peter. If you had said Analysis, I'd have understood. XP, to me, starts with the stories. I believe that the book series doesn't address how to get the stories (though we know a guy who could write that one ...)

I would have said Analysis, but Kent said back on MetaExtremeWay that he saw no difference between analysis, design, coding and the rest, so I was hoping to talk in functional terms to get at the nub of things. I wanted to describe just what it was that I don't see how you do.

Scope, to us, is the span of the stories: what there is to do.

So that still sounds like what I'm asking; so how do you determine the span of your stories is a good representation of the scope of the problem?

I reinterpret your questions, now, this way:

If you have nothing but a notion for some software to spend lots of money on in hopes of making lots later, how do you figure out, generally, what you might do? (Vision)

I think XpChallengeCarteBlanche is an extreme case of a general problem. You don't usually get plenty money + no vision. You usually get not enough money + not enough vision - completing the vision gives you the ability to say either change the scope or provide more resources, but more important it gives you the perspective you need to make decisions about priority and risk.

This activity tells you you things like how many developers, what's the best choice of tools, what sort of context will the solution need to live in, and so on. Is it going to be a Perl module on CPAN? Is it going to be a JavaBean? If two different stakeholders want it to be two different things, who wins? Is it going to be a COM server? Is it going to integrate with or replace legacies? Etc, etc. What you're doing here is all about assuring yourself you understand the problem you actually need to solve to actually make the stakeholders happy when the project finishes.

Once you have the Vision of the Totally Integrated Web-Based Universal Billing System, how do you generate enough User Stories to know what Scope is, and to enter the XP loop? (Analysis)

I think this is not really what I'm asking. The activity is not generating user stories. The activity is synthesizing a concrete problem statement from UserStories. The UserStories are input - if you like they could be output too - but it's their synthesis that is what we're approaching here.

If these are it, I believe XP addresses neither of them. I may discuss later how I address them. Especially if the guy with the Doom WAD wants to locate the company in Kauai or Turks and Caicos. -- RonJeffries

It would really be enough for me if you just said, "we get a good Vision in our heads by collecting the initial UserStories. In our experience you don't need to do any more than that. Scoping is just iterative listening." Something like that's really what I thought you'd say - it might not be enough, but it would fit XP. I'm genuinely surprised you say that XP doesn't address the activity at all because that seems to conflict with what Kent said. This makes me think we may be talking at cross purposes again. We could take this show on the road: WhosOnFirst?

ExtremeBookSeries??

"enter the XP loop"

That's the crux of it, isn't it?

Ron, could you provide us with a brief rundown of the timeframes that occurred in C3 (or refer me to a page that already covers it)? How much time was spent on initial User Stories, Spike, etc.? -- KielHodges

Most of the stories were done in the month between deciding to can the original version, and beginning the XP version. We had something like 120 stories for our initial CommitmentSchedule, which came out to 14 iterations. Some cards were added, but there were at least placeholder stories for everything important. We estimated from that, quite accurately as it turned out.

It's worth noting that this was a recovery of a failed project, so the users had very good domain and spec knowledge ready to go. It was largely a matter of showing them what a card should look like and their transcribing what they already knew onto cards. A completely new out of the box project might need more time to get stories.

We did a one-week Spike of the whole system, before the CommitmentSchedule, that actually experimented with most major sections of the architecture and paid a guy. This Spike went nearly end (legacy input) to end (legacy output). Everything was at least stubbed in.

We made a "complete" end-to-end system at the very beginning and grew it from then on. From Day One the system was paying people, inadequately but credibly. We were just adding features thereafter. Still are. -- RonJeffries

I'd like to get back to Peter's initial question, which was "How does XP do scoping?" By this, I mean deciding what is the SmallestSystemThatHelps and of those optional items, which should be implemented first.

Peter, you said, "... this is delegating the Scoping activity to someone else." This is absolutely crucial to XP. One of the ExtremeTautologies is that business people make business decisions and technical people make technical decisions. Violating this rule kills projects, but it happens all the time. Business scopes the system.

As you noted, they can't do it in a vacuum. Business has to express the scope in terms of stories. Development gets to assign estimates to those stories. Developments gets to choose what sum of estimates to commit to in a given calendar interval. Business can't be pie-in-the-sky. Business has to be concrete. Business has to make trade-offs.

Peter, you said, "Scoping is taking the UserStory(s) and filling in the blanks to build a consistent, bounded Vision, taking that Vision back to the stakeholders." This is precisely the process that I want to avoid. I got the idea from the early Alexander, that people got to make the decisions that affected their lives, but in a safe context provided by an expert. This is a completely different model than the "interview" model, where you ask some questions, go up the mountain, and bring down "the answer".

Peter, you said, "Scoping is something you want to keep doing throughout the project." You are dead right on this one. The thing that surprised me about CommitmentSchedule and to an even greater degree WorkQueue is the amount of change they can absorb without the result feeling like "a different project". I was also shocked at how much change Business would tolerate. I thought when you signed a contract to produce X, you by god produced X or you would get your ass sued. My experience of business is that a contract to produce X is really a formalization of a relationship to produce something at least as valuable as X, and that Business expects X to shift, both from business and technical pressures. If you keep the relationship alive, the difference between X and Y isn't going to get you sued. In fact, it may get you the contract to produce Z.

In summary: XP does scoping, but the responsibility is primarily on Business within a given set of rules and with estimates and limits provided by Development. -- KentBeck

Thanks Kent, that's what I was trying to get at. This change to the relationship between business and development is a healthy way to do things. But I think it requires competent, hands-on technical management. I'm curious about your managers' experience of this; did C3 management find it easy to take on this scoping thing? Apart from the domain-specifics, did they find they needed to use any tools or techniques that were novel or difficult for them? -- PeterMerel

So far the XP articles that I have seen have focused on work for hire. How do you (Kent, Ron) work when you are developing an off-the-shelf product (e.g. Kent's Smalltalk performance profiler, which I used to use, or Object Explorer)? -- Bob Haugen

I suppose you mean a product of our own, something to sell and create an annuity for our old age. I've done that, but I haven't ever done it Extreme. Here's what I've learned, and what I'd try to do now.

The biggest thing [I/you] need in this situation is someone to sell the product. XP doesn't address this, but you need someone to knock on doors, make phone calls, sell it. Nothing sells enough on its own to keep you alive (maybe excluding games, and I have serious doubts there).

That said ...

I'd try hard to find a technical partner as well as a sales partner. Keeps one honest, gets one out of the holes.

I'd make UserStory cards for everything I saw, and would give them user priorities HML and risk ratings HML. The main reason I'd do that would be to force myself to take all the ideas I'd get along the way and put them back into the overall context of the stories.

I'd select out a first subset to Spike (presuming that I didn't already have the rudiments done before this all started, as Kent probably did for the profiler (just guessing of course)). Spike it, see what I learned.

I'd select out a second subset that was just enough to make a viable first release, and do it. Release, get it to the market, get customer feedback.

With analyst hat on, convert customer feedback to Stories.

Iterate until done. Keep it simple, simple, simple, but ... new releases as fast as the market could stand it, probably monthly at first until users started complaining about too many.

All that said ... products fail, IMO, because they aren't sold, not because they aren't good. The world will not beat a path to your door. It's a bummer, but that's the way of it. -- RonJeffries

There's a trend right now to build HUGE products. The essence of doing a product on your own is to go to the core of the matter, not to build something huge. You need multiplier. So you have to stay simple. For an example, look at the credits for a game like Doom, Quake, Unreal. There are LOTS of people involved, not just the engine people. And they take LOTS of time - multiple calendar years. I don't think two guys in a garage are going to make a killing with a killer game. But two guys in a garage will make a killing doing something we don't expect. -- RonJeffries