Is Earlier Cancellation Failure

From AnAcceptableWayOfFailing:

JohnFavaro has predicted that more XP projects will be cancelled earlier, because you have so much early high-quality feedback about progress and value. How can we cast this in a positive light? Perhaps we call them "explorations" for the first six months, even though to us we treat them like regular development? What do real businesses do when they decide not to make an investment? Is that considered failure? -- KentBeck

IsEarlierCancellationFailure? Some say EarlyCancellationIsSuccess.

The guys that applied option pricing theory to large complex capital projects in the late eighties/early nineties began to come up with some answers to this. A lot of this work was done for the oil industry, so I'll illustrate from there.

One way of illustrating the problem is that the simple "discounted cash flow" view of the future (which ignores risk and uncertainty) never says "build a pilot oil refinery" or "drill a few more holes". It says do the whole project or nothing. Yet real businessmen know that these intermediate decisions are often the right thing to do.

Applying option pricing models helped economic theory to catch up with reality here. In doing so phrases like "learning value", "information value" and "option value" were coined to explain what the pricing formulas were saying. In other words, as common sense tells you, if in six months, having spent a fraction of the budget, you know enough to know whether or not to build the main refinery or dig the full oil well, then because that information is very valuable the interim step is easily cost justified.

Most software projects are less well understood in advance than large capital projects in the oil industry. (And please don't let's return to WaterFall just to make our option pricing models more convincing, software is more flexible than a refinery - let's use that one fact in our favour for goodness sake!) But the analogy is entirely correct in most situations and says: taking the steps that enable you to know that you should not continue with the project sooner rather than later is true professionalism and helps lead to success (albeit elsewhere where the risks are better).

As Kent implies, the key word here is cancellation. It subtly carries with it all the fallacies of the WaterFall view. Exploration is certainly better, learning value and information value often work well even if people don't know about option theory. Then we try to crystallize agreement in one or more SuccessStatement's that say that knowing how to proceed in six months will be judged as fully successful (by all users) at that point in time.

Because no, it should not be considered failure to have more money available from your annual budget to spend on commercially fruitful activities in the second half of the financial year because of your excellent choices in the first half.

-- RichardDrake

I would agree that early cancellation should not be considered failure; however, I saw the exact same concerns when my previous employer tried to promote another methodology: with better understanding of project prospects, early cancellation was a real possibility.

The problem arises, I believe, from some AntiPatterns. I cannot think of names for this, so I will just describe them.

See XpAndAnnualBudgets

There seems to be a number of reasons for bring a project to a halt before the point originally planned.

Failure. It's not going anywhere much, and what we're getting isn't worth what we're spending. If we want the benefits we set out to achieve, we're going to start over. This is the one I think Cancellation with its connotations of "didn't happen/won't happen" should be reserved for.

Unexpected success. What we have now is enough. It's not perhaps as far as we expected to reach, or the goals have moved but no-one cares. It works.

Partial success. We've done some useful things. Given what we know now, it's clear that continuing towards the original (or modified) goals isn't worth it.

Lessons learnt. This was an extended SpikeSolution. We knew that we didn't know enough to know where we were going or how we were going. We know now, and it turns out not to be worth it.

So, whether a project is cancelled depends on what it was for initially. If you want to achieve "Lessons learnt", you need to set that as a goal initially. Or it will just appear a Failure to everyone else. If something needs to be done to achieve the goals because this project did not achieve them, but was expected to, then that's a failure.

-- PaulHudson

If, at the end of a prototyping session, the client decides not to build a system, the prototype has been a success; it's given the client sufficient information to make an informed decision.

Absolutely! Without regard to the XP methodology, when the inception phase activities show the project is bigger than originally thought, or the benefits less than thought, this is not failure, this is success. Stopping when you should, assuming you can, is always a positive. Arguably the wise development manager explicitly recognizes and plans for ProjectCancellationJunctures throughout the project. That said, unplanned and disorderly cancellation is a failure of sorts. -- StevenBlack

moved here from TerminationCanBeSuccess

From AnotherXpExperiment

: Anyway: that project is now ended. The client eventually did absolutely the right thing and reduced scope! We were astounded. After wrapping up a few loose ends and preparing a list of to-dos for whomever picks up this work later we were out the door with smiles, handshakes and free drinks all round.

Brilliant. That sounds like a great customer. What do you think made them a good customer?

They took us at our word: When we produced estimates for the outstanding work that were unpalatable, they believed us. Of course, we had previously demonstrated our trustworthyness by delivering one iteration bang on time, reporting progress honestly, etc. etc.
They were prepared to separate business and technical decisions: And were prepared to let us take the latter, and were prepared themselves to take the former and stand by them. Which is worse, a client that dabbles in technical issues or one that leans on developers for business decisions?

Of course, there were many, many problems with this client, but they were pretty much mitigated by the above.

Note that we weren't doing XP, but were using some of the practices.

I HaveThisPattern... At my previous employer, which had a habit of starting things but not finishing, we managed to get a project drowned at birth. Why is this good? Because we were able to say reasonably accurately how much it would cost and that cost was more than any expected revenue. -- SteveFreeman

That's good to hear, Steve. At my ex-employer there was a project that should have been killed off several months before this page was created, when we did an estimating exercise that showed exactly the same cost/benefit relationship you mention. It staggered on, however, haemmoraging cash and staff, since no manager was prepared to bite the bullet. Eventually, it took a major political upheaval at the client's end to kill the thing. That project was a classic case of termination being failure. -- KeithBraithwaite

This kind of experience could be a partial answer to the question IsEarlierCancellationFailure?

In my experience, the reason why projects in (typically large) organizations roll on and on until some external influence puts them out of their misery is that Project Manager success is often measured by whether or not the project delivers some software. If their success was measured by "bringing the project to the most appropriate conclusion" - which may be early closure due to an unflattering cost benefit case - then I think we would see more bad projects being killed earlier -- MattStephenson?

What does this suggest, if anything, about XpAndSunkCosts??

What's your perspective on the "too many customers" problem - conflicts between the many different interested parties in a large organization (as I gather this was)?

We were building infrastructure that would support enhanced versions of their internal and external Web sites. As such we were, from the point of view of other managers within the client, buried way down in some IT oubliette, so we avoided being pulled in two many directions politically. On the other hand, every business unit that felt like it might want to get its services onto the site some time felt at liberty to toss in requirements. "Our" managers couldn't stop that, being a service group themselves, but they did halt the whole project as described above.

So, in this case the GoalDonors were all over the place, but the GoldOwners took charge, and this turned out to be the right thing.

"Which is worse, a client that dabbles in technical issues or one that leans on developers for business decisions?"

Both can be disastrous, for sure. But some of the best results that I've seen from software projects have arisen from "hybrid" people in the heart of development who understand both the business and the technology and get involved in both kinds of decision. I even have some sparkling anecdotes on the subject, some of which are printable.

Do other Wiki people have positive experience of hybrid developers? -- RichardDrake

A bit like asking if you prefer your pain in the neck or a pain in the --- ... other parts. My experience, for what its worth, suggests to me its more difficult to find and employ a skillful, experienced business analyst with appropriate tech skills then it is to find a tech wiz (meant as a compliment) who also comprehends the essential business issues.

In answer to Richard Drake's question, yes I have some "positive experience of hybrid developers" and I also have some negative experiences with the same breed. The negative experience was a result (at least in part) of the individual's failure to listen. The individual in question already knew it all. -- DaveSteffe

So far, I have not seen a positive effect, although I tend to see the business types with technical dabblings - they love to specify the technical solution in their problem report, but rarely include enough info to tell if you've actually solved the problem -- PeteHardie

BarryBoehm has published an article on "Project Termination Doesn't Equal Project Failure" in 2001: http://www.cs.unc.edu/~welch/class/comp145/media/docs/Boehm_Term_NE_Fail.pdf

"It can take some adjustment to realize that terminating projects can be natural and even healthy."

If you go into a project expecting it may fail, for whatever reason, then cancellation is not failure. This is not common. Most of the time cancellation is failure because it means something you invested in doesn't give you a return. When my highly diversified mutual fund doesn't make me money it's a failure. When the small percentage of my portfolio that is in junk bonds loses money then that is disapointment.

But in the latter case, it's the cancellation which prevents you from loosing even more, isn't it? It's not the cancellation which is the failure, but starting the project with the assumption that it can't fail.

"If, at the end of a prototyping session, the client decides not to build a system, the prototype has been a success; it's given the client sufficient information to make an informed decision."

Of course, the information the client may have obtained is that you are simply incapable of completing the project in the way they want.

Indeed. But that's a very useful thing to know.

Of course, it is better to find out at the prototype stage, rather than later on, that something is seriously wrong, but for most projects, if it gets as far as the prototype stage and then fails, someone must have made a serious mistake earlier, either in the definition of the project or the selection of the development team. I dont think it's justifiable to call such an outcome a "success". A throw-away prototype is only a "success" if you really didn't know ahead of time where the project was going or if you really didnt know your team AND if you can actually take advantage of that information in a subsequent project.

See WhatIsSuccess

CategorySuccess