Ignorance Fear Pride Or Fraud

AnAcceptableWayOfFailing is a great example of a Wiki page where a very fruitful discussion has been kicked off but has, strictly speaking, gone "off topic" from the original thesis of the page. As one of the culprits but without daring to refactor while the interchange is still "hot and heavy" (see GoodWikiCitizen) I'd like to suggest that there are four basic motivations or reasons for WaterFall and to ask how people think that they interact in practice.

Alistair's suggestion at the start of AnAcceptableWayOfFailing is stressing (in line with much of my experience) fear based on ignorance. I don't believe though that it's possible to ignore deliberate, unethical commercial motives on the part of software service providers as another reason for the persistence of WaterFall. When we recognise the fear working with the fraud (with pride and ignorance on an "as needed" basis) we may know what we're really up against in changing the global culture of software development.

It may look hopeless - except that our ally is the person that pays.

Or have I missed something?

--RichardDrake

Not malice. ("Never attribute to malice what can be adequately explained by ignorance" - anon.)

I have encountered large, traditional commercial contract houses whose internal reward mechanism is hours billed to the client, measured in log(10) units. The principals are thereby content (or encouraged) to use inflated project management techniques (WaterFall being the star example). Since it is AnAcceptableWayOfFailing, and failing takes longer than success, there are more billable hours, ergo, more reward. Just perhaps, maybe, some sharp young cookies in development, by brilliance and overtime also measured in log(10) units, will actually cause a success. Then there is double reward - an advertising banner as well as the money.

: Surely this can't be right. One project at 200 hours generates 2.3 units of reward, while 2 projects at 100 hours each generates 2 + 2 = 4 units of reward. It seems this reward scheme should cause people to favour many short projects rather than one long one. -- ChrisBooth

In those organizations, they add up all hours billed at the end of the year. If you only had 200 hours billed, whether on one or two projects, you get 2 points. Your colleagues, however, billed 20,000 and 120,000 hours, for 5 and 6 points respectively. You get fired, the 5-pointer keeps his job, the 6-pointer is promoted and told to bring in 7 points of business next year. Suddenly a great big WaterFall project looks really attractive, and a 6-person XP project completely uninteresting.

You can imagine how difficult it is to introduce incremental, ED, XP, etc. into such a situation.

However, that having been said, my tour of duty indicates that there are a fingerload of those companies compared to a wagonload of the other situation... "I haven't done it, I don't know what it looks like, feels like, acts like. I'm nervous and have enough other worries. I already have AnAcceptableWayOfFailing for the project scheduling technique, whereas I haven't a clue about other parts of the project. Go away and leave me alone, I'm overloaded and I need to suffer through this situation in ways I understand."

Even counting in the big consulting house, I have not yet seen anyone propose WaterFall out of malice. Ignorance, fear, pride, and even some fraud, yes. --AlistairCockburn

They would tell you if it was out of malice? Seriously this time, the presence of a consultant the quality of Cockburn is likely to drive such motives (if they exist) out of view. --rd

I'm assuming enough natural social apparatus to detect when someone is both against a project and recommending it use WaterFall.

So that's what I've been missing: EnoughNaturalSocialApparatus

"It may look hopeless - except that our ally is the person that pays. --RichardDrake"

Sometimes the person who pays is not your ally. Sometimes the real goal is a beautiful vision of a project rather than delivered results. Visions don't crash, underperform, or otherwise disappoint.

Even worse, there may be competing factions within the customer organization. Some factions may want to kill the project or mangle it beyond recognition. WaterfallInertiaMayBeHelpful to those who want to continue the original project.

Can I humbly suggest that this was TooMuchCynicismTooEarly for this page? (...and if I said "no"?...) It's better down here thanks. My own specific interest was to explore how "fear" works with "fraud" across the whole industry to maintain the status quo. Although the incidence of fraud may be statistically small the amounts of money may not be - and from these few but strongly commercially motivated situations may come a strong aversion to admitting that most software can be approached a very different way.

I certainly believe that if we as responsible professionals can communicate that (whether through ignorance or deliberate choice) the customer is being consistently "ripped off" through WaterFall then we will have a much speedier revolution in adoption of evolutionary methods like XP. Hopefully a pretty bloodless one. --RichardDrake

Too much cynicism on a page which suggests that WaterFall development has four irrational (at best) reasons for support? And then suggests that purposeful fraud is a substantial reason for WaterFall? Perhaps a less shocking page name like WaterFallMotives? would be a better "place" to discuss less cynical ideas.

I have been extremely up front about my views on WaterFall since 1986 in the software industry at large and on Wiki, with a large number of signed contributions. The "Have I missed something" was intended as the cue for those that beg to differ. My nameless protagonist initially went down what I would normally consider (though it would be impolite to assume it here) the cowardly but nonetheless traditional WaterFall route of "blame the customer" rather than deal with the central issues of the reasons for the pervasive stupidity of WaterFall.

Having said that, there are some important issues to deal with in each sentence here. The simplest thing to say is that if the customer is really not your ally - find another customer. In the case of factions among the customer (situation normal in my experience) learn to practice ImpactModelling, but without the double "l" of you prefer.

The testimony of DaveSteffe as a fairly powerless end user in AnAcceptableWayOfFailing I found very compelling in 1990 and I continue to find some of my own personal observations and some of the stories end users tell difficult to explain without the fraud factor.

This author personally believes WaterFall is mostly supported by EngineeringEnvy--there is a widespread belief that programming should be like other top-down BigDesign disciplines. This belief is often supported by "software engineers". For large projects within established business procedures, alternatives to WaterFall are not always obvious.

I agree with this assessment pretty well. It comes under the ignorance category doesn't it?

It's interesting to see unsigned, ghostly replies on a page named IgnoranceFearPrideOrFraud. --Alistair

It's also interesting the effect of a initial, trenchant unsigned response to my page introduction - see the first block of italics now moved down to the third section (not by me but thanks). This seemed destined to yank the dicussion in a number of directions that I wasn't prepared for and hadn't really wanted to deal with on this page. Initially I just felt hopeless in trying to deal adequately with each issue raised, even if I had the time to create the other pages necessary. I've got over that but I think the resulting discussion has gone "off subject" just as much as AnAcceptableWayOfFailing (which I explained in the first paragraph was my original reason for this page!) No blame attached but a few more questions here on the nature of Wiki refactoring for that long lost race of WikiMaster's.

Meantime I'd like to give one example of how the problems we face with the waterfall model are NotJustIgnorance. --Richard

First, a confession. I have often been the advocate for the waterfall model on many past projects. In fact, for the right project, I wouldn't hesitate to recommend it again.

However, I will admit that I haven't always selected it because I thought it was ideal. I would have to agree that the motives which give this page its title all came into play at various times.

Early in my career, ignorance of other (effective) development models combined with a fear that my customer would find out I didn't know EXACTLY what I was doing sometimes lead me to jump on the waterfall bandwagon. In the worst cases, my pride "forced" me to select AnAcceptableWayOfFailing because it gave the customer the illusion that I knew what to do while I was, in fact, WingingIt?.

On the other hand, I don't think this version of fraud is nearly as nefarious as the one implied above (deliberately inflated project hours) or on NotJustIgnorance. I also refuse to believe that our industry is *primarily* driven by those motives when selecting methodologies.

In my opinion, it is still "early days" in software development and, eventually, each niche in the software development "ecology" will be filled by a suitable development methodology regardless of what forces have established the current arrangements. Thank God I get to participate in the active phase of the evolutionary cycle (some pun intended :-).

-JeffShelby

Thank God for an honest product of evolution - absolutely! ---

In government projects, the adoption of heavily formal methods of project management seems to be caused by fear on the part of the purchasing authority. They (civil servants) have been bitten many times before by projects that got out of hand. (I speak from a UK viewpoint.)

The reasons for this may have nothing to do with the contractor, and everything to do with poor specification followed by weak monitoring and willful ignorance of emerging problems, but the result is that the purchasers feel vulnerable. Their political masters demanded results and all they were able to provide was a very public failure.

The civil service then swings to the other extreme and demands detailed specifications and heavy controls. The system marketed here as PRINCE was devised to deal with this problem. The purchaser fears that there won't be a good result so a huge "project initiation document" is demanded.

And that leads, in my view, to a kind of fraud. The big management consulting firms press for PRINCE so that they can make a packet manufacturing convoluted specifications secure in the knowledge that their work will not be tested for its usefulness to the final customers (tax payers like you and me). But by claiming that they are looking after the tax payers' interests they have generated a wonderful new income stream for management consultants.

And yet, a recent report from a respected body in British government has said: "To our knowledge, there is no real properly documented evidence that the adoption of project management methods actually does lead to greater effectiveness in the delivery of projects." (This statement is made in a paper called "Project Management in Local Government" published only a few days ago by the local government employers association. The contact point is george.nahlis@lg-employers.gov.uk and summaries of this report can be downloaded at www.lg-employers.gov.uk.)

I think the non-cynical response to this trend is to accept that often public-sector projects have disappointed and try to re-build trust through small ventures, accepting (for the time being) the heavy burden of bureaucratic methods as a punishment for past mis-demeanours.

RobinWilson

Mostly agreed, but don't lay too much blame at the feet of PRINCE. We use it internally; it doesn't have to be bureaucratic (and the PID doesn't have to be huge).