Levels Of Software Success

There are increasingly broad ways to judge the success of a software project.

Works well enough to be used.
Can be maintained and improved.
Does what the stakeholders want.
Provides value to the organization.
Provides more value than it cost.
Provides more value than any other use of its resources could have.

Success at each level trumps failure at the previous level, so an unmaintainable piece of software that isn't usable but still provides value to the organization has achieved level 4. An example of this is a piece of demo-ware that's used to get a round of funding and thrown away afterwards. For most software, though, success at lower levels is necessary to achieve success at higher levels.

For a project to be minimally successful, it has to achieve level 5. Developers doing ExtremeProgramming focus on levels 1, 2, and 3. They count on the customer to attend to levels 4, 5 and 6.

Other methodologies focus on level 3, too. In fact, they didn't even say much about levels 1 and 2. They count on the programmers and the stakeholders to achieve the other levels. And we all know how that turns out: lots of projects that don't even function, or can't be maintained.

The neat thing about XP was that it brought levels 1 and 2 into the equation. With XP, software is actually getting built and is maintainable. But the picture won't be complete until we bring levels 4 and 5 in as well. If we truly want to be excellent, we need to include level 6.

-- JimShore

Extreme Programming is what it says, a programming methodology. Success above 3 is beyond the scope of programming. In particular, with no offense intended to customers, it is not necessary to attain #3 before attaining #4. The stakeholders may be mistaken in what constitutes "providing value", and it is faintly possible that the programmers may override the putative stakeholder and provide more real value then what the stakeholder literally wanted would have. (It might help to imagine that the "stakeholder" is a person assigned to the XP team because they are the least valuable and most easily missed, and not because they work well with programmers.)

No programming methodology can attain 4, 5, or 6, and no programming methodology can do more then encourage 4. Those are for management methodologies. I would not try to define "success" for programming methodologies in terms of 4, 5, or 6.

"Stakeholder" is not a euphamism for "on-site customer." It means, literally, "person with a stake in the project's success," and most projects have many of them. But your point is well taken and I've modified the text. I disagree with your other point, though. I'm interested in success at level 6, and if a methodology doesn't help me get me all the way there, it doesn't meet all my needs. --JimShore

This, and some related pages, are implicitly assuming a stance that corresponds to game theory involving rational participants, which is not too controversial aside from PointyHairedBoss kinds of issues, but it also assumes that rationality is concerned purely with issues of software cost and software utility, which is possibly incorrect more often than it is correct, in the real world of business.

Classic example: a software project must be done simply as a cost of doing business at all, even though expected return is negative. For instance, at one time spell checkers were independent software products that yielded profits for the companies that wrote them. But then they became incorporated into word processors, and it became a "checklist item": word processors lacking spell checkers simply would not be bought at all.

Therefore all word processor companies who didn't have spell checkers had to scramble to buy them or develop them, despite the fact that, as a profit center, there would only be negative yield from those projects.

On the other hand, if the cost of not doing something is greater than the cost of doing it, then the return is positive, is it not? If the choice is spending a million bucks to buy a spelling checker or losing two million in profits due to lost sales; to me that sounds like the yield from buying the spelling checker is +1 million dollars. (Numbers pulled out of the air).

Far too many analyses of projects; in particular go/no-go decisions, assume that the CostOfDoingNothing is zero. Often times, doing nothing is expensive; though it is often hard to calculate the CostOfDoingNothing.

Well, yes and no. If you're the poor schmuck managing the individual project, it is a rare corporation that will be enlightened enough to credit you with that million bucks. More often it's a black mark on your record, despite the argument that you are indirectly responsible for 100% of your company's following sales. Nor is the company doing its accounting incorrectly, for that matter; companies do not (and should not) credit receptionists, janitors, and similar support staff for that million dollars in revenue, even though the infrastructure staff is essential to do business at all. Same thing with the checklist item software project.

This is an example of something that violated "Provides more value than it cost" and potentially other rules above as well, yet was necessary to the business (so it did satisfy "Provides value to the organization").

Similar things happen purely within an organization, when vice president A has some political agenda that requires project X to crank out some software, which needn't even be shipped, let alone make a profit, and therefore sometimes needn't work, violating rule "Works well enough to be used". And yet the project can still be technically successful by some measure (as the poor saps doing the programming scramble to pursue a self-contradictory moving target), and garner kudos within the company, with no one ever unhappy about it (the stockholders never realize what was going on, of course).

So all of this discussion about SoftwareSuccess seems myopic.

"Value" has many facets and it doesn't necessarily mean "profit." As you said above, if something is necessary to the business, it provides value to the organization. Similarly, the vice president of your example sees value in a political advantage. In his mind, the value of software to achieve the political goal exceeds the cost of developing it.

Value and cost aren't necessarily measured in dollars, but someone, somewhere, is making a value/cost judgement call when they commission the software. Their judgement may be poorly considered, but someone had to think the value exceeded the cost, if only on an emotional level. If we want our work to be successful, we need to deliver something that actually does have value exceeding cost... even if only on an emotional level.

Many of the factors contributing to success are out of our direct control. Such is life. I try not to take on projects that can't be successful, and I always work to expose issues early so we can find out if a project can be successful. I don't always succeed, but if success was easy, everyone would be successful. --JimShore

Very true. However, when you widen the definitions of value/utility and cost like that, notice that you also widen them to the point where programming methodologies partially or wholly break down. It's hard enough to find useful methodologies just for the direct aspects of software development, without having to take into account the entire span of business practice (and mispractice).

I'm not sure I agree. I think ExtremeProgramming plus IncrementalFundingMethodology plus IndustrialExtremeProgramming could get us to level six reliably. (Or early cancellation: EarlyCancellationIsSuccess.) Sure, we've got to do a lot of stuff correctly, but that's true of programming, too. It's hard but if people believe it's important it will happen. --JimShore

CategorySuccess