We Will Try

: Of all the words of tongue or pen, the saddest are these... it might have been. --John Greenleaf Whittier
: More sad are those we daily see: it is, and hadn't ought to be. --unknown
: Do, or do not. There is no try. --Yoda
: Stop trying to hit me and hit me! -- Morpheus to Neo in TheMatrix
: There's nothing you can do that can't be done. -- JohnLennon

"We'll try."

These are the saddest words a programmer has ever spoken, and most of us have spoken them more than once. These words are often the preface to months of grueling effort against a deadline we know in our heart we cannot make. At the end, we come up tired, burnt out, beaten, and short. Management hates us, we hate ourselves, our families don't know us any more or have fallen by the wayside. The software, if it works at all, is nothing to be proud of.

Oh, there have been exceptions. Successful products have been launched this way, and there is a certain pride in having gone through hell and survived. We have to believe it was worth it, if the alternative is to believe we wasted a big chunk of our lives.

There has to be a better way. Here's one that couldn't really happen.

Suppose you knew everything they were asking for, and suppose you knew how long it would take your team to do every one of the things they were asking for.

Suppose you knew that, and you weren't afraid of the truth. Suppose you wrote down everything they were asking for, maybe on little cards, and you went in to them and laid your cards on the table.

Suppose you said: Here's everything you have asked for, and on each one I've put down how long it is going to take to get done. I've broken them down into three-week periods, and in each period I've put as many cards as will get done in that period. As you can see, it will take 14 periods to do all this.

You lay a pencil vertically between two of the columns, and say: Here's the date we want to deliver. We have too much to do. Our job now is to put the cards we want the most on the left side of the pencil. When we put a card over there, we have to remove a card with the same number on it.

They rant. They rave. They call you names. Secure in your perfect knowledge, you say, This is how long each of these things will take. To get the best product by your date, we need to put the cards we want most on the left side of the pencil, removing cards with the same number.

They threaten your job. You say, This is how long it will take our current team to do it. Maybe if you fire us, you can find, recruit, hire, and train a team that will get it done sooner.

Bask for a moment in how calm, how strong, how totally cool and heroic you would be, because you KnowHowLongItWillTake.

But that couldn't happen, could it?

--RonJeffries

DeathMarch is about similar situations, but Ron is more eloquent.

Ron - this is one of the most poignant vignettes on the state of software development that I have ever read - it ranks up there with passages from DeathMarch and the opening and closing of ExtremeProgrammingExplainedEmbraceChange. But it also hinges on one of the aspects that most mystifies me about ExtremeProgramming - estimating - which you've correctly identified, IMHO, as being at the crux of the matter (assuming the business culture cares about its people and doesn't just burn through them to get near-term financial gain [Deming, anyone? long term cost is inversely proportional to quality]).

I suspect a lot of people mistake BigAnalysisUpFront as BigDesignUpFront. Your soliliquoy is clearly based on being able to play the PlanningGame well. That assumes you can estimate sufficiently well, which assumes you've developed requirements (UserStories / UseCases / features / what have you) for a release, or series thereof, in sufficient detail to estimate effort. But even still, estimation is a hard business. Technology keeps on changing. Team composition keeps on changing. Process keeps on changing. How do you know you've identified and accounted for all the elusive little tasks? There are always development process overhead aspects like CM environment setup/maintenance, data environment setup/maintenance, technology research, porting to new vendor releases, mentoring, technical support, preparing documentation, etc.

Meanwhile, we keep our heads down in the pig bin, saying "keep on digging", as Pink Floyd puts it. I suppose it's inevitable that some programmers start thinking about organized labor viz. JustStopDoingIt.

What is JustaProgrammer to do? --RandyStafford (06/12/00)

See also: news://edemo.gemstone.com/38C885BF.3CB7EE23%40gemstone.com for more of the same

Technology keeps on changing. Team composition keeps on changing. Process keeps on changing.

Yes, but they don't change so rapidly that things take different amounts of time. When you get a new technology, you have to play with it before estimating on it. Anything else would be, um, unwise. But read on ...

How do you know you've identified and accounted for all the elusive little tasks? There are always development process overhead aspects like CM environment setup/maintenance, data environment setup/maintenance, technology research, porting to new vendor releases, mentoring, technical support, preparing documentation, etc.

You know you haven't identified and accounted for them in your estimates. You estimate stories. You measure velocity - your rate of producing stories - which takes into account all the overhead, including those maintenance tasks. Does every hour of actually coding a bit of function include another hour of s/m? Two things will happen - you'll have velocity 1/2 due to that, and you'll learn how to improve those estimates, because you're paying attention.

That said, it's important to keep overhead low. As long as it's "each new feature includes an hour of database migration", that's fine, you can add it to your estimates with impunity. But if for every feature you implement, you spend 10x the time in other activities, something is wrong. Often, the stories are wrong. If what you're doing provides business value, then it belongs to the customer and the customer should be giving you stories to do it, which are then under the umbrella. If it doesn't provide business value ... ;->

Management doesn't want to hear the truth. They just want to get as much effort as possible from the developers. (Well, then they should be happy: the software will be done when it's done, though treating the developers badly might get less output than treating them well.)

hmmm See SweatMore'

But... this is American management. Other countries behave differently.

-- EricUlevik

How does management in other countries behave differently? Examples?

-- DavidHurt

Japanese companies listen to their engineers.

We could have a long chat about that one :-)--ChanningWalton

Australian companies wouldn't threaten your job.

The whole situation seems much too confrontational for a British company.

I've worked for Australian companies that would threaten my job (not my current employer, thankfully). I took the last one up on the offer. -- RobertWatkins

Loved the main text here, Ron. A minor aside to the secondary commentary: I understand that some people really have to stay in crummy jobs with crummy companies doing crummy work for crummy pay. For those of you who are really in that situation, you have my condolences. The following sentence is for the rest of the folks:

If you hate your programming job, find another one.

Seriously. You really can do it. Have you looked at salaries lately? Do you know why they're so very high? Companies out there are desperate for talented developers with decent skills and decent work-ethic. Go find one. You really can do it. -- MichaelHill

So can we explain how to go about QualifyingEmployers?

In the U.S. in 1993, when the Clinton administration proposed a national system of guaranteed health care, a survey was done and 13% of respondents said that the OnlyReasonToStayAtTheJob? was that they needed the HealthBenefits?. For non-U.S. readers, typically that means they or someone in their family has a health condition that would not receive coverage under a new employer's medical benefits.

Never NegotiateEstimates.

My experience of software development in Britain leads me to conclude that the following process is clandestinely taught to people with a leaning towards managing software projects:

Someone comes up with a vague notion that it would be a good idea if something was done. From now on it goes downhill.
A load of middle managers agree that this is an excellent idea.
They ask someone how long this will take. Ideally this person was last actually involved in development a sufficiently long time ago - albeit using the latest technology at the time - that they can now be considered an expert.
This person makes an estimate (it must be admitted that this is often not too far out). Credit due.
The middle managers then get out a calendar and work out if it was started now, when would it be finished.
They think of other projects that could be done if this one is completed by this time (this of course kicks off the process for these other projects). This is pure empire building of course.
They take three months to come up with a list of other projects and another month to think about who they can shaft in order to further their own ends. In the process of producing this list, more stuff must be added to the original product in order to enable it to fit into the new scheme. The completion date stands (because the expert said it could be done by that time).
They get some business analysts to start writing specs. It's at this time that more features are added. The original completion date still stands (because the expert said it could be done by that time and these extra bits are judged (by the middle management) to require very little effort ).
They put together a development team who have to start coding against incomplete specs. As the development starts the developers find significant problems in the specs. The business analysts start complaining that the development team keeps changing their specs and are hence making it take longer than it should. (They have sign off dates you know).
The development team starts getting nervous that this is not going to get finished on time and that they will be blamed. This is passed on to their manager. They are told that it must be finished on time because there are dependencies (the fact that these dependent projects are also showing signs of stress is not relevant - they have dependencies too). The evidence of slippage goes no further. This item will be repeated many times.
The development team starts to feel lethargic and bored because they know things are going wrong. This exacerbates the slippage.
The project slips. Some of the middle managers have the sense to move to a different project.
Once the project is finally delivered, the developers are blamed for the slippage. Those managers that left declare that it was going fine while they were there.
The developers move onto a new project with the words 'This project was completed on time and to the satisfaction of the client' on their CVs.

Having said all that, I still love software development and respect everybody in it. Just the group behaviour is demented, not the individuals (mostly).

--LanceWalton

Nicely said, Ron. I have spoken those same words, knowing as I said them that this was probably the project that would end my tenure.

It sucks to be afraid. -- GarryHamilton

CategoryQuote