Estimation Woes
Estimation WoesIs schedule estimation harder than implementation?
Estimation is really hard. Sometimes you just want to get in and do it, partly because the estimate has a much better chance of being accurate when you've done it before.
PlanToThrowOneAway is a great way of getting a more successful second iteration.
Estimates should not be believed but the process provides the value. -- DavidLiu
The team on one old project fell into a pattern of using a stock answer to the ever present "How long will this take" questions. The answer was a quote from a popular TV show in Australia at the time, Comedy Company. One of its characters (Con the fruiterer) was a rotund greengrocer whose produce was always Beauuuuutiful, at least for a CuplaDays. He had a way of answering just about any question at all with "CuplaDays", but most of his questions were to do with how long some piece of produce may take to go bad.
It all seemed fair and reasonable, but management would go nuts not knowing if the feature would take two hours or two weeks. and rightly so.
Management often has trouble dealing with reality. That's fair enough, as most people do. Reality is a very hard thing to deal with.
I worked for a manager (of about 100 people) at a multi-national company whose stock answer was "six man months." He said, "if I haven't the foggiest idea how long it will take to do something, I say 'six man months.'" -- JeffGrigg
Some people, understandably, want to know when something will be done. When developing a complex system, the answer will have an uncertainty associated with it, but that shouldn't mean that no answer can be given. A good project manager will be able to explain this to a good customer and provide an increasingly accurate answer over the life of the project. Unfortunately, software like MicrosoftProject, clever as it is, doesn't easily allow for uncertainty in project plans, and then unreasonable customers demand Gantt charts with the delivery date down to the day, and give you lots of grief for not meeting it. A product like Pertmaster does allow a probability to be associated with a milestone. Whether a customer will be happy with this depends on their position. -- AndrewJoyner
From ManagersViewsOfDevelopers:
Gut feel is based on prior experience. See YesterdaysWeather.
If you don't like the estimates, why do you keep asking for them?
You get what you pay for: It's rare for a project manager to be willing to pay anything to improve the estimating process. But they keep (often "suddenly" and "unexpectedly") finding themselves in need of estimates. So until someone can see the pattern, and develop a willingness to do something about estimates (other than complain), nothing will happen.
Developers are not plug-interchangeable parts. Have you considered asking those three different developers to clarify some of the forces behind the estimates? Ask me how long I'll take to build your web application and I'll say one thing, ask one of my co-workers who just finished the introductory Java class how long she will take and you'll get very a different answer. Which one is the "right" estimate? On the other hand, don't ask me to estimate how long it would take me to build a GUI application in VB - if you do, your managerial incompetence exceeds my VB incompetence.
In order to have quite good estimates you need:
I have been asked many times for an estimate, and after giving one, get told, "Well, we need it by Friday", when Friday is at most half of the time I estimated. If managers have already established a deadline, why am I being asked for an estimate for completion?
To see if it's feasible? I'd then expect a conversation along the lines of: "well, you can't have what you asked for by Friday - but we could do X or Y"
That's why you need a TaskCompleteDefinition. It won't solve the problem with vague estimates but at least your developers have a common viewpoint of what it means to finish a task (ie. TaskComplete). -- PeterAxelsson
This sounds very much like the complaint book publishers have about authors. Guess what? Authors can't estimate when the book will be done because it's intrinsically not an estimatable process. I believe software is like that too. When this "industry" stops pretending to itself that we do "engineering", they'll stop asking stupid questions.
I don't believe writing software is not estimatable. A lot of what's done is predictable based on how things have been done before, and isn't particularly innovative. What I do find takes unpredictable time is rework - investigating and fixing bugs, which is one reason to adopt a process that minimizes this (likeXP)
This misunderstands the problem. Developers are the better than anyone else at making estimates. (This manager should perhaps be advised to make their own estimates.) The problem is that managers are very uncomfortable with uncertainty. They like things to be known, and they like to have control. They like to be able to rely on a delivery or know who to blame. This is perhaps fine for managing street sweepers or accountants, but it is an unrealistic expectation of something as uncertain as development.
I don't agree. Developers (in my experience) are often hopelessly optimistic, and estimate for only the time spent coding (not for thinking, investigating the problem, setting up test environments, completing documentation etc...(see the comment about TaskComplete above). They don't look back at their estimates to see how they did, and so they don't improve.
SweepingGeneralization, of course, but it's been true enough in the past.
This is true of all people, not just developers. This is known as Planning Fallacy in psychology. People who are asked to estimate any task produce an overly optimistic response. This does not improve with experience. Managers need to add a chunk of time onto any estimates they get, especially in development where there are a wealth of details waiting to be missed when providing snap estimates.
Yes, managers often then play games with the estimates, but the raw data is pretty dodgy too.
The problem is that managers are very uncomfortable with uncertainty.
Sure. Maybe they work for an organization that makes money and has customers, so they want to know about the time taken and the cost of everything. Is there something more natural ? By the way, developers too are very uncomfortable with uncertainty. But from the fact some developers have been caught taking estimates not seriously enough, we cannot infer anything about :
Of course it is not impossible to estimate. But it is impossible to estimate exactly. The manager who asks "How long will this take?" will not like the answer "Between 2 hours and 2 weeks, and there a 1% chance it will turn out to be practically impossible." Instead they demand a Wrong Answer, such as 2 to 3 days, which becomes a deadline. If I miss the deadline, I look like a bad developer, rather than someone who gave a second-best estimate under duress.
Somebody has to manage this uncertainty, and somebody has to make a decision whether it is worth trying the task, and/or what to charge for it. If the manager wants to make these decisions, he or she should demand something better than a point estimate of duration.
Exactly. But 2 hours to two weeks isn't an acceptable one of those
If 2-hours to 2-weeks is truly the most accurate estimate possible, that should be the final word. Whether that estimate is "acceptable" to the manager is irrelevant. Whether that information is useful you, whether you can build a higher level plan from that information, is irrelevant. You can't squeeze blood from a turnip. But if you demand better than the best estimate possible, then you'll get exactly what you ask for: less accuracy in the form of a wild-ass guess that will be wrong as often as right. "It will take two hours." I'll pretend not to know that it might take up to two weeks - yet I do know this - since you won't accept the true range.
So many problems in life come from people believing that anything is possible. Some things are not possible. If I believe hard enough, a jelly donut will not appear on my desk right now. If I believe hard enough, certainty will not appear where there is none. I will always make my best estimate, and I can make the estimate better than anyone else, but I cannot create certainty where there is none. Whether I deliver that true estimate to you, or invent some fantasy estimate that you'll accept, depends on your attitude.
Of course. However, I can think of few reasons that any task would have a real variation of two hours to two weeks, so my first impression would be that the developer is not giving a true estimate (and is probably taking the michael). I can think of tasks for which it is not yet known if it is a two hour task or a two week task (i.e "either", not "between") - with some uncertainty on both possibilities, of course, and a response like that is fine.
It's quite reasonable at the earliest stages of estimating for the level of confidence to vary from one-quarter to four times the actual. That corresponds to a range of 5 hours to 80 hours. Throw in the unknown variables - like how often you'll be called to a meeting to give an estimate and report status - a skilled manager will not be afraid of this range. -- StevenNewton
(to get to 40:1 - equivalent to 2 hours to 2 weeks - you need get to to 5 hours to 200 hours/5 weeks, since the interruptions won't bring the minimum down)
I don't think so. A 40:1 range? How much will this project cost me? Somewhere between $1000 and $40,000"''
No matter how skilled, a manager cannot do anything much in planning with estimates where the high end is 40 times the lower.
A skilled manager will want a tighter range. And should ask what it would take for the developer to gain a better idea of the effort. But it's very unlikely that an estimate with a 40:1 range is going to be useful
If the project is at a stage that asking for estimates is reasonable, a 40:1 range in the answer is not a reasonable estimate. If you're saying that a 40:1 range indicates the project has not progressed enough to permit reasonable estimates, then I'll agree with you.
In other words, I think 2 hours to 2 weeks is a way of saying "I don't know".
[I normally see (and have given) this sort of answer in response to something like "How long will it take you to fix Bug X?". This is a terrible question that a GoodItManager should rarely ask, because I don't KNOW how long it'll take to fix, because I don't know what's causing the bug. I have an idea, and if I'm right, it'll take a couple hours to make and test the fix. If I'm wrong, I'll have to start brainstorming other places and who knows when I'll find it. 2 weeks is a reasonable estimate for me to work top to bottom on a system.]
Not if you can also give a confidence level in there somewhere. Something like a 1% chance it will be done in 2 hours, a 90% chance it will be done in 2 weeks. Now, how confident do you need your estimate to be? 75%? OK, there's a 75% chance it will take an effort of 7 full workdays, or 56 hours, or development. You the manager and I the developer agree that on average I can only count on committing 4 hours out of 8 to this tasks. Result: there's a 75% chance it will be done in two calendar weeks. You simply cannot demand and expect better accuracy from a preliminary estimate. If the uncertainty that it will take too long and cost too much is higher than you can risk, then as a manager your responsibility is to forego the development until you can supply enough information or redefine you goal to get within your window of acceptable risk. -- StevenNewton
OK, I think we're saying the same thing now
Is it too much to expect a manager to ask "How much time and what information will you need to give a more accurate estimate?". I haven't yet met a manager who understands that making an accurate estimate takes resources.
Nope, it's not too much. I try to do this
The problem is that managers are very uncomfortable with uncertainty.
They might be, but good ones learn how to manage it. Management is about handling uncertainty, in many ways
The reluctance of managers to give people adequate resources to make good estimates is common. But I have worked at a company where management did make a good-faith effort to address that problem. They devoted a significant amount of money and allowed time to improve the process. The developers were allowed to define this new process, and managers promised to follow it. Developers and managers attended training courses where the new process was presented and discussed. Two separate attempts were made at this change in procedure over the course of a few years, and in both cases, most developers didn't follow the new process, and estimates did not improve. After both failures, developers generally blamed management for not putting stricter controls in place. This reinforced management's views that developers are unable to make estimates, no matter how much help they are given. (Note: I was a developer during these times, not a manager.) -- KrisJohnson
1) Effort = Time * People = Size * K, count the object classes or count the UserStories, or count the FunctionPoints.
2) It will take as long as you have time to work on it, so always be ready to ship something.
-- KentSchnaith
You can say, "I estimate it will take me one hour to tell you how long it will take me to study the problem long enough to give you a solid estimate. That might take two days or two weeks, and the actual job could be two months or two years, I don't know." You gotta Put Your Foot Down� when it comes to giving estimates. If you let management back you into a corner all the time you'll have nowhere to go.
Consider, however, that estimates form the basis to get approval to do work. Without approval to do work, you will really have nowhere to go. Frankly, estimating one hour to provide an estimate of how long it will take to provide an estimate doesn't cut and the value of the estimate decreases as the cost and time to produce it increases. In practice, I have usually found that the aggregate of quickly defined estimates to be at least as accurate as those produced by more time-consuming methods, and often more so.
"It will all be over by Christmas". Schedule estimation isn't always as easy as counting classes, user stories or function points because programming isn't always like Engineering. Sometimes it is more like being a police detective; no-one asks a detective "how long is it going to take to solve this murder?" Sometimes it's like playing a new video game: you may be able to complete Grand Theft Auto IV in 3 days, but it may take you three weeks depending on how tricky some of the levels are (and you won't know how tricky they are until you play them). Sometimes it's like buying a birthday present for your mother-in-law -- you may find the perfect gift in the first shop you come to, or it may take you days or weeks. Sometimes it is even like eating your dinner -- you may be able to wolf it down in a few seconds, or it may be too hot / chewy / difficult to cut, or you may be too full and find it difficult to finish, or it might taste disgusting. Sometimes, you just don't know.
Isn't estimation HaltingEquivalent? The 'problem' being solved is "what code implements the set of requirements?". The computer 'running' the 'algorithm' to solve the problem is a team of developers, doing development. The HaltingProblem is undecidable in the general case, although some cases are decidable - e.g. the time taken to clean up 20 source code files, with only 1 hour allowed to be spent on each file, is obviously 20 hours, not considering interruptions. However, if the developers have been doing their jobs correctly, all of the easily predictable parts of the problems should already be automated, leaving only the inestimable parts. If estimation is equivalent to the HaltingProblem, then it is not decidable. Even if we restrict it to the K-bounded version of the HaltingProblem, i.e. "Can you get this done by Friday?", it is still non-deterministic, which is of no use to estimators. At the end of the day, it's all wild guesses, and our time would be better spent elsewhere. --MikeAmy?
See: GopherHoles, SchedulingMyths
EditText of this page
(last edited September 13, 2010)
or FindPage with title or text search