Gopher Holes

A metaphor for the difficulty in estimating the effort required for a given software task.

Your boss, groundskeeper at a golf course, points to a hole about 50 yards away and says "I need you to fill that hole in. How long do you think it will take?" You look at it for a few seconds; small- to medium-sized, seems simple enough. "About an hour," you reply. You grab your shovel, walk over to the hole, and start filling it in absent-mindedly. After a few minutes you realize you don't see any of the the dirt you've been throwing in. You look into the hole and notice that there are depressions and pockets you didn't see before. "Okay, maybe an hour and a half." you think to yourself. Two hours later, you're still shovelling dirt into the hole, which is still disappearing. "What's going on here?" you ask yourself. You climb into the hole and test the bottom with your shovel, and it caves in to reveal several gopher tunnels, some leading to big pockets. You head back to tell your boss that your little one-hour task will take more like one day. In fact, you wonder how far the gopher holes go, and whether you might need a week!

In my 20-odd years in software development, I've run into this situation time and time again. At first new functionality or a bug-fix seems straightforward, but TheDevilIsInTheDetails: subtasks I didn't anticipate, exception or error conditions I didn't expect, bugs or quirks with a library or package I'm using, etc. I end up going back to my boss again and again explaining why the task I thought would be easy turned out to be more complicated than I thought.

Over the years I may or may not have gotten any better at estimating software effort, or anticipating these kinds of complications, but I definitely have learned to "pad" my estimates. At first I would think honestly about the task, breaking it into subtasks where possible, and thinking about the longest that each subtask should take. Then I would double this estimate (I tend to be too much the optimist). Now I quadruple my estimates. It usually doesn't take that long, but when I hit a "gopher hole" it blows the average and then need the extra time from the other tasks that went fine. And of course, it is far better to under-promise and over-deliver (e.g. finish ahead of time) than the reverse.

Possible tie-ins to EstimationWoes, TaskEstimationPatterns, IdealProgrammingTime, and CuplaDays. -- AndyMoore