Technical Debt

During the planning or execution of a software project, decisions are made to defer necessary work. For example:

It's too late in the LifeCycle? to upgrade to the new release of the compiler. We'll do it next time around.
We're not completely conforming to the UserInterface guidelines. We'll get to it next time.
We don't have time to uncruft (refactor, see RefactorMercilessly) the hyper-widget code. Punt until next time.

The list can grow quite long, with some items surviving across multiple development cycles. A big pile of deferred work can gum up a project, yet many of the items on the list don't appear on a project team's radar, especially if the focus is primarily on new product features. Yet removing accumulated sludge needs to be accounted for in planning!

Therefore: Make the debt visible. Keep an explicit TechnicalDebtList?. Group deferred tasks into workable units, note the consequences of leaving each unit unattended. Keep the list visible. Make sure that Marketing knows that the list exists, and repeat the mantra "If we don't schedule time to pay off TechnicalDebt, you might not get all of the new features that you want." Allow time on the schedule for EntropyReduction, and keep the debt manageable. --DaveSmith

Clarifications (driven from the discussion in FirstLawOfProgramming):

Technical Debt includes those internal things that you choose not to do now, but which will impede future development if left undone. This includes deferred refactoring.
Technical Debt doesn't include deferred functionality, except possibly in edge cases where delivered functionality is "good enough" for the customer, but doesn't satisfy some standard (e.g., a UI element that isn't fully compliant with some UI standard).

TechnicalDebt is a measure of how untidy or out-of-date the development work area for a product is. --DaveSmith

Don't let the debt build up. Everyone knows the list will never be addressed. Remove cruft as you go. Build simplicity and clarity in from the beginning, and never relent in keeping them in.

... but, then, even as hard as C3 works to keep our space clean, two things have happened:

We have had to schedule "clean up" periods during both major phases. The first time, we slipped an entire iteration to do clean up. We attributed the problem to people feeling time pressure and failing to realize that cleaning up made us go faster. The second time we called "fun week" and just worked on things that seemed interesting, some of which were areas someone wanted to refactor.
We have taken a period after production release of a phase to settle down and clean things up. This period acknowledges several facts: there will be an unpredictable number of support requests when we go on line; we are tired and need to rest; there is code in the system that needs cleaning up.

Our feeling is that if we could stick to our XP rules, we wouldn't need special taxes or special times to clean things up. But realistically, can you play your best game day in and day out? -- RonJeffries

It seems that XP aims to prevent the buildup of DesignDebt, and denies that other forms of TechnicalDebt are really debt.

Compiler upgrade: you don't do it unless there is a compelling technical or customer reason for it. In the former case, you do it when it makes the most economical sense; in the latter, when that story comes up. Either way, there's no interest to be paid.
UI guidelines: again, this is a customer story. Deferring it is a business decision; development doesn't pay any interest on this supposed debt.
Uncrufting code: XP-developed code is theoretically not crufty; when doing XP on LegacyCode, you only uncruft to the extent it helps your velocity. You pay a price for it being crufty, but it's not an ongoing interest payment until you uncruft it because you uncruft it as soon as it matters.

I HaveThisPattern. Deferring things is inevitable; we can never completely resist externally imposed schedule pressure to "just get it working so we can [ship|deploy] it." Good to at least explicitly record such things, to provide a stick with which to whack management later ("See all this debt you incurred? Now we have to pay it off...") -- RobertEikel

We've had some luck using fluorescent pink IndexCards for technical debt, attached to the bottom of our BigVisibleChart. This is useful mostly for debt that happens when we know a piece of code needs refactoring, but are not sure how to do it. The cards are created by the developers and are not scheduled. Instead, each week after the iteration planning meeting with the Customer is finished, the developers run through all of the debt cards, eliminate the ones that have been resolved, and pick one or two that can be dealt with during the upcoming iteration. Talking every week makes everyone aware of debt that's been around for a long time. Also, the pink is ugly and it's hard to ignore a lot of pink cards on the wall. -- AlexPukinskis

An excellent time to refactor code is right before you extend it. I think of this as making a place for a feature before I make the feature. Not only is it a great warmup for programming, it solves two important problems ...

Sometimes the proper refactoring depends on where you are going next. You may not know that at the end of a coding episode.
The customer that pays for first feature shouldn't have to pay for refactoring that benefits the customer paying for the second.

A developer's obligation is to make sure that the code as written makes the clearest possible statement as to how the solution was understood at the time of writing. This may require some refactoring as per SystemOfNames. However, the developer has no obligation to anticipate future needs, and usually does the future a disservice by trying. The future will be on us soon enough. And that is when the pressure of need and the tools of refactoring lead to great inspiration. -- WardCunningham

When a new Smalltalk or ENVY or Gemstone comes out, does C3 really switch to it immediately? That really seems like part of the technical debt, but I would hesitate to make the change when a "real release" is due. Then again, with ContinuousIntegrationReleaseFrequently?, maybe it is not a problem. -- BillTrost

No, we don't switch immediately. We choose our moment for that. However, the sooner the better, or you wind up multiple versions behind, i.e. in TechnicalDebt. We're still struggling to deal with GemStone 5.x. Bring me the head of ... oh pick any one of them. -- RonJeffries

Ron and I are most interested in the debt addressed by code refactoring. Dave includes more than that in his definition. -- WardCunningham

See DesignDebt.

The savvy developer treats technical debt just as the entrepreneur does financial debt. They use it. It speeds delivery, so long as it is properly managed.

From RefactoringDebt:

This is a state of a code base when changes have been to either fix bugs or add features that have incrementaly made the code more whiffy. The size of the refactoring debt can be determined by the number of bad smells.

This tends to happen near to a release when larger changes can be considered too dangerous. This is specifically true of projects that are not well unit tested because it doesn't feel safe to perform the required refactorings. GerardMauriceDavison

If you can have technical debt, perhaps you can also have TechnicalFutures

Here are slides from a talk I did at YAPC::NA 2006 on tech debt: http://petdance.com/perl/technical-debt/

Note: Andy Lester's YAPC::NA 2006 talk is now available in full, with a flash player that plays the talk and slides at the same time, here: http://www.media-landscape.com/yapc/2006-06-26.AndyLester/

Note that not all debt is self-incurred. Sometimes project requirements will change out from under you, putting you in debt in a way you can't possibly pay.

Consider also entirely external debt. Say there's a patch released for a newly discovered security hole. All of a sudden, you're in debt for the amount of time it will take you to apply a patch to 100 boxes, through no fault of your own.

A chapter of a book I am writing is about technical debt, how it occurs, and what we can do about it. Please take a look and give me feedback. http://anthony.lauder.googlepages.com/legacysystems -- AnthonyLauder

Very interesting Technical Debt Plugin for Sonar: http://docs.codehaus.org/display/SONAR/Technical+Debt+Plugin

Based on formulas given here: http://docs.codehaus.org/display/SONAR/Technical+Debt+Calculation

Brought to my attention by this article: http://www.infoq.com/news/2010/03/monetizing-technical-debt

In essence, the approach seems to be to take six popular (but somewhat arbitrary) metrics, establish "acceptable" levels for each, and compute how much it would cost to bring it all into compliance.

It might be simplistic, but it puts quantified dollar amounts on the table, which should be enough to at least catch the attention of some "pointy-haired bosses."

By the way, what were the conclusions of the Technical Debt Workshop? http://cs.calvin.edu/activities/technical-debt/

My senior is very experienced in my office, she has been working there for like 5 years and previously worked in another company for almost 10 years. I actually will always have to go to her to have some advices about work, since the scope of the software is very broad and searching around and diligently reading the documentation will take hell of a long time.

The code I currently work with was written by her a few years back. Recently the requirement changed, and we have to modify something tricky, and she actually advised me to update and retest the 50+ lines bunch of crap instead of rewrite a tidy one with a better maintainability? I mean, I still respect her as a senior, but I can't go with that kind of solution that is way too frustrating.

Later I insisted to rewrite because we can't afford to edit and retest the whole thing this time, and then to edit and retest the whole thing again next time and next next time when the requirement changes. And she said okay, but do not make it "loosely coupled", better make it "tightly coupled". I really wondered if she ever sit down and thinks what she's done with those codes and realize why is it so hard to implement changes.

"We're not completely conforming to the UserInterface guidelines. We'll get to it next time. "

How is that technical debt? - BillDehora

CategorySoftwareDevelopment