Westmoreland Effect

Alias for WhateverGetsMeasuredGetsOptimized.

"I don't like those numbers. Bring me better numbers." -- attributed to U.S. General William Westmoreland

When a reward system is linked to performance against some arbitrary, numeric measure of progress (i.e., a metric), a system under stress can behave in ways that undermine the quality of the information used to calculate that metric. When the metric is poorly chosen, the system will evolve ways to manipulate and subvert the metric.

Examples

When a project is under severe schedule pressure, performance against schedule is a frequently misreported. Exit criteria get relaxed, and problems are swept under the rug.
A reward system based on SLOC (source lines of code) is known to cause lots of lines of code to be produced.
Rewarding based on numbers of bugs fixed can lead to "cherry picking." The easy bugs get fixed, leaving the difficult bugs. This raises the average time to fix the remaining bugs, changing one of the "meanings" of the open bug count.

-- DaveSmith

Does anyone have a better name for this? I don't. But I do think that this is a good name for something rather different from what is described below. The thing that strikes me about the Gen. Westmoreland quote is that it seems to point to some "rational" process going on--one driven by numerical data--but that process is being subverted by having a emotional quality-of-merit (in this case, how mucht the general likes the numbers) applied to the numerical data. Since reading UnderstandingComputersAndCognition I've become very suspicious of these "rational" processes, and have come to reflect on the times when I've seen supposedly "objective" numerical data and analytical techniques based upon them massaged in various ways to support a previously made (but hidden) decision arrived at by intuitive means. It seems that often the numbers, or rather the illusion of "objective", "rational" decision making that comes from them, will be used as a shield of a decision turns out to be wrong, but de-emphasised if it turns out to be right.

Is this an AntiPattern?

: Rewarding based on numbers of bugs fixed can lead to "cherry picking." The easy bugs get fixed, leaving the difficult bugs.

Isn't it good to RefactorLowHangingFruit?

And then we digress slightly, into something related to WhatsThePayoff.

AnySystemBeaten? or CanBeatAnySystem? - A tribute to Dilbert's manager being able to beat any system that has been devised to monitor performance?

A possibly more on topic and suggestive title would be HowToLieWithStatistics or simply StatisticsLie or SystemsAndIllusions?

Another example is rating managers by employee turnover, if you overpay marginal employees, then your turnover is very low and the manager gets good reviews and a promotion leaving a disaster behind for their successor

-- PeteMcBreen

I imagine there is some threshold level of bureaucracy for any person in a project. Below this he will do some amount of work (depending on aptitude, interest, drive etc.) to make an honest effort at meeting the goal or deadline. Above it, he'll realise that playing by the obvious and expected rules isn't going to work out, so better come up with something else.

I noticed it first, I think, while talking to people about revision techniques for (first degree) end of year exams. Some people work hard at covering everything on the syllabus, in enough detail that they can do a decent set of questions on the past papers. Others put their effort into trying to extrapolate from the past papers to figure out what set of questions is most likely (there were some definite patterns). Then they put all their time into some of these, and prayed that the examiners didn't throw in something odd.

It isn't "PlayByTheRules? vs. PlayWithTheRules?" exactly. It's more like UseTheRulesToHelpYou?.

-- MatthewAstley

While this may be an UrbanLegend, industry rumor had it that in the 1970's, when IBM was using its concept of a "Word Processing Center" to sell an "Enterprise Solution" to gullible businesses, they structured the early centralized typing pools so that the keyboardists in the pools were paid by the keystroke, with no attention paid to errors. It allegedly did not take the keyboardists very long to learn how to hold the "repeat" key down, then select and delete the repeated characters, and hence dramatically enlarge their paychecks. -- TomStambaugh

Related: SovietShoeFactoryPrinciple

From one of TerryPratchett's books (not an exact quote): "We're paying a fortune in rat bounties, What can we do to recover the money?" "Tax the rat farms!"

Can't remember who this quote is from... But... Essentially "Whatever gets measured gets optimized." -- DanPlubell

See DeathByScheduling, PerformanceIndicators, StatisticsLie, JustMakeItRight

CategoryQuote