Compensation Game

An idea for compensating members of a team. The goal is to provide the most fair disbursal of funds according to contribution without relying on a single person (manager) to judge everybody's effort. Close collaboration among team members is assumed.

To play:

Each individual rates all of the others on a scale from one to ten. Somebody manipulating the game (see ManagementlessOrganization) defines how to rate. For example, "Rate the teammembers according to the contributions they made. Include any contributions, even non-technical, that you feel helped the team accomplish its goals for this iteration."
Surveys are collected and normalized so that five is the mean score for each survey.

a Average the scores.
b Subtract the difference between the average and 5 from each score.

The normalized scores for each individual are averaged.
The resulting ratings are published. Funds are disbursed proportionatly according to rating.

Missing:

Ways of detecting cheating.
Ways to avoid/handle negative ratings.
...

Example:

Ann, Bob, Chrissy, and Dave work on a project and earn $100,000. They rate each other.

Ann ranks: Bob-9, Chrissy-10, Dave-6
Bob ranks: Ann-6, Chrissy-5, Dave-3
Chrissy ranks: Ann-9, Bob-9, Dave-8
Dave ranks: Ann-2, Bob-1, Chrissy-3

The scores are normalized:

Ann: Bob-5.7, Chrissy-6.7, Dave-3.7
Bob: Ann-6.3, Chrissy-5.3, Dave-3.3
Chrissy: Ann-5.3, Bob-5.3, Dave-4.3
Dave: Ann-5, Bob-4, Chrissy-6

And the results come in:

Final rating: Ann-5.5, Bob-5, Chrissy-6, Dave-3.8

We divide up the loot:

Total rating: 5.5 + 5 + 6 + 3.8 = 20.3
Dollars per rating unit: $100,000 / 20.3 rating units = $4926 / rating unit.

And disburse it:

Ann: $27,093
Bob: $24,630
Chrissy: $29,556
Dave: $18,719

I'm the author of the above. I don't know how this sort of thing is done by the experts, so the above might be obvious and well-known, or full of holes. I've never actually tried it, even as a game. So I'd like to hear your thoughts. --JimShore

This is an interesting idea, but would it be possible for two players to cheat the other team members by agreeing to rate one another as 10, and everybody else as 1? -- francis

Or, what's more subtle (and, more important, IMHO), is that it encourages team-member to team-member competition, which isn't the best of things. Even without "Survivor"-like cheating as Francis mentions, this is a contest of perception of effectivity rather than of effectivity itself. Individuals are now forced to be concerned with making sure everyone noticed their good work rather than going on to do more good work, no?

But on an XP team, people are going to notice the quality of your work, whether good or bad, anyway, because you're working so closely together. We just had our annual reviews here, we played the I-rate-myself-and-my-manager-rates-me-too game. Now, my manager happens to work fairly closely with us, but I found myself looking over the form (to fill it out on myself) and telling one of my teammates, "Hey, you should give yourself at least a 4 in the Foo area." So, I was wishing for the ability to peer-rate. I think it all depends on whether you've got a cooperative team or a strife-ful one. -- SusanHoover

I've modified the description to mention that the game is meant for teams that work closely together (such as an XP team). As for cheating, perhaps the answer there is to look for people whose rating significantly deviates from the average, or maybe to just not play the game unless that level of trust is present. Susan, if your boss had suggested this approach instead of the you-and-him approach, would you have been interested? --JimShore

Actually I don't think it encourages Intra-team competition, I think it encourages cooperation -- but only in regard to the scoring (i.e. it encourages cheating), see this CompensationGameTheoryAnalysis I played around with using the Ranger Course example from the EgalitarianCompensation page for more details -- BillCaputo

The base concept is similar to the 360-degree performance review. 360-degree reviews often give biased inputs; some people feel loyal to the "team", so don't want to diss people even if they deserve it, while others have no loyalty to the team and bag everyone else in an effort to improve their own standing. Normalisation helps reduce the impact of this, but it doesn't remove it entirely. Managers are meant to use 360-degree reviews as a check on their own evaluations, not as a substitute for them.

Personally, I much prefer stability of income, anyway (largely because I have a need to pay fixed expenses). If an employer tried this for my main compensation, I would leave. If they tried it for bonuses or raises, I'd be wiling to give it a go for a while. -- RobertWatkins

I agree with all of the points in the first half of your note, Tom. The CompensationGame does tie compensation to an undefined scalar quantity in a zero-sum game. There are no guidelines for assigning ranks. (I added one just now.) Where we differ, I think, is our assessment of what this means. Since there is no objective criteria -- and I agree with you that there is no way to measure the contribution of an individual in a fair way -- players are left to their own subjective opinion when assigning ranks.

Since the players rank each other using a subjective measurement, all the intangibles are automatically included. Is one person hard to get along with? He probably won't do as well. Did another one comfort the shy member of the team? That shy member will probably give them a higher rank. What if the obnoxious member stayed all weekend fixing a particularly bad bug and got the whole team a big bonus? What would happen then?

I guess my feeling is that rankings are inherently subjective. Since they are, why not balance out the normal human soap opera by getting multiple rankings and averaging them, rather than counting on one omniscient manager to do it all? I have far more faith in the ability of my team members to rank me than my manager. My team members spend far more time working with me, they see what I do, and they are more competent to judge me technically. My manager provides goals for the team, not for me, doesn't work with me on a day-to-day basis, and isn't technically inclined. I'd say my team knows more about my subjective worth than my manager.

In the second portion of your note, you talk about providing pay based on the value of contributions. I feel that discussion is outside of the scope of the CompensationGame, as the CompensationGame is meant for teams that satisfy goals as a unit. The CompensationGame kicks in once the team has been awarded its half-million revenue-based pay... now, what do you do with it? Give everybody an equal share? Even to the intern that spent most of his time in training? Even to the genius who consistently solved your toughest problems?

--JimShore

The compensation model used by a lot of law firms could be worthwhile examining. "Non-partner" programmers would get a base salary determined by the partners, with the odd bonus here or there. the partners would get no base salary, but would divide the money up evenly (or by ranks, allowing junior and senior partners), and also be subject to bonuses. (Typically, partners would in pratice have a base salary, paid as an advance on their revenue share). Developers would become full partners by being proposed by two or more partners and being voted in. Personally, I would see this working at the company level, not the team level (I've got a gut objection to programmers being paid different amounts based on what project they work on, especially when they don't have much control over what project they get assigned to). In the unlikely event I ever start up my own business (by myself or with a partner), this would be my preferred model. -- RobertWatkins.

Would it not be fair to include a self-evaluation as well? I should know best how active I have been, and in a small group where people trust each other I should have a say about my own compensation.

I tried the compensation game with a team of four programmers (not including myself, who was the 'manager'). The rankings I gave the team members, and the results of the compensation game, were roughly the same. I saw that as a very small indication that the CompensationGame might be successful. But there was no money involved. -- JimShore

Money changes everything. -- AnonymousDonor

I have three things to say about CompensationGames.

Once when I was RolePlaying, the game referee decided that bonus points for 'good role-playing' should be awarded based on a voting system. You were not allowed to vote for yourself. Somehow, each of the 6 people (including me) got one vote. So the referee said, "Okay, let's try this again, but this time you can vote for yourself." We did it again, and this time everyone voted for himself. Except me. I stuck to my original vote. The referee was impressed, so he awarded the bonus points to myself, as well as the guy I voted for (since he had the most votes obviously).

The last job I worked at had a complete moron managing the programming department. He himself was in no position to evaluate anyone there because a) he could barely program, and b) he was never involved with the programmers who were doing the real work. So he decided to have each programmer evaluate himself. I thought this was ridiculous because the stuff I was doing was completely different from what the other programmers were doing (I was doing mostly R&D stuff). The questions were ranked: Needs Improvement, Meets Expectations, Exceeds Expectations, and Greatly Exceeds Expectations. Since I considered the work I was doing to be very valuable to the company, I was distraught that most of the questions didn't really apply to me. So for the question about being innovative, I made an extra rank called Sets Expectations, to show that I was setting the standards of what was possible in our field (which isn't saying much since our field was multimedia programming). Of course, the results of the self evaluation were so skewed that they were useless for determining compensation. Everyone knew it was a joke.

Despite the two examples above, I think that CompensationGames might be useful. The problem is, they can't be simplistic, or you'll get either attempts at cheating (role-players are consummate cheaters), or useless and demoralizing results. I believe that one way to combat this is to make the game highly multidimensional. Have one portion of the game for self-evaluation, one portion for peer evaluation, one portion for cross-departmental evaluations, one portion for subordinate evaluations, one portion for superior evaluations, and one portion for evaluating the product and/or the company. Combine all these evaluations (I like the normalization idea) with different weightings to determine compensation. In each evaluation section, have questions for perceived value, perceived quality, etc. That way it's difficult if not impossible to cheat, and quite unlikely that an important aspect would be completely missed. It's not perfect, but I think it's much better than a simple game like the one suggested above. Simple games are simple to figure out. Simple games assume the problem is simple. It's easy to come up with a simple game that has complex interactions (like Conway's GameOfLife), but it's hard to make it measure the right score.