Performance Experiment

Some of the C3 team recently visited a non-Extreme project that is under stress. One of their requirements is the ability to handle 100,000 transactions per day. Their architecture has every object persistent in DB2, transactions batched from the mainframe, application running in a server machine. They have been under way for over a year and no one knows whether they can make their goal. Currently they can do about 100 transactions or something.

It might be possible to convince yourself on this one using a BackOfTheEnvelopeCalculation. If not, consider trying a simple experiment:

Estimate the number of objects read and written during a transaction. Use your metaphor, your object diagram (they actually have one), or pull some numbers from the air.

Estimate the number of bytes per object.

Assume a different DB2 table for each class.

Write a simple program that runs on the server, reading and writing the estimated number of records to and from each of N DB2 tables.

Run the program; time it; run the numbers up and down until you are sure you have it bracketed.

If you're in the ballpark, you can feel a bit more comfortable. If it turns out you can only do 1/10 the data transfer you guessed about, go back to the drawing board: your proposed approach won't cut it. (Maybe the same answer for 9/10, depending how sure you are about your guesses.)

This is one of those deja-vu moments. This sounds almost exactly like a project I'm working on now. The exact same scenarios: However, the performance goals are much higher. How is it that these companies keep making the same fundamental mistakes over and over again? -- JeffPanici

I've seen at least three projects fail like this. Mostly due to big architecture up-front, followed by big spend up front, followed by big trouble at the back but the money's all gone and no-one dare tell the boss that he just stuffed various millions of dollars down the toilet. Repeat. :).

Then they were just pretending to do big architecture up front. Can you imagine someone building a large building without doing the structural engineering calculations?

Oh no, reams and reams of calculations and projections. Project plans up the Wazoo and back again. Unfortunately they had the OOAD bug and created a big mess. Calculations are better than nothing, but they are still guesswork. The classic sign is the choice of technology preceding any implementation.

Did I miss something? Isn't that exactly what was suggested at the top of this page? Calculations up-front? Prototypes are a big part of architecture up-front, as they prove your calculations. If you wouldn't take the approach suggested at the top of the page, what would you do?

Yup, you missed something :). I am agreeing with the thesis; though it takes more than an envelope and a spike to assure architectural compliance.

Last week, I did the the back-of-the-envelope calculations for the bandwidth needed for the interactions described by the business plan of the project I'd just joined. When it came out at easily more than 6Mbps - average - they nearly choked; no-one had even tried to figure it out before, and they only had about 0.5Mbps planned in. Generally speaking, I've seen very few projects which bothered to do this kind of estimate, much less an experiment or even proper CapacityPlanning. And it can be an incredibly expensive mistake to fix (ok, so let's redesign for clustering and quadruple our hardware budget... aagh, what do you mean this thing we licensed doesn't scale?)

A bit of CapacityPlanning isn't quite the same as BigDesignUpFront. It the thing that will often let you know if YouArentGonnaNeedIt. Ignoring it means you don't even know what you need. -- BrianEwins

Maybe CapacityPlanning should be thought of as a PerformanceSpike rather than BDUF.

There's a chapter or two on this in ProgrammingPearls. It starts with one person asking "how much water flows out of the Mississippi in a day?", then takes the answer, uses the same techniques, and shows the data-handling system proposed for the 1984 Olympics will only work if there are 120 second in a minute.

30 minutes thought saved a lot of problems there.