Dont Puta Number On It

[Note: we need to combine the concerns expressed here and on PutaNumberOnIt and ClearRequirements to get some community consensus on this stuff. Help, anybody?]

On PutaNumberOnIt, somebody said, "How about making sure that the client's requirements incorporate everything he thinks is a characteristic of value?"

For a complex problem, the client is usually not in a position to put a number on everything, especially if he's not a technical person himself. Let's say I, as a client, want a frutzbar. I want it small and fast, so I don't have to upgrade so many of my computers, and I want it flexible so I don't have to pay an arm and a leg if something changes. Also, I want a lot of features and a low price.

And IwantaPony.

How am I going to know, if I specify a maximum memory footprint of 30 MB for the clients, if that's even possible for a full-featured frutzbar? And if I specify 300 MB, how do I know if I'm not giving away 200 MB?

If I specify that the frutzbar must run well on a 750 MHz machine, how do I know that's not driving up the price by an insane amount? Or that it makes some nice features unfeasible that would have been possible on a 1 GHz machine?

About flexibility, I might specify that it must be possible to switch the database vendor because that bit me on the last project, not knowing that this time, there is this application server thingie involved that's much more likely to have to be switched. Also, maybe for frutzbars it's easy and customary to put a business rule engine in, but I don't know that, so I make the mistake of not specifying it, etc.

As a client, I would much rather have the developers work with me, initially to understand my goals and values, and then later, when there's a concrete trade-off regarding these goals to be made, to just ask me about it: "We can do it in Java or in C++. C++ will run fine on 500 MHz PCs but add 30% to the development time." "Using features X and Y of the app server and database would allow us to shave another 10% off the development time but make changing either one more expensive later" "We could use this business rule engine that costs $ X but would allow you to change..." etc.

In other words: Don't make the customer pull numbers that he can't know the implications and trade-offs of out of thin air**, and later use those numbers to protect yourself / CYA***. Instead, try to understand the customer's goals, present the important trade-offs to the customer in terms of these goals, and let him make the choice.

(**) This, of course, doesn't apply to cases where the customer really has known numeric goals or constraints, like the maximum budget he can spend, or the number of users the application should be able to handle.

(***) In some political settings, CYA may really be necessary, but I don't want to go there.

[Hmm. I hadn't intended to make PutaNumberOnIt to be a didactic statement of fixed position before the start of any project. I specifically did not intend for non-technical clients to make blind, uninformed, technical decisions. We need to talk about how much involvement the client has in marketing and technical research prior to the start of the specification phase. Can we merge these two pages? Perhaps create a ShouldWePutaNumberOnIt page or something similar? -- MartySchrader]

Sure - feel free to merge/refactor as much as you like. One possibility that might be clearer than a combined page or the current state might be to reorganize the content into twin pages WhenToPutaNumberOnIt? / WhenNotToPutaNumberOnIt?, but I'll be interested to see whatever else you have in mind.

[Excellent recommendation. Action to follow - let me cogitate a little, eh? This will probably take the Wiki community some time to get worked out.]

What about asking the client to prioritize? Maybe ask them to give a number from 0 to 9 for factors such as execution speed, code maintainability, delivery deadline, deployability, etc.

Yeah, there's already the QualityFunctionDeployment thing for that. Very helpful for examining the value of features in light of the overall value of the product.

The customer always puts a number on it. The customer decides how much he will pay for it. It's better for everyone concerned if that amount (or an algorithm for it) can be negotiated up front. -- EricHodges

The truly important qualities of a software project cannot have numbers placed on them and the predicted numbers stated at the beginning of an effort are often subject to change. The development team may overrun or underrun their budget. The business may cut the budget mid-year. Setting man-loading values and adjusting burn rates are quite easy, doing something useful for the end users within these constraints is the difficult part.

The numbers that need to be expressed up front on any project are those that the client comes up with on his own bad self. The client wants a frutzbar. Cool. Now, let's talk about what kind of frutzbar. How big? How light? How fast? How costly? How many per year? All these kinds of questions need to be answered in terms the client understands before design can begin. He may express these numbers in frutzs per bar, or whatever other measurement he finds convenient. Once the client defines numeric values for these measurable quanta, you can translate these numbers into requirements - and measurements - that you understand.

If you can't define what you are building in terms of measurable quanta, you really don't know what the heck you're talking about, whether you are the client or the developer.

Why not? The critical operating characteristics are usually subjective and often assumed. Even for an existing system, measuring values is an inexact science; not only does the measurement need to be specified, but also the the range of measurement and the measurement process need to be defined. To be able to define any sort of descriptive measures, one needs to be an expert in the system to be described.

The reliance on a set of measurable quanta put the user and developer at odds. The user and developer need to cooperate to discover the true, subjective ideals of the system and make the best possible trade-offs to approximate them. This is the only approach that minimizes over-engineering to meet an unnecessary specification as well as under-engineering where the specification is met but not the underlying need.

Yeah. This is why the client needs to be the one to put numbers to these things. Suppose Philips want to make a "better" TV than Sony. Better how? Bigger? You can measure that. Brighter? The same. Thinner? More energy efficient? Lighter? Longer-lasting? Less costly? All these things can be measured with relative ease.

The problem comes in when the client wants to do something "better" and they don't know what they want. Better how? Well, we're not sure. Bigger? We don't know. Brighter? We don't know. Thinner? More energy efficient? Well, we don't really BlahBlahBlah.

How do you deal with this situation? You force the client to specify how this New! IMPROVED!! thing will be an improvement over what's already there. When you compare attributes, you have to ask the client to provide a measurement -- even a relative measurement -- of one thing to another. That way the client can't just say something about making this new widget cooler than the old one and stick you with the task of determining what's "cooler." You have to ask, Hey, um, how do you measure coolness? Do you have a standard cool factor to compare against? Do you use external testing facilities? How is it done? How is the measurement expressed? In what units?

Doing all this forces the client to think about the real qualities of the thing they are trying to improve. Without being able to set measurable standards nobody knows whether or not you've met the goal. Eh?

Why do we need to meet a goal? The client is asking for improvement. The technical people need to identify possible means for an improvement. The problem is one of people making up numbers just to have a number and then fretting when the number is not met. Trust in the process of improvement and let the magnitude of improvement take care of itself.

You miss the point completely. The client is risking some fixed amount of development and marketing time, money, and planning to get a new product that will surpass the competition. This risk needs to be offset by measurable gains. If an improvement project only adds 5% to the fritziness of the old bar and the development project costs 25% of what the old bar took to create then the project is a failure already. All of this stuff has to be expressed in numbers so that everybody involved knows exactly what they're talking about. You can't just say, "This thing is gonna be better" and expect the client to give you a blank check. Resolving all the issues into measurable quanta gives you a solid basis for comparing costs to benefits.

Yes, I have lived it. You really need to know how to measure improvement, because without a way to measure it you may end up building a system only because DifferentIsBetter, and sooner or later, someone in the customer side will start asking "why is the new frutzbar that cost $XXX better than the frutzbar we had before? And believe me, he/she is not going to like the idea that we built that new frutzbar just because DifferentIsBetter.

Hah! Exactly the point I was trying to make above. If you are going to make an improvement, you need to know what is being improved, how much delta there is going to be, and what you are paying to get that improvement. In race car design you don't see too many engineering houses try to surpass the Big Boys, because the measurable improvement would be negligible compared to the capital outlay for development. Only an amateur gets in over his head by starting a development project without knowing what it will cost to attain a certain measurement of gain.

It's also why you don't see anyone but the Big Boys win the races.

See: ClearRequirements, PutaNumberOnIt

CategoryRequirements