Unit Testing Is Design

People have noticed that XP doesn't talk about designing. Here's what I noticed the other day. When I write UnitTests, I am doing some of my most sophisticated design. But I don't do it in the pursuit of truth, beauty, or even money. I do it because I am lazy.

Let's say you have a couple of degrees of freedom in an algorithm. I have five of these and seven of those. So I have to write 35 tests to test everything.

I'm lazy, so the first thing I do is figure out how to write my objects so I can write 5 + 7 tests instead of 5 * 7. Then I write the tests, first for the seven variations, then for the five.

What I've found is that the designs that result from this process are the best I've ever done. But I'm not thinking about the far-flung future, or aesthetics, or even design patterns. I just don't want to write all those damn tests.

Try it. But you have to write the tests first.

-- KentBeck

I hate to be a PartyPooper - but - similar can be said about BigDesignUpFront too. We are so lazy that we don't write any code - we take a sheet of paper and a pencil and start designing because we are lazy. We write a proof that will nail all the problems in one terse paper rather than 3 million tests. We let some of the compiler do tests for us instead of having to write them, because we are lazy. Also, Kent seems to contradict himself when he says that XP is not about design - when in fact testing is about design? I have to say - I don't have much faith in XP the way people discuss and promote it with contradictions.

On a related note: should we test whether 1 plus 1 equals two, 1 plus 2 equals three, 1 plus 3 equals four - until infinity? Or should we design a proof for the problem as a scalable whole? I suggest using logic and reasoning wherever possible - and using testing too - but not one or the other. I get the hint that extreme programmers and unit testers are on some sort of rebellion where they try and do everything opposite to another approach, just for the sake of being rebels (and, of course, to sell lots of books, which may even include a free packet of snake oil in them in the inside cover at no extra cost).

Sorry again to be a PartyPooper - but I really smell a BadSmell.

The problem with diagrams as a design method is that you often fail to capture the important details of the system. It's like thinking you can draw a picture of the body of a car and expect it to contain all the details needed to build the car. Writing tests gets you to make all the hard decisions first, and with much more of the relevant information. In addition, it forces you to design the system in such a way that its components can be tested, which invariably means loose coupling.

See TestDrivenProgramming for similar ideas. Thinking of tests is one of the best ways I know to break up design problems. Tests serve as precise and refutable design descriptions. -- AamodSane

I've noticed this too. And another thing. Unless you have tests, what you can accurately say about your software is limited. You can talk about what it is supposed to do, and talk about what it will do, and get involved in all sorts of discussions about what it might do, but without a good set of tests, you can not say what it does. -- MichaelFeathers

I've been thinking about this topic for the past few days. While I agree with the spirit of the page, it seems to me that when you write the tests before the code TheUnitTestIsTheSpecification. Think about it.. when you make a UnitTest before writing the unit, you are thinking about what client code expects of the unit. The internals are another matter. As an example, consider KentBeck and ErichGamma's recent article on testing. The Money class for JUnit originally used a vector internally but it replaced with a Hashtable.The implication in the article is that the tests do not have to change in that type of refactoring.

That said, as Kent has mentioned: when you are doing XP, analysis, design, and testing all get juggled in the air in simultaneously. In the sense that design happens concurrently with testing, it seems that UnitTestingIsDesign. But, when you notice that UnitTests test only external behavior and can live beyond several internal refactorings, it seems that writing UnitTests is the act of specifying concretely what the unit will do. Fulfilling the test by writing code is design.

Good. Now we can talk about ExtremeCatalysis ;-) -- MichaelFeathers

This became most apparent to me during the DesignByContract discussion. UnitTests are similar to assertions, and assertions are clearly about specification. Eiffel was designed as a specification language as well as an implementation one. -- DaveHarris

Yes. And it seems that UnitTests are targeted specifications. By virtue of the fact that they are outside the code and they tell you only what will happen under certain conditions (as Dave mentions in AskTheComputer) they are specific questions which a programmer asks of the code being used or developed. A way of communicating with the code. It seems that the key difference between DesignByContract and XP UnitTests is that DBC places the spec in the code when the class is written in anticipation of a variety of uses. In my meager attempts at XP, I've noticed that I revisit the tests when I am ready to use a class. I look at the context to see what I need. Then, I check the class and its tests to see if they support what I want. If need be, I add methods and test as I go. Then when the tests run, I have confidence that I can use the methods in the app.

DBC seems anticipatory. UnitTests seem to encourage me to look at the point of use. -- MichaelFeathers

That TheUnitTestIsTheSpecification was no surprise to me. The encouragement to do excellent design while following purely selfish motives, that was a great surprise to me.

Re: comparison with DBC. How often do folks add a new assertion? How often do folks add a new UnitTest? I add tests rarely but steadily to already tested code. --KentBeck

Something similar is true for assertions. The need for a simple expression of class invariants makes for a simple class. It also drives towards more generality, fewer special cases. Sometimes it forces the introduction of new abstractions because they make it easier to talk about what the class does.

In my experience, new assertions mainly get added when there is new code, or when some oversight gets noticed. Ideally, every bug should show up as an assertion failure so if you come across one which doesn't, you add a new assertion which detects it.

Sometimes the bug is caught by an assertion, but in the wrong class so you add a new one closer to the point of error. For example, popping an empty stack might violate a bounds-checking assertion in an underlying array class. It's arguably better to have the error caught by the stack itself. (I expect to get heckled on this point.)

Sometimes I might add an assertion to test an assumption I now want to make, but I'm not sure if it's true. That is, the code itself isn't changing but the properties required of it are. This may be taking an private, incidental implementation detail, formerly subject to change, and turning it into a published, guaranteed part of the interface, so that other code can rely on it. For example, a collection might happen to be ordered because it is implemented by an array. We don't assert that because we might want to use an unordered collection later. If other code starts to rely on the ordered property, then an assertion to that effect should be added to document the fact. -- DaveHarris

I'm conflicted about saying what I have to say here, but I'll say it anyway on the premise that it's always better to understand what you're doing than not.

I had a piano teacher who advised me to play a particular Chopin etude "as though I were pressing my fingers right through the keybed". In the same sense, "UnitTesting is design" and "UnitTesting is specification". The point is that if the phrase "press through the keybed", although physically erroneous, better communicates the thing to me in a way I can do it, then the phrase has value. However, when one "presses through the keybed", what one is actually doing is a combination of wrist and finger attitudes. Should the metaphor break down, we at least have the facts upon which to build another approach.

Testing is neither designing nor specifying: it's testing. Testing means running tests. When you design a test, you are designing. When you specify a test, you are specifying. It's the designing and specifying of UnitTests that gives the direct value described on this page, not the actual testing (but don't not test).

If the current generation of programmers is allergic to specifications, so be it. Let them design and ... er, code ... their tests and run them.

This in no way seeks to invalidate the techniques described on this page, but it does appeal to your better sense of observation. Thanks.

-- WaldenMathews

Moved from UnitTests

The purpose of UnitTesting is ... decidedly *not* to verify the correctness of your program

Would the author of this statement care to expand on this? This seems to be a major bombshell to drop without any sort of justification about why it was said.

True verification can only be done with a formal process such as a proof. Unit tests are not intended for this purpose. The ultimate purpose of UnitTesting is to help you program faster by increasing your confidence in the existing program and any changes that are made to it, and also by automating repetitive checking that would otherwise have to be done manually.

If the UnitTest does not verify correctness, how does it help program faster? If the tests don't verify correctness why even bother with them?

The word 'verify' has a specific meaning in ComputerScience. Verifying a program is like doing a mathematical proof. UnitTesting a program is like doing a scientific experiment to provide supportive evidence. Positive evidence is not proof of positive, but most often it is good enough to establish a great deal of confidence (the more tests, the more evidence you have). UnitTesting takes a lot less time to do than formal verification, and is a lot more flexible to change (if you modify your code, you have to redo your verification, but with UnitTests, you just click a button to re-run the tests).

To read more on formal verification, also referred to as model checking, you can do a search for keywords SPIN and PROMELA. PROMELA is a language to express models in, and SPIN is a software tool which applies Computer Science to formaly verify such a model definition for violations of invariants. This is a much more rigorous testing than is generally the case with unit tests and provides exact answers as to the validity of the model assumptions.

Is it possible to do ,two Hashtable caomparisons in Junit?

The purpose of UnitTesting is ... decidedly *not* to verify the correctness of your program

I'll try to expand on this. The author of the statement defined the meaning of 'verify'. To 'verify correctness' is to provide a formal mathematical proof. That is obviously not the point of unit tests.

If verification (as defined above) is not the purpose of UnitTests then what is? Simply put, UnitTests is a means of checking and reassuring the original assumptions of the coder. Notice that the coder here may or may not be the original author of the code under testing. Mind you, this doesn't imply that the assumptions made are correct in any way. Nor does it say anything about how these assumptions relate to the purpose and requirements of the program in the first place.

It's quite common that an incorrect assumption results in incorrect code and the same (incorrect) assumptions get checked by equally incorrect UnitTests. This of course doesn't in any way contribute to verifying the correctness of the program, rather it simply checks if the original assumptions made during the design and/or coding stages of the program are preserved. This is very similar to the reasoning behind CodeReview by peers as opposed to by the author (because the original author will probably oversee errors as the checking is done with the same assumptions as designing/coding, and therefore may equally be incorrect).

And from this comes the assertion that UnitTests "help you program faster by increasing your confidence in the existing program and any changes that are made to it". The confidence comes from the fact that your assumptions are checked and reassured and any changes done to the code, is automatically checked against the original assumptions... assuming you run the UnitTests, of course.

-- AshodNakashian