Unit Test Tutorial

Ok, I'm a computer hobbyist and a novice programmer. I get a lot of the ExtremeProgramming concepts, but I'm just not getting UnitTesting, probably because I don't understand testing. Instead of waiting for someone to explain it to me, I'll roll up the sleeves and give it a shot. I'll explain it to myself and you all can elucidate. -- SeanOleary (April 2001)

Well now, I've learned quite a bit since I first wrote this page. I'm going to take another stab at it. -- SeanOleary (August 2002)

I'm going to use a 'real' toy example. I'm writing a program that will create characters used in a RolePlayingGame.

The UserStory:

When we create a character, we roll dice. We want to specify dice rolls using the widely used RolePlayingGame format:

"2D6" or "2d6" - Two six-sided dice (as in MonopolyGame or CrapsGame?)
"3D6" - Three six-sided dice.
"1d100" - One onehundred-sided dice.
"2d20" - Two twenty-sided dice.

Ok, I'm guessing that's an OK UserStory. (Since I'm a user, and it's my story, and I'm sticking to it ;-). I tried avoiding computerese and using the nomenclature specific to the problem space.

I'm going to use PythonLanguage, and for this simple example, I'm not going to use PyTest. And I'm not even going to bother with objects until I decide I need them. I'm also going to go in excruciatingly small steps (ala KentBecks TestDrivenDevelopment book).

Based on the UserStory, a string argument will be passed in, and a result will be returned. Here's a failing test:

 testRoll = roll("1d1")
 assert(testRoll)

I run it and it fails... And that's a GoodThing! It's supposed to fail. There isn't any code. Now I DoTheSimplestThingThatCouldPossiblyWork (DTSTTCPW) to make the test pass.

 def roll(text):
    return text

 # Test code goes here...
 testRoll = roll("1d1")
 assert(testRoll)

Now the test runs. This is the state commonly celebrated as GreenBar. Granted, this doesn't satisfy the whole user story, but it does satisfy one part of it: I pass a text argument in and something comes out.

According to TestDrivenDevelopment, I need to refactor any duplication... But since this is quite basic, there's not much to refactor yet. So I iterate again: Write a failing test. This time I want the function to return a "1" when I pass in "1d1".

 testRoll = roll("1d1")
 assert(testRoll)
 assert(testRoll == 1)

It failed as expected. Now I have to make it pass... Remember while coding to DTSTTCPW.

 def roll(text):
    return 1

Well, that works... Now my code passes two tests: It returns "true" value, and that value is 1 when I pass "1d1" to it. I suppose I could remove the first test, but the point about writing tests is that you amass a collection of tests, one for each thing that could possibly go wrong.

Sheesh, this could take all day :-)... Running UnitTests seems to be some form of acquired OCD. Constantly checking that your code is OK.

From now on I'll be writing the failing test first, then following it with the code that makes it pass. I constrain my comments to refactorings.

Next step, testing that the input "2d1" returns the proper result 2.

 testRoll = roll("1d1")
 assert(testRoll)
 assert(testRoll == 1)
 testRoll = roll("2d1")
 assert(testRoll == 2)

I then code...

 def roll(text):
    if (text == "1d1"):
       return 1
    else:
       return 2

And the tests pass. But now I have to invoke the third part of the mantra: ReFactor. Get rid of duplicate code. The way to get rid of the conditional it to generalize the function so that "nd1" will return n. In the language of the domain, I need to get the number of dice from the passed in argument. So I'll extract method.

 def numberOfDice(text):
    if (text == "1d1"):
       return 1
    else:
       return 2

 def roll(text):
    return numberOfDice

Wha?!? I got an AssertionError! What went wrong? Upon inspection, I determined that I forgot to add the argument to the numberOfDice function call. It just goes to show you that MartinFowler is right when he says "A suite of tests is a powerful bug detector that decapitates the time it takes to find bugs". I fix that and then I decide I need to write some tests for numberOfDice...

 testNumberOfDice = numberOfDice("3d1")
 assert(testNumberOfDice == 3)

...More to come...

Older (pre-August 2002) contributions follow, I will refactor them later...

Well, I'm modeling Dice, so I suppose Dice will be one of the classes.

Therefore, I should create a UnitTest called "TestDice" that will test the following things:

The function should only accept a properly formatted string in the format of "xDy".
The function should return an integer.
It shouldn't matter if the upper- or lowercase "d" is used.
Characters before the "d" should be a positive integer.
Characters after the "d" should be a positive integer greater than 1 (a 1 sided die?).
The number rolled should always be greater than or equal to the lowest possible roll.
The number rolled should always be less than or equal to the highest possible roll.

Please put your thoughts under the line. I'll check back in a day or so and refactor them into my understanding. Once I'm that far, I'll actually write some code... Yikes... -- SeanOleary

[Partially Refactored Discussion Ahead]

"I think it's very interesting how much I've learned about ExtremeProgramming and how comparitively little I've learned about UnitTesting. Everything really is connected to Everything! Thanks to all the commentors so far. Please keep it up! Thank you!" -- SeanOleary

ObjectOrientation

"working with objects"

"think in terms of "which classes" rather than "what function"

"A potential problem is that you're not dealing with deterministic behaviour"

"it's hard to write tests when you don't actually know what the methods should return."

Random (Ha!) Numbers

"assume random() can't possibly break."

"issues surrounding UnitTestingRandomness"

OffByOne Errors

"I never can remember how to get a random number from 1 to N, in any language."

Thoughts on Tests

"if you go on that assumption [that random() can't break], all you have to test is [...] that your object has gotten the number of dice and the range right."

"I wouldn't test the return type, unless a dice roll could return something other than an integer"

YouArentGoingToNeedIt

"Verifying preconditions falls into the YouArentGonnaNeedIt category. Which is to say, [the class] doesn't need to check the input at all, because it's being passed from someplace else in the code."

"verifying input from the UI conforms to the protocol belongs in that logic, not the roll dice logic."

DesignByContract

"your initial stab at tests is verifying preconditions and postconditions."

"A UnitTest should verify that an object meets its postconditions and maintains its invariants if the caller meets the preconditions of the object's methods."

This comment does not correspond to the XP examples that I have seen; the Bowling Game exercise[http://www.objectmentor.com/publications/xpepisode.htm], for instance, avoids several opportunities to inject tests of preconditions and postconditions. (Of course, if your object's implementation does include a precondition test, you should add a UnitTest to verify the correct behavior.) - DanilSuits

The function should return an integer.
The number rolled should always be greater than or equal to the lowest possible roll.
The number rolled should always be less than or equal to the highest possible roll.

These might have tests associated with them.

Consider instead the following tests

RollDice( "1d6" ) returns an element of the set [1-6]
RollDice( "1d4" ) returns an element of the set [1-4]
RollDice( "1d100" ) returns an element of the set [1-100]

Now, if you are implementing the SimplestThingThatCouldPossiblyWork, then all your tests are still satisfied by a trivial implementation ( return 1; ) - so we need some more sophisticated tests

RollDice( "2d4" ) returns an element of the set [2-8]
RollDice( "3d6" ) returns an element of the set [3-18]

Of course, these can be satisfied by simply returning the number that appears before the d, so we'll need something harder. That's when we start playing with "call the function N times, and ensure a reasonable distribution of the outcomes" tests and "make sure every roll in a sequence falls in the expected range" tests.

Does that help? -- DanilSuits

In languages without DesignByContract you have to mix the specification of an object's contract with the code that verifies the implementation of that contract. DesignByContract gives you clean SeparationOfConcerns, but does not remove the need for unit testing. However, you change the process from CodeUnitTestFirst to CodeContractFirstAndUnitTestSecond. --NatPryce

When something is hard to test, sometimes that indicates that it's doing too many things at once. Suppose you split up your die-rolling thing into smaller bits.

One takes the string "3d6" and turns it into the numbers 3 and 6. Since you're writing this in a language that supports OO rather than enforcing it, I suggest that you just make this a plain ol' function. You test it by feeding it a bunch of sample strings and making sure you get the right numbers out.

The next part takes the number 6 and returns a random number uniformly chosen from {1,2,...,6}. That's harder to test, but there are well-known techniques for evaluating random number generators. For your application, quite simple tests may suffice. I suggest that you actually make a "Die" class; its constructor takes one parameter (the number of faces), and the instance has a "roll" method that returns a random number. So you test this by making some Die objects and then rolling them over and over, and doing statistical tests on the results.

The final piece in the puzzle takes the number 3 and a Die object, rolls the die 3 times, and adds up the results. You test this by making a surrogate "Die" object which records the fact that it's been called and what value it returned. Then you check that the thing's been called the right number of times and that the sum of the results is what it's meant to be.

One final remark about all this: actually this particular problem is simple enough that such heavyweight testing might be a mistake. But the same principles can be applied to bigger and harder problems. If I were writing your program, I'd write a single short function to turn "3d6" into a suitably-distributed random number from 3 to 18, and I would test it only informally. But then, I'm not a fully paid up member of the Extreme Party, and I have a pretty decent idea of what sort of thing I can write without making non-trivial mistakes and what sort of thing I can't.

-- GarethMcCaughan

Although (again, from what this egg understands), the XP way would be to delay refactoring until the first engineering task has been completed. It's very easy to look at "3d6" and think you need a function to parse it, but you don't need anything like that yet.

Yes. I'd think in terms of testing a Die object first, with a constructor that takes an integer. Perhaps later I'd get to the user story that mentions that we might want to role multiple dice for a single score. Perhaps then I'd need a Dice object, whose constructor might accept an integer and a Die. I could then test Dice using a DummyDie that isn't very random!

I would separate out the Dice object (that interprets the "xDy" format) from the object that does the random number generation. That way you can write a MockObject random number generator that tests that your Dice parses strings correctly and passes the expected arguments to the RNG. The random number generator itself will typically be provided as part of the standard library, and so you won't have to test that class explicitly.

I will shortly post an even simpler example in C so that only simple programming structures are used. -- DaNuke?

CategoryTesting