Code Unit Test Third

[From CodeUnitTestFirst]

// I use java, so if you don't, and anything looks or sounds funny to you, that might be it

I modify this rule slightly - CodeUnitTestThird. The first thing is the interface ("public int countVowels( String str )", plus a dummy body so it will actually compile), the second is the method documentation ("/** Counts the number of vowels in the string. Vowels are A, E, I, O and U. @param ... @return ... */"), third is the UnitTest (which, as i have the interface built, will compile), fourth is the actual implementation.

Maybe this is what people mean by CodeUnitTestFirst. Until you have the interface, you can't compile the test, and, IMHO, writing something isn't finished until it compiles.

I use method documentation as a specification. It should (but rarely does) completely describe the behaviour of the method. I can't write a test of a specification until i have a specification. I know some people see the test as a specification, but, IMHO, a test can never be general and thorough enough to completely specify a method.

-- TomAnderson

TestFirstDesign is actually much more parallel then you seem to describe. It is an iterative process of adding to the test, adding to the code, adding to the test, adding to the code, etc., until complete. It is not writing the full test suite then writing all of the code for the test suite.

But how do you know what the interface should look like? I find it easier to make up the interface as I write the test. That way I can change my mind about the design without actually going back and changing the interface. Therefore, the interface and the dummy body always come _after_ the test for me.

...a test can never be general and thorough enough to completely specify a method.

If you follow the principals of TestFirstDesign, you will only add things to a method in order to satisfy a test. Anything not tested is unspecified. The method may have unspecified and untested characteristics, but you only need to specify and test these characteristics if they are of interest.

Your tests must be exactly thorough enough to specify your method. Obviously you can't run every possible input through the program, but you should be able to test all the important boundary conditions. Developing good test cases is an important skill, which can be learned through experience and study. For example, read TheCraftOfSoftwareTesting? by BrianMarick.

See also TheUnitTestIsTheSpecification

CodeUnitTestFirst would say that you first code 

  assertEquals(0, vowelCounter.countVowels(""));
which would force you to create the class and method, as otherwise it wouldn't compile. At this point, the implementation might be 'return 0;' Then maybe each of... 
  assertEquals(1, vowelCounter.countVowels("a"));// 'return str.length();'
  assertEquals(0, vowelCounter.countVowels("x"));// now we have to recognize that 'a' is a vowel and 'x' is not
  assertEquals(2, vowelCounter.countVowels("aa"));// I'm not counting UNIQUE vowels
  assertEquals(1, vowelCounter.countVowels("A"));// upper case
  assertEquals(5, vowelCounter.countVowels("aeiou")); // other vowels
And after each of these lines, you'd run it and update the implementation to pass the tests.

If JavaDoc is your coding standard, you'd put it in when you declare the method.

Wait, you can't be serious about this. Is this just some exaggerated mock up to prove some point, or is this really how you would go about writing a stupid method to count the freaking vowels? It would take you maybe half an hour, or even more to finish it, with all the recompiles and reruns, and all the artificial contortions you would write and then immediately delete, just so you could do it "in small incremental testable steps", for no apparent logical reason. Whereas any halfway competent programer will do it in under a minute (or thereabouts). I'm sorry, I'm just not convinced. Even if this was just a toy example, it still doesn't prove the usefulness of CodeUnitTestFirst programming, because it doesn't have even the remotest connection with reality. Trying to code anything of real-world complexity by going through such an absurd series of steps as depicted above would be simply infeasible. So, please provide a more convincing real-world example of this approach, or atleast stop using such ridiculous toy examples.

This is exactly how one would approach the problem, especially if one started out with little domain knowledge. One might take bigger steps, writing more tests at once, if one was already pretty familiar with the domain. If the overhead on your compiler for the unit testing fixture is half an hour, then you seriously need to upgrade your dev system. (And if you think vowel detection needs just a "stupid method" that you could write in "under a minute", then you haven't done much international software.)

You can find many other examples of TestDrivenDevelopment by browsing around this Wiki, many of which are more complicated.

A side note - why do you need the comment for a descriptive method like "int countVowels(String)"? -- DaveMikesell

What is a vowel? Does upper or lower case matter? What happens if there are non-ascii chars in the string? Is it counting the total number of vowels or is the count of just the vowels found regardless of the number of occurrences of each vowel? Might want some idea of the algorithm used so I can judge performance on larger strings. Just some initial questions I would have. That's what doc is for, to answer questions. -- AnonymousDonor

What if we prefaced the documentation, if only in imagination, with a disclaimer of the sort "whenever a method such as countVowels() is encountered, it does what you would expect it to do according to the PrincipleOfLeastAstonishment, unless a comment states otherwise"... If I may suggest an exercise : with respect to the above questions, state what you would reasonably expect the method to do. Then try to think of alternative method names that would be more appropriate if the method does not conform to these expectations.

For extra credit, write UnitTests that distinguish between the "normal" and "abnormal" cases. For instance: 

 public void testNonAsciiVowels() {
  assertEquals(3,vowelCounter.countVowels("Öl, öl, öl!"));
 }

Given the PrincipleOfLeastAstonishment in naming methods, and UnitTests, might we skip the "spec" and "comment" parts ? -- LaurentBossavit

Sorry I don't buy into least astonishment because everyones expectations are very different. What is acceptable is very different. I believe in stating the expected behaviour and answering any questions a developer may have when encountering a class, method, argument, block of code, etc. Almost every time I rely on "common sense" expectations I get very burned. And I don't think the method name can encode all the above information. I don't want to look at your UnitTests for a negative statement. I want to know what should happen. -- AnonymousDonor

Could you try to answer the above questions ? We might see how different our expectations actually turn out to be, and how much method names can encode, and how UnitTests would tell us what should happen.

But I honestly don't know. What should I expect? It can be done any which way all of which are valid. -- AnonymousDonor

Well, suppose you ran across the definition "int countVowels(String)" in someone else's code - say, a library class without the source code. You'd want to document it (for the same reasons you'd want to document your own code, i.e. save the next coder the trouble of asking the same questions), so you'd call the method and check out its behaviour. What would surprise you?

Speaking for myself, I'd expect the algorithm to be straightforward - e.g. iterate on the characters in the string and call "boolean isVowel(char)" on each. So I'd expect performance to be linear with respect to string size. I wouldn't bother to document this, or write a UnitTest for performance unless it was a critical section of code.

I would expect "count" to mean "the total number of characters for which isVowel() is true". I would rename the method to countDistinctVowels() if that wasn't the case. I might write a simple test case to make things clear (1 for "a", 2 for "aa").

I wouldn't expect case to be significant. Both an uppercase vowel and a lowercase vowel are vowels, to implement it otherwise would be abusing language. I might write a test to guard against an overly naive implementation.

The only aspect for which I might consider a comment warranted would be ASCII vowels vs. Unicode or international vowels, since the concept of vowel differs from language to language. Things being what they are, I would expect "vowel" to mean one of A,E,I,O,U unaccented. I would definitely write a test to clear up any doubts.

I would write at least one or two of the above test cases (within a single UnitTest) before coding the first line of the implementation... or at least, I'd like to think I would.

Would your expectations be very different?

Not to be repetitive, but I don't know what to expect. You see, this an interesting case because it actually maps to a bad experience of mine using a counting chars method. It tried counted distinct chars and tanked very badly on a large strings and bombed on no-ascii values. I like your countDistinctVowels, that would help. I wouldn't speculate about isVowel because it doesn't really matter. I would wonder long and hard about what languages it handled and how it handled non-ascii values because this could be an unexpected crash.

I can sympathize with that. It's hard to write code that works, and when you find you've written code that doesn't work you start questioning "reasonable" assumptions, in particular when these assumptions turn out to be the cause of the problem. That doesn't mean the PrincipleOfLeastAstonishment isn't valid, though. It's just that we've been kind of "conditioned" into questioning it and being suspicious of it. I find that UnitTests, and CodeUnitTestFirst, help get rid of the conditioning, which is good, because I don't like being paranoid about my code.

It's not conditioning it's reality. I could say if everyone behaved we wouldn't need jails or laws. I'm not conditioned to think people break laws. They do. I've interviewed hundreds of people. I've seen a lot of code. The majority of programmers suck and you are using their code. Even if they don't suck there is no reason to expect someone else to do something the way you would. You are having to make a lot of assumptions so as to not have to make a positive statement about what a method does. It's actually quite is easy to be specific. -- AnonymousDonor

I'd rather work on the assumption the programmers I work with are good, and document less. (I'd also want do away with jails and laws, but let's not take up that discussion here.) Your mileage may vary. -- LaurentBossavit

It's not something that can be assumed. You have to know and design process and procedures appropriately. I would like to assume nobody will commit crimes, but unfortunately I can't assume that. -- AnonymousDonor

"I don't want to look at your UnitTests for a negative statement. I want to know what should happen. -- AnonymousDonor" I, for one, would trust a UnitTest far more than whatever the function's doucmentation was. Imagine that the function was originally intended to count AEIOU, but someone later added Y along with a whole bunch of other changes (perhaps Unicode support). If the CodeUnitTestFirst philosophy were followed, there would be a test for Y along with the whole bunce of other changes. If the Y counting code was added in the middle of all the other code, would you remember to document it? I agree that you cannot always trust other people's code, but would you trust it *less* if it had passing unit tests? Would you trust code that had lots of documents if you found out an instance of a comment being out of sync with the code? Maybe that was the only out dated comment, but could you be sure? I find that unit tests to become out dated much less frequently than code docs. - cmc

One issue to be considered is that no concept is an island; it takes two to tango. If the code has a concept of a vowel, then that concept will be used in at least two places. Which of these places is the correct place to define it? Perhaps a Glossary could be maintained (on a Wiki?). (of course, that assumes that a programmer will think to look at it when faced with a seemingly obvious method name).

In fact, the introduction of a non-technical (solution-domain) concept into code should be justified by a business requirement. I believe that programmers should become conversant in the business-domain of their product (at least at a bluffer's level). This need not be achieved by documentation: it does require communication.

Hmm, this seems somewhat off-topic for the page-title. -- DaveWhipp.

And on writing the test first, who really cares as long as the tests are written? To write the tests use must already know what the method will do so the order isn't important.

On ordering: Writing tests first means you don't have to think about what you might need in the interface, only what you need right now to finish writing the test. Interfaces tend to be simpler and quicker to devise with TestFirst. -- AnonymousDonor

You have to think about it so you know what to tests to write. The order isn't relevant so the interface won't be any simpler or any more complex. -- AnonymousDonor

How would JavaDoc comment blocks on methods have prevented the coder from using an inappropriate implementation algorithm that does not scale well with large input strings? How would method comment blocks have illustrated that the implementers' 128 entry lookup table for characters was a 7 bit ASCII character assumption, and that the C/C++ program would crash (or worse) with 8-bit character values? Bad code is bad code regardless of corporate standards for header comments.

Except for static type checking you can't prevent anyone from doing anything. Prevention can't be a goal. Correct information for developers is the goal. What they do after that you can only catch with assertions or crashes. -- AnonymousDonor

One major point of CodeUnitTestFirst plus DoSimpleThings plus Refactoring is to test the tests. Once, before the interface exists, again after it exists, again after it does something, and again after we refactor. CUTT is TestFirst if it does all that. -- PhlIp

I think we are merely quibbling about terminology here. The concept in CodeUnitTestFirst is that we are really attempting to simultaneously develop both the code and test. When you are using this approach you are constantly bouncing between the test code and the code to be tested. It really doesn't matter if you first type in the test code to call a method and then type in the method declaration or if you declare the method and then type in the call code. You will then compile, test, and change both within two minutes and repeat. It doesn't matter which typing is initiated first if you follow the basic concept. -- WayneMack

Ok, I'll buy that. Allthough sometimes my iterations are larger. -- AnonymousDonor

I think it does matter what is coded first since if you don't CodeUnitTestFirst you can't guarantee the absence of FalsePositivesInUnitTests?. --anon2

UnitTests are not tested on the theory that, well, that would be silly. I don't think I am any more worried about false positives than the other hundred things that can go wrong when creating a test. -- AnonymousDonor

Actually, UnitTests are tested on the theory that incorrect code is just as likely to be in the test as it is in the code-under-test. By writing a test that fails, and seeing it fail, and then seeing it pass after the code-under-test is written/modified, you've effectively tested that the test recognizes a change in the code-under-test. Beyond that, the test and the code-under-test provide two independent statements of the intended behavior. Testing can only verify that these statements agree, not that they're correct. -- GeorgeDinwiddie

I think it matters a lot, you should code the test before the method. When you code the test first, there is no method, so you can just pretend the perfect code exists and write test code from the client's perspective. It's this shift to the client perspective that makes you think, and start writing simpler code. If you think of a cleaner interface while coding the test, you can change it immediately without wasting any work, since you haven't written the method yet. You wouldn't be so quick to change the interface if you'd already written the method. Once the test is written, it can be almost trivial to write the actual method, you just have to get a green bar. However, you only get that perspective shift if you actually write the test first. RamonLeon

An example when writing test can save you from introducing duplication (or at least help you recognize there is something unexpected going on) is when you are deriving from a class that already defines the method you are writing the test for. Example: 

 class Parent
 {
 public:


   bool method ();
 };


 class Child : public Parent
 {
 };
Now the test case: 

 AssertTrue? (child.method ());
Obviously, the test compiles and (possibly) runs correctly without your adding method() to Child. The problem mounts up if you have a pretty deep hierarchy. --MartinBilski

EditText of this page (last edited February 21, 2005) or FindPage with title or text search