Unit Tests Tell You When Youre Done

UnitTests tell you when you're done.

: Many programmers don't know when they are done. They keep implementing long after the thing they need is sufficiently complete. Unit tests help you know when to stop - when you can't think of anything relating to your task to test. And when you are refactoring, your refactoring works when the tests run again. -- RonJeffries

The question naturally arises, how do you know when you've written enough UnitTests? Strictly, unless you can enumerate the entire domain of a function, then you haven't written enough tests. The aim for a complete UnitTest is for complete functional coverage.

When you CodeUnitTestFirst, you know that you're done when...

All your UnitTests pass.
You can't think of any other tests to write to meet the requirements you are implementing.
...unless the CodeSmells, in which case you'd refactor it. (See WhatIsRefactoring?)

If you continue writing code after that point, then you're wasting your time; no additional functionality is needed.

I submit that FunctionalTests tell you you're done. One doesn't need to test every class to know that the XML parser is complete. One just needs to feed in enough cases of XML to know that it is working. The entire subsystem then can be verified to a sufficient level of confidence without verifying each method.

: Why then are you bothering to verify the subsystem? Won't the FunctionalTests for the whole system reveal any problems in the subsystem?
: In this scope, we're not looking for problems. We're looking for completeness. Different problems. There is a difference between verification and validation. Verification aims for defect-freeness (consistency), whereas validation aims for requirements satisfication (completeness). So, yes, strictly speaking, one would only need the topmost tests. Users don't care if the networking module works, but whether they can send and receive e-mail.

Moreover, this is much easier to work with, because most well-written subsystems use a narrow public interface (see NarrowTheInterface, FacadePattern) to which they must code to. Everything beneath that interface is inconsequential to the rest of the world. Thus, having the friction of functional equivalence at these lower layers is pointless. Testing beneath this layer should not be strict nor restrictive with respect to the interfaces of classes because those interfaces should be pliable in order to work fast. In other words, you don't gain much by doing complete testing, but it costs you in time while you must continually fix tests as you rework the implementation.

FunctionalTests do a good job of telling you what functionality is present, but won't tell you how reliable and bug-free the software is. Like, XP customer-specified FunctionalTests probably wouldn't test to see what happens when the file contains invalid, unmatched, or otherwise malformed tags. UnitTests will, because they're typically written at a lower level of abstraction, where those issues are more obvious.

See above re: validation vs. verification. You're right regarding verification but that's not the debate here. Actually, you can test invalid XML files at the functional level. I'm not really sure that's the best example you could have provided. Perhaps, say, testing against errors in the Java Virtual Machine if you are writing a secure application? Maybe the line between functional and unit tests is weak.

"When you're done" leaves out an important parameter: what you're doing.

A single FunctionalTest tells you when you're done implementing some large piece of functionality the customer wants. Your whole suite of FunctionalTests tells you when you're done implementing everything the customer wants (and has expressed as FunctionalTests).

Now assume you're doing CodeUnitTestFirst (otherwise, you don't have the UnitTests for what you're working on yet, so of course they can't tell you when they're done):

A single UnitTest tells you when you're done implementing some very small piece of functionality, which needn't be directly visible to the end user. The collection of UnitTests for a given Unit tells you when you're done implementing the Unit. The main insight I gained by trying it was this: you only end up changing the UnitTests for the Unit you're changing, and (since you're changing the tests just before you change the code) it's only a tiny bit more work than just changing the code.

The "friction of functional equivalence" should be empirically measurable. The claim of ExtremeProgramming is that it's lower than the cost of not having UnitTests.

: True. More information is required to resolve this problem.

So is the question settled, then? Do we all agree that: "A single UnitTest tells you when you're done implementing some very small piece of functionality"?

Finally, UnitTests can have bugs too. Sometimes there are BugsInTheTests!

This discussion would be easier for me to follow with an example.

How about chapter 14 of ExtremeProgrammingInstalled?

: Many programmers don't know when they are done. They keep implementing long after the thing they need is sufficiently complete. Unit tests help you know when to stop - when you can't think of anything relating to your task to test. And when you are refactoring, your refactoring works when the tests run again. -- RonJeffries

Not wishing to knock it if it works, but why does it work? How is this different from the implementation tells you when to stop - when you can't think of anything necessary for the spec that you haven't implemented? -- DanBarlow

One counterargument is that FunctionalTests tell you you're done, and as a complete subsystem has a narrower interface than the units comprising it, are better than unit tests because they don't get in the way when refactoring the internals of the implementation.

Testing beneath this layer should not be strict nor restrictive with respect to the interfaces of classes because those interfaces should be pliable in order to work fast. In other words, you don't gain much by doing complete testing, but it costs you in time while you must continually fix tests as you rework the implementation. -- SunirShah

Serious question: how do you unit test a UI? Is it user experience testing at that point or is there a programmatic framework I don't know about yet?

Read up: TestFirstUserInterfaces. Because Views are the only part of a program that you "look at", they are high-bandwidth but low feedback. Tests can't check for esthetics or usability, but they _can_ preserve these at refactor time. And, at their highest level, tests can help you build-out new forms and widgetry without excessively stopping to "look at" your view.

CategoryTesting