Unit Tests Defined

Failure of a UnitTest implicates only one unit. You needn't look elsewhere to learn why it failed --PhlIp

Integrated UnitTests are not:

Manually operated.
Manually operated from memory without a script.
Automated screen-driver tests that simulate user input (that's AcceptanceTests).
Interactive. They run "no questions asked".
Coupled. They run without dependencies except those native to the thing tested (see LargeScaleCppSoftwareDesign).
Expensive. No new tools, civilians, or engineers required.
Complicated. UT code is typically straightforward procedural code that simulates an event-driver.

That said, let's build a UnitTest to test a class called eyeScream.

Create a project called eyeScreamUT, in a sub-folder of the same name, below the home folder of eyeScream.
- See below "a good folder structure" to view a tree of the folders
Inside that project, import eyeScream, construct an instance, and call a method. Now recompile.
If you can't compile due to coupling, fix eyeScream so it's not coupled to whatever it should not be. This doesn't work. You need UnitTests before you start to ReFactor, otherwise you risk breaking the code.
If you can't compile due to coherency, import whatever eyeScream needs.
If eyeScream needs external things, like a database, remove the classes that supply this to eyeScream and replace these with local spoofs that return fixed data. Again, this may well be a non-trivial ReFactor, for which you'd need to have a UnitTest before you even started.
Now add Assertion statements that prove eyeScream constructed, and it did all it was supposed to. If these Assertions pass they are silent (or optionally they state the item tested). But if they fail they halt the program (and optionally raise a Bug Enhancer on the problem statement).
The last line of the test is an unconditional print statement that says "All tests passed".
In the root of your project add a Makefile. Yes, a lowly UNIX-style Makefile. We need only just this kind of scripting host, because UnitTests need A> automated compilation dependencies, and B> automatic termination if a compile or test fails. Makefiles give both of those out-of-the-box, so don't go re-inventing any wheels here.
Inside the Makefile, do not add any of the high-tech Prolog-style twizzlers found in, for example, the Makefiles that compile Linux. Just add a dependency called 'all', and after it write the command-line commands that raise your tool and compile with it. For VC++ that's like this:

all: msdev Project1.dsp /MAKE "Project1 - Win32 Debug" msdev eyeScream\eyeScream.dsp /MAKE "eyeScream - Win32 Debug"

Notice I did not sweat over "exporting the MAK file from the project", or calling NMAKE or CL.EXE (or even changing the current folder). I got the same compiling environment as developers get inside the IDE, by calling the IDE itself in command-line mode.

Inside the Makefile, add a dependency called 'test'. It fairly writes itself:

test: all msdev eyeScream\eyeScreamUT\eyeScreamUT.dsp /MAKE "eyeScreamUT - Win32 Debug" eyeScream\eyeScreamUT\Debug\eyeScreamUT.exe

Observe we have no reason to compile UnitTests in Release mode. The hardest bugs to find are often those that only show up in Release mode; why on earth wouldn't you want your UnitTests to catch them?

Also observe the last line runs the test.

As you add more tests you just trot them out under the target 'test'. And you provide a shell short-cut to open a console and run the tests and "pause" after them so the line "All tests passed" can stimulate your pleasure centers. Then you order the crew to stimulate their pleasure centers every half hour or so.

That's all there is to it. We did it without buying more tools, or adding risk to the project, or hiring a "test engineer" (that's AcceptanceTests), or learning new languages. We got there by doing things we already know how to do, and obeying the definition of IUT:

Integrated - written by the same coders, in the same language, on the same platform, as the bugs, with support from the objects to be tested, and tuned to run automatically and with dependencies correctly built.
Unit - test one "component" (clump of classes) at a time. This permits (per LargeScaleCppSoftwareDesign) units to test in dependency order, so bugs in client objects are known to not arise in un-tested server objects.
Test - call Assertions all over the place, and JUST CROAK if they fail. Don't log anything, or e-mail anyone, or "count" or "rate" the failures. If ONE FAILURE is enough to fail all tests, and if the test watchers are empowered to fix whatever broke, then don't bother testing anything else after the failure.

Hammering these simple concepts into some programming shops' brains will allow them to not hear "resource-burning overhead and complicated 4th-party tools" if you ask them with feigned innocence "do you UnitTest?"

I have today fooled around with ways to do the above a bit simpler using Visual C++ 6.0, (I don't know how to use and create makefiles under Windows). I will describe what I did here so that someone can tell me if I am missing the boat, and because it might be of use to others.

I have added another project called Tests to a Demo project so that there now are two projects in the Class View. Tests are the tests that need to be done on the classes in the Demo Project. A failure of a test causes Tests to write an error to a console (printf or cout). And the Tests executable returns a 1. You will see where this goes shortly.

I have made the Demo project Dependent on the Tests project and I have added the executable of the Test project as a pre-link step in the project settings of the Demo Project.

The result of all this is that all the output of the Tests executable is printed in the compile window and that the Demo project won't link if the Tests do not work.

I think this will basically be as functional as the process with the makefiles.

If someone wants to see the VC workspace, please leave a note on my WikiPage and we can make a plan.

After some hacking I got CppUnit integrated into the VC++ 6.0 IDE as well by using the above method. It works great and I am definitely converted to UnitTesting. I am still working on the powers that be to start implementing the rest of XP.

-- LourensCoetzer

Projects fully involved with UnitTests should work from a completed framework such as TestingFramework, CppUnit, JavaUnit, etc. For Cpp, you can also use the test unit that's built into the BoostLibraries. And if you don't have Boost, you ought to get it anyway.

UnitTests permit fast & safe RefactorMercilessly sessions, and are a natural complement to DesignByContract.

Do UnitTests have to be deterministic? (I.e. You set up a fixture, provoke some canned behaviour and then test the result. Must the behaviour itself be deterministic?) --

Speaking as one who has had to UnitTest mail systems, you'll really hate yourself if the test only works most of the time. I kept coming up with excuses not to run those particular tests, because they were a) incredibly slow and b) failed about 1/6th the time because of subtle timing weirdness with the mail daemons. -- GrahamHughes

Their results have to be "testable", not "deterministic". If a function produces a pseudo-random number, you can do a few quick tests on a sample of results to see that it has a linear spread, a given range, etc. -- PhlIp

The results have to be reliable. - If they say pass, they mean pass, and if they say fail, they mean fail. When testing a unit that interfaces with unreliable or unpredictable subsystems, you have to fake those subsystems to give predictability to your UnitTest. -- WayneConrad

To discuss UnitTest isolation factors, see StandardDefinitionOfUnitTest.

To discuss unreliable subsystems, see ExtremeProgrammingChallengeNineteen.

I realize this is probably my naivete with the msdev tool, but I'm confused how to do this one thing: "provide a shell short-cut to open a console and run the tests." What is a "shell short-cut"? I assume that it's some way to invoke nmake via batch file?

: I believe it means creating a short-cut on your desktop that invokes said make command. Thus, just double-clicking on the short-cut launches the tests.

Install IE4 before you go to IE5, or get Win98 or Win2000, and you get the Quick Launch bar, typically between the Start button and the Task Bar. Drop the icon there, and it's generally always available.

If you're sufficiently clever, you can use the Post Build Commands to do this for you. Then, every time you compile you will test.

So now, the question is how do structure the folder structure of the tests. I don't want to interminge code and tests because that distracts me from my code. I also don't want to hide my tests because that slows down the process.

See UnitTestFolderStructure.

Does this page yet contain a definition? There's a list of things "Integrated UnitTests are not", then a very platform-dependent example, and a lot of interesting discussion, but nothing else that looks like a definition.

Is the not-list supposed to be a sufficient definition?

-- DavidVincent

Listen, at a new job, to the following whining...

"We can't do UnitTests. We don't have...

enough resources (people)
a language that an existing xUnit framework adapts to
enough time to write them
a UnitTest guru who knows how to do coverage analysis
Brand X's latest smarty-pants commercial UnitTest framework
an XP culture with all the other trappings
enough time to research all the Wiki pages about UnitTests"

The "definition" is the minimum needed to JUST START WRITING THE TESTS. No framework, gurus, 4th-party tools, light methodology, or excess overhead.

I recently read about PairProgrammingCostsBenefits, and this discussion makes me yearn to hear about the UnitTestingCostsBenefits. -- RobHarwood

More than a year on, I regret asking about the definition. At the time, the "discussion" above was helpful in that it spurred me to learn more about UnitTesting, XP and Wiki. Definitions are available at UnitTests and StandardDefinitionOfUnitTest. I still don't think this page's name reflects its contents very well. -- DavidVincent

See StandardDefinitionOfUnitTest, UnitTestCookbooks

CategoryTesting