Effective Junit

Usage is something you can learn with experience, while working on something for years, at a very high cost. Alternatively you learn by reading a page like this for days at a very low cost.

This page is an attempt to share my experience with you so that you can avoid common pitfalls and adhere to best practices. I borrowed the format from Scott Meyer's EffectiveCeePlusPlus and Joshua Bloch's EffectiveJava.

This page is designed to help you make the most effective use of the JUnit framework. You can read more about JUnit at www.junit.org, where you can download junit.jar and other simmilar unit testing frameworks for other languages.

Originally JUnit was based on SUnit (unit testing for Smalltalk) by Kent Beck. Unit Testing is one of the twelve practices sponsored by ExtremeProgramming and it is now called ProgrammerTests. There are differences though. The main difference is that ProgrammerTests assume that tests must be written by someone who know how the code is implemented, so for example all methods must be public in order to test them. This appoach is also known as WhiteBox? testing while traditional UnitTesting only tests the external behavior of modules (or classes in this case). This is also known BlackBox testing and it is important because it encourages clear separation of interface and implementation.

This page assumes that you are already comfortable with JavaLanguage and JunitFramework?.

JUnit Rules

1. Each public class must have a corresponding test class.

1 Common usage is that the test for class FooBar must be called FooBarTest?.
2 The test class must be in a test package and packaged in a different jar. One easy way to achieve this is to

create a src root directory for all sources in the project and another src-test root directory for all test sources. Then for each com.company.country.project.FooBar there will be a com.company.country.project.test.FooBarTest?.

2. Test the uses of the class, not its methods.

It is common for novices to test each method in isolation. That's useless, from the point of view of users of that class. What is important is how this class will be used, so for example if after I add an element to a set, the element must be there, so first write down all possible uses of the class. For example, if the class is an StringTokenizer?, there is only one use for it and that is to tokenize an String. In the case of a socket, there are many uses of it: It can be used as a service requester or as a server provider. Testing its individual methods simply makes no sense, but testing sequential combinations is what the clients will do. For each of its uses, test all important working scenarios and all important failing scenarios (which are called NegativeTest?s). Make sure you test all border conditions, besides the happy road.

3. Use assertions, do not print anything in the tests.

Tests must be run tens or even hundred of times everyday. Make them easy to verify by not printing anything if a test succeeds. When a test fails, each assertion can display a message explaining why it failed. Also a call stack is displayed indicating exactly where the test failed.

4. Make sure all tests may be run easily and quickly through ant or by pressing a button.

1 Ideally the whole project can be recompiled in seconds and then tested throughly even faster. When that's not

possible, try to separate the project in several subprojects that compile independently and that run all tests independently.

2 Tests should have enough data to prove a point and make sure the functionality is present, they do not need

realistic amounts of data.

5. When a test fails, prepare to debug.

When the failure is not obvious, it means you have a hole in the tests. Debugging is a very expensive operation, but sometimes it is necessary. Once you have found the hole, make sure you better all that information in meaningful tests. Make sure it fails, and then apply TestDrivenDevelopment: Fix it the failing test. Make sure the bug is not a repetitive bug: Make sure a similar bug is not present in other classes.

1 Debug until you figure out what the coding mistake was. Make sure that coding defect is now detected through a

meaningful test. Reducing debug time in a project can only mean achieving objectives faster. Adding meaningful tests is a very inexpensive way to reduce debug time.

6. Store the source code in CVS or another CMS (ConfigurationManagementSystem?)

I mention CVS because it is the industry standard. It has been developed and tested through too many years in thousand of projects, so it remains as the less buggy CMS around. Plus it is Open Source, meaning it has more users and therefore it has even less bugs, although SubVersion seems to solve most of its shortcomings.

Storing tests in the same CMS as the source code is really important. You should use BVT or another build verification/release management tool to verify that everything compiles and passes all tests.

7. Tests should be autocontained

Each test must be independent of the rest. If a test fails, all other tests must not depend on that test to succeed.

Therefore each test must setup all its initial data, do its operations and verify its results.

Tests that retrieve information from a database must setup that information in the database. Autocontainment also means that there must be a script to setup the database itself, create its tables, stored procedures, etc. and that that script must be run each time the tests are run.

This also means that each developer should have its own database, probably in its own computer. HSql and MySQl are two good lightweight databases to be used for development. This means that all SQL scripts must be stored in CVS and part of the repository verification must be to create the database, run the tests and destroy the database.

8. Test interfaces rather than classes

Testing interfaces is wiser than testing classes, because when you test an interface, you can switch implementations and make sure every implementation adheres to the interface contract. Testing interfaces is a great way to reuse tests.

9. Test equals and hashCode for every class

Almost all classes can be stored in hashtables, therefore all of them must be comparable and have reasonable hash codes.

10. Test clone for each class that implements Clonable

This may seem as too much, unless clone begins to fail of course.