Abstract Test Cases

See JavaUnitBestPractices, AbstractTest and TestingPatterns

I have begun calling these "Contract Tests", because they describe the contract that all implementors or inheritors must respect. Violating those contracts violates the LiskovSubstitutionPrinciple, and we all know that's just bad.

I use Contract Tests very aggressively to support Isolation Testing (testing objects in total isolation from the implementation details of their collaborators). I tried to write a good example, but it wasn't good. I'll try again later. The expected results in the Contract Tests for interface X become the assumptions I use when testing class Y that uses interface X. This is especially useful in one general area: business logic and the database.

I generally introduce a Repository interface to hide the database. I test-drive the database-aware implementation, but pull up the general "push and pull data" tests up as Contract Tests for Repository. These Contract Tests now describe the assumptions I'm allowed to use when I introduce fake or mock Repository objects into business logic tests.

DaleEmery once wrote that when he uses only Isolation Tests he sees disagreements between what objects do and what their clients/users expect them to do. Good Contract Tests help me avoid this problem so much that I rarely use Integration Tests or end-to-end tests for myself any more. I let my Customer write them, but I generally don't care. -- JbRainsberger

An Abstract TestCase is a TestCase for an AbstractClass that ensures that concrete implementations of the abstract class behave as expected.

The Abstract TestCase will have abstract methods to enable it to obtain concrete subclasses of the Abstract class under test, to obtain appropriate arguments for tests and expected results.

JbRainsberger put it well when he said:

This kind of test case ensures that concrete classes do not violate the contracts of their superclasses.

A suite of AbstractTestCases are available here: http://sourceforge.net/projects/junit-addons

Contrived Java Example:

 /**
  * A source of messages (serializable objects).
  * Implementations may be JMS queues, file systems, etc.
  */
 public abstract class Source {
/**
 * Receive a Message from the Source.
 * @param timeout length of time in ms to wait for a message
 * @return a message or null if the source timed out
 */
public abstract Serializable receive(long timeout) ;
 }

 public abstract class AbstractSourceTestCase extends TestCase {

/**
 * Get the Source to test
 */
public abstract Source getSource() ;

/**
 * Prepare and get a message expected from the Source.
 * e.g. put a message on to a JMS queue so that 
 * a JMS Source will then produce it.
 */
public abstract Serializable prepareAndGetExpectedMessage() ;

public void testMessageReceived() {
 Serializable expected = prepareAndGetExpectedMessage();
 Serializable received = getSource().receive(1000);
 assertEquals(expected, received);
}

public void testNoMessageReceived() {
 Serializable received = getSource().receive(1000);
 assertTrue(received == null);
}
 }

OK, so the above example is a little contrived but is based on something I have written and found very useful. My 'real' Abstract TestCase has about 9 test methods which would have been replicated for all the tests for my implementations - lots of effort saved and it has caught a number of subtle bugs.

Given that an abstract class defines the behaviour of concrete implementations, an Abstract TestCase tests that behaviour and ensures that implementations behave as expected. It also helps developers build new implementations - the new testcase implementing the Abstract TestCase helps them to do it much more easily.

-- ChanningWalton

Discussions

ChanningWalton wrote:

: This technique is not applicable to all AbstractClasses as one expects different behaviour in concrete implementations that might make it impossible to use this technique.

Someone responded:

: If you follow the LiskovSubstitutionPrinciple, then this should not be a problem

ChanningWalton:

: Indeed. In fact I am struggling to find a case where AbstractTestCases wouldn't work, it would be nice to have a list of exceptions if any. ChanningWalton

JbRainsberger:

: This kind of test case ensures that concrete classes do not violate the contracts of their superclasses. Assuming there is no extra behavior to test, this is sufficient. If there is more to test in the concrete class, then there needs to be an additional test case for that extra behavior.

ChanningWalton

: Exactly so! That's what I wanted to say and was unable to find the words :-) When I have a class that implements two interfaces, I have had to write two or maybe three TestCases for it, one for each of the interfaces it implements (a TestFooAsX and TestFooAsY), and another test for anything else.

(JbRainsberger) Real-life example: I wish IBM/VAJ would have been more careful. I was working with EJBs/Access Beans and wrote a test.

public void testFindOnlyOne() throws Exception {
MyAccessBean finder = new MyAccessBean();
Enumeration e = finder.findByUniqueIndex();// Expect one row
assertTrue(e.hasMoreElements());
MyAccessBean first = (MyAccessBean) e.nextElement();
assertTrue(!e.hasMoreElements());// Shouldn't be any more.
try {
e.nextElement();
fail("There's more?! You just said there wasn't!");
}
catch (NoSuchElementException? success) {}
}

This test failed. Why? com.ibm.ivj.ejb.runtime.AccessBeanEnumeration does not respect the contract of java.util.Enumeration. When hasMoreElements() returns false, nextElement() returns null instead of throwing NoSuchElementException?. In short, their "enumeration" is not an Enumeration, even though they implement that interface.

An abstract test case enforcing the contract of java.util.Enumeration would have helped here.

ChanningWalton

: Perhaps API designers should include abstract test cases too ;)

JamesAbley

: I completely agree with that last comment. I recently had to implement a JSR and frequently found myself wishing that it was specified in terms of tests rather than a large document that I had to repeatedly reference and make judgement calls about what the authors intended.

CategoryTesting: TestingFramework for Java. CategoryJava