Unit Test Trial Four

As advised on many pages here, I wish to implement UnitTests for all the interfaces of each class in the system. But I have encountered a problem which seems to make this kind of detailed UnitTest a lot more difficult. As usual, I seem to have several options. I'm programming in Java, and using JUnit for my unit tests.

The difficulty occurs when some important interface has only partly defined semantics, and is by definition free to vary in important ways. As a concrete example consider the case of iterating through the keys of a Java Hashtable.

The "keys" method returns an Enumeration which steps through all the keys in the Hashtable, and it is important for my test to make sure that all (and only) the values I expect to be there are present. However, it is part of the definition of a Hashtable that the order of the enumerated keys is undefined.

For normal uses this is unimportant, client code just does what ever it does to all items in the table, but for a UnitTest it seems very important. A naive test such as the following may, but probably won't, work:

  public void testHashtable()
  {
    Hashtable h = new Hashtable();
    h.put("first", "apple");
    h.put("second", "banana");

    Enumeration e = h.keys();

    assert(e.hasMoreElements());
    assertEquals("first", e.nextElement());
    assert(e.hasMoreElements());
    assertEquals("second", e.nextElement());
    assert(!e.hasMoreElements());
  }

My options seem to be:

1. Try the test above, if it works - great, if it doesn't - re-order the tests until it does.
2. "Improve" the offending method to provide fully defined semantics, such as returning a guaranteed ordered Enumeration.
3. Write a utility helper which returns a sorted version given an Enumeration, and call that with the result of the "keys" method before the asserts.
4. Implement some sort of "checklist" class in which I can "check off" that each expected value has been received, and then perform asserts on that class.
5. Not bother with the test, it's just too hard.
6. None of the above.

I can see problems with some of these approaches:

Option 1 is easy, but very fragile. It depends implicitly on an undefined part of the interface and could easily blow up on the next refactor.

Option 2 seems on the one hand to follow the ExtremeProcess, but I have niggling worries about the idea of having any code in the product which is only used for UnitTests, and not used in real operation. The code tells me it's not wanted, but the tests say it is.

The TestingFramework is an example of code which is only used for UnitTests. Test-supporting objects are quite OK ... tests should be well-crafted just as "real" code should. Refactoring of test support may lead to new objects or to support methods on the objects being tested. Generally just let it happen. Consider using a separate testing protocol.

I guess my original problem statement was not clear enough, or I'm misunderstanding you now. The difference between option 2 and option 3/4 is that for option 2 the code under test is modified (in effect the interface is respecified to define and order to the enumeration) adding more code to the delivered system. In options 3 and 4 extra code is only added to the test framework. --FC

Options 3, 4 and 6 seem possible.

Option 5 is a troll.

What does the panel think?

-- FrankCarver

PS. I don't know if anyone else finds these explorations useful. I know I do.

---

Don't bother with the test unless the Hashtable implementation supplied by your Java vendor seems to have bugs. Sun has (hopefully) already tested it for compliance.

Sorry if I misled you. My choice of Hashtable was only to use an example which should be widely familiar. My actual example is a more complex input-stream parsing class which (at the moment) stores its results in a private Hashtable, and presents the results as an Enumeration. The real test might be something like:

  p1.load(new StringBufferInputStream("some text to be parsed ..."));
  Enumeration e = p1.getTopLevelTags();
  ...

--FC

---

Are the constraints to be tested for something like:

1. Each element of the keys occurs once in the enumeration;
2. Each element occurs only once;
3. No other value occurs?

Then could we not construct a Hashtable containing known keys and test those very constraints?

Further, could these constraints perhaps be stated as a single constraint on the set of known keys and the set of keys enumerated? --rj

Am I misunderstanding you, Ron? This sounds very similar to my option 4 to me. say:

  Checklist c = new Checklist(new String[] { "first", "second" });
  Enumeration e = h.keys();

  while (e.hasMoreElements())
  {
    c.check(e.nextElement());
  }

  assert(c.allCheckedAtLeastOnce());
  assert(!c.anyUnknownItemsChecked());
  assert(!c.anyCheckedMoreThanOnce());

Looking at this I quite like it as a general purpose tool. I think I'll code it up this afternoon. --FC

Well done, to both of us. ;-> --rj

---

1. "Improve" the offending method to provide fully defined semantics, such as returning a guaranteed ordered Enumeration.

A problem you have with this option, was that you would add code which only servers testing. This sounds weird at first, but in some situations it is absolutely normal. Producers of chips always add logic to make it possible to find errors in the production process, see ChipTesting?. Once I made such a thing myself, and one error we made (as I can see now), is that we did not add logic just to make simulation faster. This is, we should have modified the chip itself, just to make it easier to simulate it while designing the chip.

---

Any reason you don't do the following?

 {
        // hashTable contains the same elements of testList
        Iterator iterator = hashTable.iterator();
        while(iterator.hasNext())
                testList.remove(iterator.next());
        assertEquals(0, testList.size());
 }

---

CategoryTesting