Enhancing Junit

This page is intended for discussion of possible enhancements/fixes to JUnit (whose WikiName is JavaUnit). Implemented enhancements should be added to the JavaUnit page.

If you're testing web applications, you could try ServletTesting, perhaps using some tips about ServletUnit. Or, you could test the web server response directly with HttpUnit.

JunitTwo?, JunitThree?, JunitThreeTwo?

Maybe there's some way that JUnit itself could be enhanced to help with JournallingPattern testing. -- DavidPinn?

I have just struggled through the issues concerning the testing of abstract classes and their subclasses. I am wondering if a software tool, perhaps delivered as an enhancement to JUnit, could ease the way a little.

In part, the complexity arises because of polymorphism. Consider abstract class Telephone. I would write a trivial subclass of Telephone that does not override any of its methods, and use it to test all the non-abstract methods of Telephone.

Now consider MobilePhone, a concrete subclass of Telephone. In addition to testing all the methods in MobilePhone, I also need to retest any methods of Telephone that polymorphically call methods of MobilePhone. Keeping track of which superclass methods to retest takes some brain energy. The problem is exacerbated when the inheritance tree is several levels deep. I'm wondering if a program could do the work for me: give me a list of the methods that I need to test. In so doing, the program would help with the more generic problem of determining test coverage. Could it be activated by a button in JUnit's TestRunner? -- DavidPinn?

Here's another enhancement that would be good. Two years ago, back before Erich and Kent wrote JUnit, I stole all of the good ideas that were in Kent's original Smalltalk testing framework and helped one of my customers build a Java version of the same thing. A cool feature they added on their own was a tree-view (built using Swing) that showed a composite test "split out" into its pieces. You could see results or failures at any level that way. You could also choose to re-run tests at any level in the tree. Sigh... too many ideas, not enough time...

-- KyleBrown

Something like this: http://www.voelter.de/rad/software/junit/right.html ? -- ErikMeade

In the version of JUnit2 that I have, the constructor of TestSuite that takes a class uses getDeclaredMethods to look for the list of test methods (using the naming scheme above). By changing to getMethods, one can get the list of matching test methods not just in the current class, but in any superclass. By building a test class hierarchy that mirrors your class hierarchy, you can now test your entire hierarchy without rewriting a lot of tests. Kent and Erich and everyone else, what are your thoughts on this? -- HankRoark

Erich and I were talking about this today. There is a danger in using getMethods() that you'll accidentally pick up stuff you don't want. But you can't have abstract test case classes with concrete test cases in them. We'll probably default to the version with inheritance, but offer the current behavior in a separately named static FactoryMethod. -- KentBeck

Just a minor change: Adding "Main-Class: junit.ui.TestRunner" to the manifest file of junit.jar makes it possible to run the graphical TestRunner by double-clicking the jar file. -- YonatSharon

I have added two things for our own (proprietory) convenience to JUnit 2.1:

You can find these (simple) enhancements at http://www.riehle.org/sources/junit-21/index.html.

If you like them (or not), let me know! -- DirkRiehle (June 7, 1999)

I have written extension to JUnit that allows tests written for CppUnit to be run by the JUnit user interface. It has a code generator that turns your CppUnit test into a shared library that can be loaded into a JUnit ui via a Java class that is also generated. All you have to do is write your tests, run the generator, and compile. If you want a copy, email me at: mailto:pgoodwin@ix.netcom.com and I'll send you the .zip (it's about 1MB). -- PhilGoodwin

I need a set of UnitTests that consist of two phases that have to be executed in separate Java VMs. (This is to test some persistence stuff.) Although I can figure out a way to do this, I'd be interested if anybody else has already done this before. After all, I'm a lazy person. If not, I'll publish my code when it's less buggy than now. -- HaskoHeinecke

[Added Nov 19, 1999 - JohnGoodsen]

Maybe I'm just missing something here, but what I'd like to do is build a test suite that runs several test suites (e.g. I want to run tests on every class in a particular package at one shot). What I've got is a "master" test suite TestAll?. In TestAll?, I add each UnitTest suite (one for each class being tested).

public class TestAll? extends TestCase { 

public TestAll?(String name) {
super(name);
}


public static Test suite() {
TestSuite suite= new TestSuite();
suite.addTest(CurrencyTest?.suite());
suite.addTest(EntryTest?.suite());
return suite;
}
}

The issue with this approach is that the compiler forces me to have the .suite() operation on each TestCase class defined apriori in order for TestAll?.suite() to compile. The result is that I don't get to use the nifty reflection based identification of test methods that are prefixed with "test" in each test class, but rather I must maintain the .suite() operation to add each test case operation when it is added to the class. I think I'm looking for an operation like TestSuite.addSuite() that will do all of the nifty reflection work and dynamically build those .suite() operations on the fly.

Am I missing something, or am I asking for more features in JUnit release 2.x or 3? :-) Other than that, I am loving JUnit! -- JohnGoodsen

You just add a method 

  public static Test suite() {
return new TestSuite(CurrencyTest?.class);
  }
to the CurrencyTest? class, and similar for all test cases. This constructor of TestSuite does the nifty reflection work for you. You then just add a testXYZ() method, and it is automatically recognized and included. -- MarnixKlooster

Or you could change your suite method to 

  ...suite.addTest(new TestSuite(EntryTest?.class));...
and avoid the need for the subsidiary suite() methods entirely.

Actually, both approaches have to be used in order to get tests to run that are embedded in suites that are themselves inside of suites.

Example code that solves both problems:

public class TestAll? extends TestCase { 

public TestAll?(String name) {
super(name);
}


public static Test suite() {
TestSuite suite= new TestSuite(TestAll?.class); //Put in this class
suite.addTest(CurrencyTest?.suite());
suite.addTest(EntryTest?.suite());
return suite;
}
}

If the this.suite() always adds the current class in the TestSuite constructor, then all local test methods are added dynamically.

Doing 

suite.addTest(new TestSuite(CurrencyTest?.class);
suite.addTest(new TestSuite(EntryTest?.class);
fails to add any addTest's in the suite methods of EntyTest? and CurrencyTest?.

Is the logic here solid?

-- lby@bigfoot.com

Someone has convinced the US government to fund SoftwareCarpentry, the development of a set of four easy-to-use open source tools. One of the tools is "a unit and regression testing harness with the functionality of XUnit, TclExpect, and DejaGnu."

While TestSuite extracts inherited test methods, TestCase.runTest won't find them.

I've fixed this (and the corresponding test case), you can find the patches at http://cc.bost.de/junit/ (apply 

 patch -p0 < patchfile
) The same patches modify junit.tests.ThreadTest? and junit.swingui.TestRunner as JDK 1.2.2 didn't want to compile those.

junit.tests.TestTest.testRunnerPrinting fails for many non english locales - but I don't know how to fix this without making the test trivial. -- StefanBodewig

While talking about enhancing junit, has anyone noticed that the 3.x TestCaseClassLoader? makes a questionable use of the available method in its loadClass method? It probably gets away with it, as it loads from disk, but this method says it returns the number bytes it can read without blocking - if you disk was busy this might not give you all the bytes you need right? -- TimMackinnon

Here are a few items from my JUnit wish-list.

If I get the time, I may implement them myself...

-- NatPryce

I've hacked a version of junit.textui.TestRunner to create junit.servlet.TestRunner for those of us who want to be able to run our tests through a browser. Please note it is NOT pretty and needs refactoring, but it might be useful. The source is at http://www.mcbreen.ab.ca/pete/TestRunner.java. A more complete implementation would use the swingui as a starting point since it is a pain to have to restart the webserver just to get the next version of a class loaded. -- PeteMcBreen

Alternatively, you might want to try the servlet and TestRunner that I wrote before I discovered that someone already did this. :-) It has reasonably attractive output, will run multiple tests at once, and it uses the dynamic classloader of your (presumably intelligent) app server. The TestRunner can also be used standalone if for some reason you wanted a html file directly. I've called it JunitEe and it can be downloaded at http://www.infohazard.org/junitee. Also, I'm willing to entertain enhancement requests if anyone has any ideas. :-) -- JeffSchnitzer

The LoadingTestRunner? seems to be too eager to reload classes, even if they didn't change. This becomes a problem especially with static attributes, which get reseted. Look at this example classes, which fail if I use a LoadingTestRunner?, but work fine with a simple TestRunner. 

  import junit.swingui.LoadingTestRunner;
  public class Main {
public static void main(String[] argv) {
  Main.counter = 10;
  LoadingTestRunner?.main(new String[] {"StaticMemberTest"});
}
public static int counter = 0;
  }




  import junit.framework.TestCase;
  public class StaticMemberTest extends TestCase {
public StaticMemberTest(String name) {
  super(name);
}


public void testCount() {
  assertEquals(10,Main.counter);
  Main.counter++;
  assertEquals(11,Main.counter);
}
  }
It doesn't help to create an object of Main in the beginning with "Main obj = new Main();". Even though there is a reference to an object after that, the class still gets reloaded.

I encountered this when I wanted to pass some arguments from the command-line onto some tests and while using a singleton (Yes, SingletonsAreEvil) another time. -- MarkoSchulz

Sadly, the custom classloader used by the LoadingTestRunner? seems to be getting a major hassle for us. Today, it was the cause for a ClassFormatError while using CORBA. :-( -- MarkoSchulz

I modified junit.textui.TestRunner so that instead of printing a whole stack trace when a test fails, it just prints the file & line of the failed test. Helps a lot if you have tests that fail. I've submitted the patch back to the JUnit authors. -- NelsonMinar

I would like to see an improvement on LoadingTestRunner? which allows it to load classes from .jar files. Right now I would need to expand all my CORBA jars to test parts of the base layer, which would be hassle.

[added October 4, 2000]

I made a modification to Assert.java, because it did not do what I expected when I compared two double values. If one of the values is Double.NaN, then the assert will not call failNotEquals.

I made the following change:

before:

static public void assertEquals(String message, double expected, double actual, double delta) { 

  if (Math.abs(expected-actual) > delta)
failNotEquals(message, new Double(expected), new Double(actual));
} after:

static public void assertEquals(String message, double expected, double actual, double delta) { 

  if (Math.abs(expected-actual) > delta || Double.isNaN(actual))
failNotEquals(message, new Double(expected), new Double(actual));
}

Rick Ratliff 

  /**

  public void addTest(final Class testClass)
  {
Test suite= null;
try {
Method suiteMethod= null;
suiteMethod= testClass.getMethod("suite", new Class[0]);
suite= (Test)suiteMethod.invoke(null, new Class[0]); // static method
}
catch(Exception e)
{
// try to extract a test suite automatically
suite= new TestSuite(testClass);
}
addTest(suite);
  }
-- JeroenMostert

-- RobertHellwig?

I was wondering if some knows a way to perform a time 'critical test'. I'm using Junit to test my JavaSpaces stuff and I was wondering if I can set up a test that will fail if a certain object does not arrive in a specified time frame.

-- RobertHellwig?

I wrote a general class "UnitTestGenerator?.java" that you can point at a directory tree of java source, and it will traverse the tree and generate a unit test class [ stub - assert(String, FALSE); ] for every Java source file (that doesn't end in "Test.java"!), and a PackageTest?.java for every package, and a top-level SystemTest.java that calls the tests from every package.

I've found it useful, especially under an environment like JBuilder, where I can right-click on any test and run it, or run on an entire package, or across the whole system.

It's not perfect, but it is documented, and it works well. My question is - where can I put it for others to access?

regards,

Jeff Kyser

kysers@bellsouth.net

I have the same question. I've written 3 additions. A TestCase that creates a TestSuite from every class in its package (and nested packages) that extend TestCase, a TestCase that makes testing MT-safety a little easier in JUnit (since calling fail () in a non-main thread doesn't do a lot) and and a fix that stops JUnit swallowing exceptions from TestCase constructors.

AndySchneider

Hi there,

About 18 months ago, I started an OpenSource project called "simscomputing.Test Bed" (http://testbed.simscomputing.com) with the same goals as JUnit. Sometime in the summer of 1999, I discovered JUnit. I had not previously known about JUnit. At the time, I felt that my GUI and APIs were, frankly, better, and I kept developing the Test Bed.

Since then, I think the JUnit APIs have caught up and perhaps passed Test Bed. But I tried the JUnit GUI yesterday, and Test Bed's GUI is still much better.

So I wanted to talk to you guys about a merger. It'd be nice if my GUI ran JUnit unit tests. And it'd be nice that if you were a Java programmer, you could write unit tests that ran in either testing framework.

What would you think about abstracting out the JUnit and Test Bed APIs and create a small set of interfaces? Then Junit can implement those interfaces and so can the Test Bed. Then my GUI would work with both JUnit and Test Bed unit tests.

If we could simply agree on a few interfaces, then I think the whole Java unit testing community wins.

Anyway, I'd like to open a dialog and see if we can come to some kind of merger agreement where we take the best of both frameworks and avoid our duplication of effort. I'd also like to move simscomputing.Test Bed to sourceforge.net for a more open development process.

Thanks, David Sims david@simscomputing.com

For the last couple of years, I've been using a piece of code taken from JavaWorld (I think) which takes a directory name, loads all java classes from that directory, then runs a test for any class which implements TestCase. In this way, I've not had to create any suites or explicitly call any TestCase classes.

It would seem sensible for this functionality to be included in JUnit itself, particularly since JUnit already uses Reflection to determine test cases within a particular class - this would just determine them within a particular class hierarchy.

-- IanMayo

Refactored JUnit: SuiteRunner

See: HelpWithJavaUnitProblems

EditText of this page (last edited June 6, 2006) or FindPage with title or text search