Help With Java Unit Problems

A place to talk about problems and solutions to various JavaUnit issues:

Should Each Test Method Be Forked?

I'm sure someone else must be having this problem. Is there a version JavaUnit that runs each test in it's own JVM? I try to tearDown and setUp test state before each test, but the problem I have is with singletons and other static data. As a result, I end up putting a bunch of smelly methods in to reset static stuff when it seems like the proper answer is to run each test in it's own JVM.

Has anyone done this?

Let me give you an example. Say I have a registry of meta objects that map java objects to persistent storage. Say the registry is a singleton. When I register a new mapping, it looks to see if the mapping has already been cached. If it has NOT, it looks in the database and ensures the DDL for that mapping. I want a fresh database for each test, so I create an Abstract Persistence Test and have it's setup and teardown methods drop and create the database.

Well, first problem, the registry is a singleton that caches objects. When I re-register mappings in setup, the registration will be cached but the DDL will not be created because I dropped and recreated the database. So now I have to clear the registry cache on each teardown. Of course, this violates the design because the registry should not be able to be cleared, but what the hell, I add a #clearCache method to the registry (opening up the system to bugs, i.e. if someone calls clearCache in production code).

But wait! That's not all! I also have code like this: 

 class MyPersistenceObject extends ....
 {
     private static final Mapping sm_mapping;


     public static final Mapping getPersistenceMapping() 
     {
         if ( sm_mapping == null )
         {
             sm_mapping = PersistenceManager.registerMapping( 
                 new MyPersistenceObject.Mapping() );
         }


         return sm_mapping;
     }
 }
So NOW, I have to add a kludge to each of these classes to reset their static state! Another smell. All these smells when the clean answer is to run each test in it's own JVM.

Thoughts? Does anyone have a version of JavaUnit that runs each test atomically? -- RobertDiFalco

Lots of folks have problems unit testing singletons. I think the most popular solution is to stop using singletons or use a factory that can produce mock objects during testing.

Seems like a lot to go through when just running each test in it's own JVM would solve the problem. While singletons may not be an issue, what about resources that should only be allocated on JVM startup? These too would need to be handled. In fact, I'm not even sure how a factory or mock object could help the problem.

If it's easier to make JUnit run each test in its own VM, do that. If not, you can make a factory for the singleton with two modes. One produces the same object on each call, the other produces a unique object on each call.

Okay, here's one example. Say I have a composite structure with a root node. Somewhere I have a method like this: 

 public static Group getSystemRoot()
 {
     if ( sm_root == null )
     {
         sm_root = lookupRootInDatabase( .... );
     }


     return sm_root;
 }
The root group is immutable and I don't want to have to look it up each time. Of course, if I kill the database in between test methods, the root may be created with a different unique identity so I can't use the cached version. Now, I could put something fancy into place like this: 

 public static Group getSystemRoot()
 {
     return ObjectCache.getInstance().lookup( "System.root" );
 }
Make sure all my singletons use the object cache and in the tearDown method I could call: 

 public tearDown() throws Exception
 {
     ObjectCache.getInstance().clear();
 }
The only problem is that I have put in a bunch of infrastructure and introduced a hash look up for all cached data for no other reason than to make the system more testable. The overhead isn't required for anything else. Furthermore, I would have to redesign my existing code to use the new object caching facility. Are there other solutions, or is this what most people do?

what about resources that should only be allocated on JVM startup?

Sounds like something other than a UnitTest if the tests are dependent on that. With the sort of ubiquitous use of SingletonPattern the code is demonstrating, perhaps the smell isn't in the tests? One of the values of UnitTests is that they help a programmer discover areas where the design (as implemented in the source, of course) is problematic. Perhaps the difficulties in testing described here are a hint the code is trying to tell you something? ListenToTheCode.

Well, it could also be a smell in the unit testing infrastructure. After all, if you find yourself straying from the simplest thing in order to make things work with a test infrastructure, I would ListenToTheCode and suspect the testing infrastructure. Most large, scalable, inter-process systems require a good amount of cached data to perform adequately. This is different than the will-nilly use of singletons. Remember, when all you have is a hammer, everything looks like a nail. When using a system that does not atomically handle static state to test your system, static state ends up looking bad. For what it is worth, how would you handle the code fragment above? Would you just never cache the root node and retrieve it from the database each time?

I would set the ObjectCache?'s state at the start of every test. -- EricHodges

So, you would introduce the ObjectCache even though the only thing you needed it for was testing? Why introduce the extra overhead if it would never be used (and in fact never needed) in production? I just want to thoroughly understand this and the rationale.

No, I thought ObjectCache? was part of the code being tested. If the code just used a Group, I'd set the state of the Group before each test. -- EricHodges

So something like this? 

 public static Group getSystemRoot()
 {
     if ( sm_root == null )
     {
         sm_root = lookupRootInDatabase( .... );
     }


     return sm_root;
 }


 public static void resetSystemRoot()
 {
     sm_root = null;
 }
No, like this: -- EricHodges 

protected void setUp() throws Exception {
Group theGroup = Whatever.getSystemRoot();
theGroup.setState(...);
}
Cool. That's better, but my only problem is that each clean of Group would need to know that it was a mock object with a setState method. I guess I'd also have to redesign Group to be an interface instead of an abstract class and create one that simply wraps and delegates to a concrete group.

I'm not sure what you mean. All you need is one or more setState()-type methods on Group (regardless of whether it's a class or interface). Nothing above shows a need for a mock object, wrappers or delegation. The rule for unit testing is very simple: set state to known good values before each test. -- EricHodges

Okay, that seems to be an argument for each test creating a clean context and environment before running. I would think the easy way to do this would be to run each test method in it's own JVM

Each test should create a context before running. Ideally you should be able to do that in a few lines of code. If forking a VM for each test starts to look like a good idea, you may want to rethink how you manage context. -- EricHodges

Or not. Complex, highly decoupled but interacting systems are not always so simple to create a clean context for each time you start a test. Certainly not in a couple of lines of code. Plus, you have to ask yourself why you are building the infrastructure in your code to do that. If it is to support test, you may have a smell in your code.

[Building infrastructure into code to create a clean context to support testing is indicative of a smell. The question is, was the smell already there before adding the test-supporting code?]

There are some situations where context management can't be simplified. If you're working in one of those then a JUnit that forks a VM might be a good idea. Before you write one, make sure you can't simplify context dependencies. Also make sure your unit tests aren't really acceptance tests. -- EricHodges

I often hit problems testing legacy code that relies heavily on singletons. This is a smell in the production code, and this problem needs to be fixed in the production code.

For more short term solutions, I clear all static state before and after each test. It's not an ideal situation, but it does provide an immediate solution to a pressing problem.

If you really want to go down the "separate JVM" route, you can instruct ant to do this. An alternative, within a single JVM is to run each test in a separate class loader. With JUnit 3, one can do this by overriding a TestCase method in a common abstract base class that you create; check to see if you're running under a new class loader, and if not, load yourself (the current class) in a new class loader; use reflection to set a 'running under class loader' flag there, and then and invoke yourself (IE: the current test method) within the new class loader (using reflection). Be aware that doing this -- with separate JVMs or with separate class loaders -- will be SLOW. -- JeffGrigg

Distributing JavaUnit or UnitTests with code licensed under the GnuGeneralPublicLicense?

My company is about to release a piece of java software licensed under the GnuGeneralPublicLicense. The source tree includes junit.jar and our UnitTests, but JavaUnit is distributed under the IBM CommonPublicLicense, which according to http://www.gnu.org/licenses/license-list.html isn't GplCompatible?. It seems to me that we sadly must somehow remove junit.jar from the distribution, but i wonder if we're allowed to distribute our UnitTests under the GPL, even though they are "based upon" JavaUnit? Please AnswerMe -- ChristofferHammarstrom

See: JavaUnit

CategoryTesting

EditText of this page (last edited July 5, 2008) or FindPage with title or text search