Java Gui Unit Testing

I want to get the ball rolling with a page on GuiTesting, but specifically on Java. Since I am new to GuiTesting, I only have a few observations, but will share them gladly. Feel free to hack at this page however you feel is right, and maybe we will come up with something very useful for Java programmers trying to tackle GuiTesting.

As a 'prerequisite' please read GuiTesting and GuiUnitTesting and some of the articles linked to there.

-- JeffTulley

For a list of some (hopefully most) of the currently available Java GUI testing tools, check out http://www.superlinksoftware.com/cgi-bin/jugwiki.pl?TestingGUIs Feel free to add new tool listings, and especially to add information regarding your experience with the listed tools.

Some initial observations of things that are not covered on the general GuiTesting pages. First, active testing of a GUI in java where the GUI is up brings up some really nasty potential threading issues. The description given by the tool JfcUnit? describes this well. See http://jfcunit.sourceforge.net under "Need for JfcUnit?". Shoot, I cannot keep up with these guys. I just added a comment on GuiTesting about their great description there, and they just completely changed their site as of two days ago. Bye-bye to that great description. I'll see if I can contact the authors and get permission to reprint the old paragraph here.

In summary it was that the AWTEventThread has control after the GUI is displayed, and you should only manipulate GUI components from one thread at a time. Otherwise you have very strange errors occurring at "random" times, as well as other threading issues. JFCUnit largely relieves you of these problems by Stopping the AWT Thread at the right time to test.

-- JeffTulley

JFCUnit is a good tool when you have to actually test the GUI while it is displayed, allowing for recording of events sent and therefore playback (serialization of the events is not there yet, last I checked, but could be done fairly simply).

One thing I have learned, however, is that you do not always need to display the GUI you are testing, thereby skirting around the threading issues. (Generally, with Swing, you do, as many Swing components are not fully initialized until they are displayed. Unfortunate, but true.)

A program I am working on has the concept of panels embedded in a GUI framework, and so the particular panel class has a method with the signature "JPanel getContent();" From that returned JPanel, I can hash all of the components, and then start making assertions on them. I got the same functionality from JFCUnit with a displayed panel, but had quite a bit of overhead added to my tests time-wise. I'm sure I'll eventually use JFCUnit, but until then I'll use what is simpler and works. (Not knocking JFCUnit, it seems to be a great tool, and really needed also).

-- JeffTulley

One last GUI testing observation, not necessarily specific to Java: Test only things that are not brittle. For instance, I'll test that a particular Component exists and has contents that I expect it to based on how I instantiated the panel, but I do no testing on the GuiLayout?, where things are in the JPanel, or what they look like. That changes to frequently, resulting in breaking tests. (That is also very difficult to test, let alone CodeUnitTestFirst). -- JeffTulley

"I've been talking to myself just to suggest that I'm selfish" (Electronic)

Me again. So, I ran into a problem of how to test a JavaSwt-based GUI, where there is no existing GUI-testing framework. A Yahoo mailing list came to my rescue, and here was a recent email I sent there summarizing my findings:

Thanks to whomever it was that pointed out the paper, "The Humble Dialog Box". ( http://www.objectmentor.com/resources/articles/TheHumbleDialogBox.pdf )

I've found an approach that seems to work well for testing out SWT-based GUIs, lacking an existing GUI test framework.

The first is to follow the techniques outlined in "The Humble Dialog Box". They seem solid and really do make a lot of sense. I took some time and reworked his example in Java and SWT and found that my code was mostly tested, only the simplistic Dialog box itself lacked tests. (Which did bite me a few times, with some fairly good bugs).

What I did then was adapt some helper classes I had written for Swing over to work with SWT, and I found I could pretty much unit test everything that was necessary, and have fairly non-brittle tests.

The technique is to expose some way for your test code to get at the dialog's shell. (I need this anyway, since I am currently creating nest-able "panels").

Then, build a hash table of the dialog's controls, optionally recursing down lower if a control is a Container ("Composite" in SWT). Then your tests can get the control by name and do Asserts on it.

With JavaSwing this required you to do control.setName("SomeName") on every control that you wanted your test code to be able to access; (good programming practice anyway since it improves accessibility).

With JavaSwt it was a little tricker, I chose to use widget.setData("name", "SomeName"); Maybe there is a better way to get at a widget's name in SWT and it should be used instead.

Thought I'd pass it on, I'm going to try this technique and see how far it gets me.

-- JeffTulley

Also see: http://www.sdmagazine.com/documents/sdm0206b/. This is a technique that has worked well in many shops.

If you are building a JavaSwing application, I would suggest that you have a look at an open source project we have put together with some friends. The tool we are building is called UISpec4J (http://www.uispec4j.org), it is a collection of Swing component wrappers which help you test your GUI with high-level, test-friendly APIs - without needing to actually display any dialog or window. -- RegisMedina

CategoryTesting