Extreme Programming Testing Gui Intensive Apps

The more "GUI-intensive" the application, the harder it is to automate UnitTests. Are tools the solution?

There are a number of related problems with testing GUIs:

telling whether complicated graphical output is correct
testing input to GUI components
testing whether GUI components are created and painted properly

Here's a solution to telling whether graphical output is correct:

A common way to test complicated textual output (e.g. an HTML report) is to capture correct output, then simply compare any later output with that known-correct output. You can either fake things like time to remain constant for the unit tests, or have a more-sophisticated comparison function (the former is easier).

You might worry about false positives: i.e. a change might make the output different from the reference output, even if it doesn't make it incorrect. But the reference output is part of the unit test, so it should actually be changed before you make the code change (CodeUnitTestFirst).

You might also worry about changes to the order of the textual output which don't actually make it incorrect. You can deal with this by coming up with a more-sophisticated comparison function; it doesn't have to be much more sophisticated: e.g. you can simply sort the lines of the output before comparing, thus ignoring order.

[Some references to other places on Wiki that talk about this would be helpful.]

So if we can transform graphical output into textual output, we can apply this approach to good effect.

Let's assume we're working in Java, so I can keep knowing what I'm talking about. All drawing in Java is done to a Graphics object.

So your graphical code has stuff like this:

  g.drawLine(x,y,x2,y2);
  g.drawFilledRect(x,y,x2,y2);
  g.setPenColor(Color.red);

Say we had a special subclass of Graphics named TextGraphics?, with one extra method:

  String getTextualRepresentation()

What does it return? Well, every time you call drawLine(), it adds a line to an internal StringBuffer:

  g.drawLine(x,y,x2,y2); // where params = 0,0,10,20

results in this being added to the StringBuffer:

  drawLine(0,0,10,20);

...and similarly for the other calls. The getTextualRepresentation() method just returns the StringBuffer as a String.

Now you can substitute a TextGraphics? for the Graphics in your complicated drawing method. When you're done drawing, you can automatically check what you drew. Just get a text representation of your correct reference graphical output (either read it in from a file, or plunk it in as a String constant in the unit test). And now you can check the messy drawing stuff as easily as this:

  assertEquals(referenceString, g.getTextualRepresentation());

Again, if you don't want changes in drawing order (e.g. lines drawn first, then rectangles, vs. rectangles drawn first, then lines) to show up as failures, you can make your comparison function more sophisticated:

  assertEquals(sortLines(referenceString),
sortLines(g.getTextualRepresentation()));

(where sortLines() just sorts the lines of a String in alphabetical order).

There are other, probably obvious, benefits to having a String representation of a drawing. You can:

get away without actually drawing during your unit tests
pass it around, and write an interpreter method to turn the commands back into method calls
use it to delay drawing until you actually need it
animate drawing by doing it slowly
easily write a proxy Graphics, so you can write a distributed app without having to send every pixel that changes over the wire

A less-slick (and quicker) way to do this is to just have your drawing methods accumulate text while they do their work. They can return the log as a String, or they can add to a log instance variable. Either way, the unit tests can grab it and automatically compare it to a known-correct reference value.

There are a couple of wrinkles to doing the logging so that the comparisons come out right: sometimes you'll need to rewrite calls into their canonical equivalents - e.g. drawLine(10,20,0,0) becomes drawLine(0,0,10,20); the order of drawing calls can sometimes make a difference (e.g. if you count on a rectangle being on top of a line); setting the drawing color definitely makes a difference depending when it happens (so you might want to include the current color in the log entry); etc.

But by and large, this approach makes it easy to test something that otherwise would be hard - that is, would require repeated human inspection, judgement, and time.

-- GeorgePaci

The problem with the text approach is that you are unit testing the 'how', not the 'what'. If you refactor the drawing method you break the test. Perhaps a better approach would be to supply a Graphics object that painted into an image which can be compared with a reference copy.

See

CategoryTesting