Xp Challenge Graphics Packages

Formerly ExtremeProgrammingChallengeTwo

From ExtremeProgrammingChallenge. Let's take this thing called ExtremeProgramming and play with it... push it's limits. Here is a scenario that takes XP out of it's element. How would we do XP?

Scenario 2:

You are a lone developer in your own startup making a graphical editor for OML. You look at your cat lovingly, but she does not seem interested in PairProgramming. Worse still, your work is predominantly graphical. You think about writing some UnitTests, but most of the functionality you need to test is right in front of your eyes. You see what your lines and shapes are doing. Your users are not yet a glimmer in your bank account. YouArentGonnaNeedIt, but they'd better.

You'll do best to get to market quickly to get feedback on what users are going to need. YouArentGonnaNeedIt must be applied to get you to stop gilding your lily and get it on the street.

Graphical systems are hard to test accurately, because you wind up running just the tests you think are important, thus missing that the thing you just fixed broke something you would have never thought of.

Instrument all your graphical primitives to log their results in a simple if cryptic language, to a file, perhaps ala TurtleGraphics:

GO 100,200 DRAWTO 300,500 RECT 200,300

Capture the results of running your AcceptanceTests, check them for accuracy, and then make the ATs check the newly-written log against the master log to determine what's done.

To check your primitives, you need UnitTests. This can be done by eyeball, but again it is unreliable. Consider drawing the primitive pictures and reading in the screen and checking to see if the pixels are where they should be. Avoid the temptation to use the same primitives to read the data back: they might be subject to systematic error. -- RonJeffries

In this scenario, the challenge is in the testing. XP says to make auto-testing harnesses; XP and graphics means to address the question of auto- and regression-testing graphics. Hand testing is unacceptable. A good, hard question, but that is the matter at hand. -- AlistairCockburn

I did this one, and I cocked it up, so I know what I would try next time.

There is a lot in a graphic editor that isn't rendering- constraint maintenance, cut and paste, file save and restore, connection to underlying semantics, damage propagation. All of that can be easily tested programmatically, probably without any rendering at all.

That leaves the rendering code. I like Ron's suggestion. You always have to write your own brush anyway, so it's easy to create an impostor which records the graphics commands and compares them to the expected commands.

Then all you have to do is test the brush. Maybe comparing bit patterns is best for this. If it's important and prone to break, it will be worth doing. If it's dirt simple, it won't need a test.

-- KentBeck

TclTk (a graphical toolkit for a "scripting" language) has a test suite; run "make test", and it exercises its graphical capabilities. (This is what I think of as PressToTest? functionality.) The current home for TclTk is http://www.scriptics.com/

In my last job, we had software that sent e-mail to other companies. We also had a configuration file option that turned on or off whether e-mail really got sent, and whether the e-mail message to be sent was logged. I wish I could take credit for this, but one of my co-workers came up with it.

I hear Kent's suggestion above, translated into my C++ mindset, as "fill your code with internal consistency assertions; UnitTests can exercise them, as well as the externally visible behavior."

-- PaulChisholm

Well, we tend to put all the assertions in the UnitTests, not in the production code. Inbuilt integrity checks, assertions, pre-and-postconditions are definitely NOT an official XP technique. And speaking just for me, I'm agin 'em. Otherwise, yep. -- RonJeffries

I use something much like ExtremeProgramming to deliver custom direct-manipulation graphical editors to a variety of clients. I find the UnitTests are the sample programs for each layer of the application framework, and that, for the most part, testing of new features and related capability by hand allows for forward, if imperfect, progress. Automated regression testing would be nice to reduce the rate of inadvertently introduced bugs, and to that end I've integrated a scripting language into the event-handler of the drawing editor framework I use. But I think there is always a need for human beta-testing, because the space of possible bugs and misunderstandings is larger than any finite test suite. -- ScottJohnston

Potentially useful thought experiment: What would have to be true for the space of possible defects not to be larger than any finite test space, for all practical purposes? -- RonJeffries

: It sounds to me like you are trying to hint that, since the machine is finite, the test suite can be, too. If that is the case, I disagree with your conclusions -- while machine space may be finite, machine time is not. Time, I think can be a real problem for your test suites. How would you TestForaRaceCondition? If I time my mouse events just right, will the graphical editor do something completely bizarre?

Nope, that's not what I'm trying to hint at. Wanna try again?

: I have been thinking about creating another ExtremeProgrammingChallenge. Here's the sketch: You are writing an operating system. You have hardware producing interrupts at a fast and furious rate, multiple processors to sort out, etc. Can you use ExtremeProgramming in this environment? What would your MultiprocessingUnitTests look like?
: -- BillTrost

MichaelHill asked a related question here. It has been spun off into ExtremeProgrammingChallengeTwelve?.

I've got a similar problem to this one, in that the output of my program is not directly "testable". My scenario is based on controlling a serial device - it's a semi-intelligent telex unit, for those who are interested. Anyway, my interaction with the device is through sending escape sequences to it. It then responds (eventually) with responses - some positive, some negative. The problem that I have is with UnitTests - I'd like to write some, since the code is getting hideously complicated, and I'd like to refactor it - with the support of UnitTesting, so I know if I break something.

Because of the way that the system works, I'm looking at implementing a full-on TestHarness?, which pretends to be the telex unit. However, at this point, I'm going to be writing almost as much code for the test harness as for the driver itself, and I'd like to steer clear of that, because then I'll end up writing UnitTests for the test harness :-)

The other complication is that, since the process is asynchronous, I cannot simply have a linear sequence of UnitTests - since the operations take time, and everything is event-driven.

So, does anyone have any pointers on how to use XP in this scenario, or more generally, in asynchronous, event-driven programming?

-- RogerLipscombe

When I did a similar thing to drive my Sony jukebox CD player, here's what I did:

First, I modified my sequence output code to just dump all sequences to a file as it went.

Then, in testing output sequences for correctness, I would empty the file, run the commands, close the file and compare the string mechanically to the string the test expects to see.

Same thing on the input side. The Sony talked back to the computer. I modified the serial receive code to log itself to a file. I could optionally use a separate file, when I wanted to know what the box was saying, or the same, when I wanted to know the sequence. The sequences were simple enough to decode, since I was sure I was outputting what I wanted to, so I didn't go to the next step.

The next step is to log both input and output to the same file, with each character or sequence preceded by a magic character saying Input or Output. Then you can display the sequence on two lines, or in color, to see who said what when.

And, of course, it's easy to drive all your event code from a script.

Finally (I do go on, don't I) I always recommend that, where possible, the input interrupt routines just semaphor the input character onto a stream, and similarly the output interrupt just semaphors a character off an output stream. The main part of the program than then just hang in a loop, reading input sequences when it pleases. Makes most of the code look like a conventional command-driven system. Much easier.

I hope these seem applicable ... if not, that they trigger ideas. Tell us more about your situation if these don't seem to help ... and tell us what you come up with in any case! -- RonJeffries

Thanks, Ron. Those are all good ideas. It looks like I'm going to have to break out the serial control stuff more cleanly - I currently derive from a class commPort, rather than use an instance. Then I can simply replace the commPort with a fakedCommPort class. This will allow me to do all of the logging.

The other thing I didn't mention is that the inputs to the system are quite simple, and each of them actually involves several requests/responses and state changes. This complicates the system, since I can't simply call a method and examine the log file, since the next piece of output won't be sent until the correct response is received.

Similarly, there are all sorts of timeouts involved - I also need to test the behaviour of the system if a response isn't received at all. For example, initialising the unit requires sending it a reset request. If I don't see the reset response within a set time limit, I try again.

But, while this is happening, the main flow of control is not frozen - when the device resets (or doesn't - I try a set number of times before giving up), I fire an event to someone else.

An idea for how to do it is taking shape, however - if I abstract the port object away, then I can trigger the events from the UnitTest, and see what kind of events come out of the other end of the system.

The complication comes from the fact that the UnitTests have to sit around the class I'm testing, so that I can call a method on one side, and see what comes out of the other side, and then respond to that in some way, and see what comes back out of the front.

Hmmm... I need to think some more about this.

While I'm on the subject, are there any sources for patterns for asynchronous systems? I've only written this one asynchronous system, so anything I come up with is a ProtoPattern at best. I'd like to see what else is out there.

-- RogerLipscombe

The SingleThreadedAgentPattern is a related ProtoPattern. -- MarnixKlooster