Thoughtful Reactions Unit Tests

ThoughtfulReactionsUnitTests continues a discussion from ThoughtfulReactionsToXp.

How to write/automate them for large, distributed, GUI-based systems?

Some of this stuff really is hard, and maybe you do have to have some other mechanisms on top. That said:
- Every distributed system is made up of components that can be unit tested. Those unit tests exercise the protocol that the component is supposed to support, so it's a little easier to find disagreements between components. Also, unit tests make it easier to stub in test proxies.
- Underneath every GUI system is a model. If you don't have a clear interface to that model, preferably unit tested, you're hosed anyway.

See also GuiTesting. (Some of the comments here should probably eventually be moved to that page.)

We're not convinced that building unit test cases before coding is a good idea. XP discourages design documentation but suggests building executable test cases before coding! Talk about a heavy artifact that needs to be constantly maintained. In most cases, test programs that are written prior to design/code would be obviated by the time the classes are implemented.

You grow the tests with the system, so there are no make-or-break releases. You also refactor the tests, as you do the code, as your understanding improves. It's not that hard if you do it as you go. The cycle is: write a test, implement it (if you have to), refactor.

The rhythm of building tests & empty methods (to ensure compiling), testing (fails), actually getting the tests to work is in my experience quite fast - I think there is even a rule not allowing you to have red tests for more than a day. This makes it fun to do as well. -- KlaasVanSchelven

UnitTests are not a "heavy artifact." This is a misuse of 'heavy' and 'artifact' as used by ExtremeProgramming. "Heavy" methodologies are weighed down by the requirement to produce "artifacts" - an artifact is a document that describes the software but isn't part of writing code, and so must be maintained as a separate task. In TestFirstProgramming, UnitTests contribute directly to writing the code (the test code and production code have a symbiotic relationship - each checks the other, like the hands in the Escher drawing that draw each other). So they are not "heavy artifacts"! -- ApoorvaMuralidhara

I'm absolutely convinced about this one. It's the only realistic way to get good test coverage and that gives you the confidence to do the refactoring you should. I've seen systems die because everyone is too scared to change anything. Don't take my word for it, try writing unit tests first for a little while, but you do need a good source control system - I don't think it's a coincidence that most XP Java developers seem to be using VisualAge. -- SteveFreeman

What? For six months this year, I led a team using VisualAge for Java 2.0, and tried to encourage (well, demand) the XP approach to testing. VAJ and ENVY foiled us at every step, we cursed it daily. VAJ/ENVY provides good versioning, but no configuration management. -- KeithBraithwaite

True about the configuration management (which I still can't believe) and many other annoying limitations, but it's still the case that it's the only IDE where you can just type and always revert to where you used to be and where you get the interactivity in the debugger.

I have to admit that we gave up on ownership (we have a single user id), but I would have preferred to keep it if we'd had more time to ramp up. As far as I can tell from the papers for the XP workshop at OOPSLA, it really is the case that most (all right, a lot of) XP Java developers are using VisualAge.

Haven't we had this discussion in another page? (Perhaps DoesXpWorkForJava?)

I agree with UnitTests first. I am just having a problem fitting it into the Windows world of GUI and COM components and Internet pages. How do you write UnitTests for GUI apps and especially all Internet based programming particularly in ASP? I've read the xUnit stuff and it seems to test functionality of a static piece of software. How do you do it when you are developing mostly Internet-based software? Or COM components? Thanks for any advice on this as I would like to get my programmers to write UnitTests first. -- sg

I agree that we still don't have good answers for straight GUI testing. My response is to move as much as possible into model code which can be tested automatically (although even that would be hard for black-box COM GUI components). That, at least, gives you a solid foundation for the GUI. I hear that ASP testing is pretty tough because it's hard to separate out the various components. The only suggestion I have would be to script test web clients that send queries and parse the result, but maybe you're doing this already?

I develop web-based applications (in perl, not ASP). Until I started separating the page drawing, application logic and business logic it was very difficult to unit test. Now that I am getting better at the separation, I can unit test the business logic. As for the browser part, I still test that by hand. However, I will be looking into automated web gui testing tools in the future.

For ASP Unit testing try this route: http://www.microsoft.com/mind/defaulttop.asp?page=/mind/0498/aspvb.htm&nav=/mind/0498/inthisissuecolumns0498.htm

If you create objects for all your common ASP Objects up front you can easily write automated tests. For example you create a request and a response object with all the same properties of the ASP versions, then you can set Request.Form to a known condition and test to see that the code properly updates the Response object. -- JoshuaDrake

You should also keep in mind that what is commonly thought of as "the browser part" is two parts. (1) is the correct html (or whatever) content being sent to the client, and (2) does the browser render that content correctly. Piece 2 is a pain, but your favorite scripting language here should be able to handle part 1 well enough. -- DanilSuits

Re: Gui Testing...

On XpAtArinc, We're working in MFC, a notoriously difficult environment to refactor/clean/test. After mucking around with inadequate testing facilities for a while, we came up with two approaches that seem to be making all the difference.

First: Virtually all GUI components work directly with non-GUI servers that provide the data they need. The servers can be readily object-tested (our word for unit testing, as that word has other meanings at ARINC) through normal means. By taking this step we turn the GUI components into fairly pure paint'n'command objects, making them much simpler to write in the first place, and allowing us to stall on serious object tests for them, basically because they are being functionally tested hundreds or thousands of times of day. This approach also eliminates the most obvious first question when a defect appears. Is it the GUI or is it the data? In our case, it's always the GUI, because the data server has been extremely thoroughly tested.

The second approach, just now coming in to full coverage of our code, is centered around a simple but fairly robust event logging system. The windows log a variety of interesting events (by category). A log can be manually saved and then later compared against, as a sort of half- object test and half- functional. It is a very effective means of creating tests. It *does* require a real 'event' log rather than some kind of text file output. The event log has 32 event categories that can be filtered on. Further, two logs can be opened, two filters attached to them, and then the two filters can be compared for differences. Obviously, the event log is also central to the full-scale functionals.

Overall, CodeUnitTestFirst is the single most powerful Xp technique. It lets us work faster, without a documentation safety-net, and gives every developer a *powerful* sense of progress and unity. -- MichaelHill

I'm a newbie to this Wiki thing, so pardon me if I go over old ground.

I'm working in the domain of network management software for a large telco. I've only become aware of XP recently, but the techniques of UnitTest and RefactorMercilessly are ones which I have used for years - so I'm clearly inclined in that direction.

However, I can't see how to make UnitTest coverage anything like 100% of my classes. The reason is twofold: the code I write has to interface with complex external systems (via CORBA) and also with local classes whose behaviour appears complex (actually many are quite simple internally, the complexity comes from the many other classes they collaborate with.

Oh, it's multi-threaded Java, written in VisualAge.

Now I've categorized the classes along two dimensions: simple and complex; and non-interacting and interacting, plus another "extremely simple" category for things like exception objects and data holders which have no behaviour to speak of. This latter group aren't UnitTested (what's to test?). Of the others, the Simple ones I can inspect with confidence, the Non-interacting ones I can and do UnitTest, but those which are complex and interacting with many other classes are a major headache.

What is the XP solution to this problem?

-- AndrewParle

XP would have you ReFactor the classes / interactions that are currently too complex, breaking them up and/or reorganizing how they work such that they are testable (and writing the UnitTests).

Try using MockObjects to make unit testing easier.