Interclass Test Order

Jul/Aug 99 JournalOfObjectOrientedProgramming has an article: Interclass Test Order for Object-Oriented Software, by Keu-Chung Tai and Fonda J. Daniels.

We'll ignore the obvious pseudonym of the second author for the time being. The article asserts, in its abstract:

Performing interclass integration testing of object-oriented software requires a determination of the order in which classes are tested. This test order, referred to as interclass test order is important because it affects the order in which classes are developed. It also affects the use of test stubs and drivers for classes, and the preparation of test cases. This article proposed four desirable properties for interclass test order and presents a new interclass test order strategy to satisy these properties. In this new strategy, classes are integrated according to their major and minor level numbers. Major level numbers of classes are determined according to inheritance and aggregation relations between classes, where an aggregation relation refers to a class's inclusion of objects of another class. For classes with the same major level number, their minor level numbers are determined according to association relations between these classes, where an association relation refers to a class's deendency (other than inheritance and aggregation relations) on another class.

While this article may be suitable material for completing a course or getting a degree, in my experience it solves a problem that I've never had. For the last three years in particular, I've been writing tests for classes in the order produced, and running all tests all the time. I've been producing classes in the order required to produce business value as fast as possible, with little or no consideration for inventing frameworks or other infrastructure. Even complex issues such as multithreading have been left until they were really needed.

Once in a while one will add or change code that will break some class. If the class you break is depended upon by lots of others, a bunch of other tests will also break ... but if the class you're working on has decent tests (and by the ExtremeHypothesis it does), its tests will break, and since you're working on it, you'll look there first.

But I'm not a reasonable man. Some consider me to be Extreme. How about you other folks ... what do you think of examining the edge coverage criterion of the Object Relation Diagram in order to determine testing order, compared, say, to running all the tests all the time? --You Know Who

With sincerest apologies to the article authors, I have to say that I glanced at the article, determined pretty much that me and my code have been very happy testing as we go, and then felt a twinge of sadness and disbelief that somewhere in the world people go through that sort of analysis prior to testing. How do you get work done that way? I work in the most unextreme environment imaginable and we still don't have the will for that sort of thing.

In my unextreme org, the developers feel that unit testing is the test department's responsibility, yet they moan that testers never understand their code and are always an iteration behind. Duh. The testers wipe the floor with their morale because they end up testing things that are currently changing elsewhere. The poor people will never catch up.

In I walk, a developer who tests his own code. Both the testers and the developers look at me like I'm a Martian. In the tester's eyes, I'm a cute and cuddly ET-ish Martian. To the developers, I'm a strange insectoid that must be placed under glass and observed for a while. It is hard to breath.

I too saw the article, and was awestruck to see no mention of JohnLakos' book LargeScaleCppSoftwareDesign. Perhaps I wasnt looking closely enough, but it mostly looked to me like a proof of an algrorithm to determine the test order. Seems to me that Lakos' book talked about the same things, and used more practical concepts like Levelization and Escalation and Demotion and Cyclic Dependency Checking to show actual concrete practices for how to really do this stuff (rather than prove it in theory). Lakos' book also made available a working C++ program (and source code) to calculate these various things for you and determine if the program was levelizable for testing and what the relative position of each component in the hierarchy would be.

Like I said, maybe I looked at it too quickly, but I didnt see anything there that Lakos didnt write 5+ years earlier and say more clearly and concisely and pragmatically along with working code to implement it and let you try it out right away. And Lakos stuff talked at the level of physical design in terms of compile-time and link-time dependencies rather than "logical" dependencies of aggregation and inheritance which are one level removed from where the dependency "rubber" truly meets the road. Okay, so that's C++ specific; But at the very least a reference to Lakos was in order IMHO.

I'm UnExtreme?, but have found it very helpful for developers to write UnitTests as they go, and use them for RegressionTesting.

Sometimes I think I should organize the UnitTests "from the bottom up," so that low level functions are tested before being used in the tests of higher level functions. But I've yet to experience a case where it's worth the bother: Most bugs are caught during the first few runs of the UnitTests; this happens before the higher level functions are written or tied in, so it's bottom-up most of the time, even if later runs aren't. Later, assert statements still catch problems at a low level, close to the causes; and, in practice, a high level test that repeatably displays a bug is almost as helpful in debugging as a low level test that does the same. -- JeffGrigg

CategoryTesting