Cross Platform Testing

When developers work on products that are portable or interoperable across platforms, it is common to use one of those platforms as the preferred platform for development. (Note that "platform" does not only refer to OperatingSystems; it can also refer to different database backends, CORBA ORBs from different vendors, and so on.) The development platform may be the one that is most commonly used by customers, or may be the one with the best development and debugging tools.

In such a situation, there may be forces that cause UnitTests to be run on other platforms only sporadically, or not run at all. This can lead to nasty surprises. When developing cross-platform products, running the tests continuously on all supported platforms is a necessary part of TestDrivenDevelopment. Otherwise, you are really only running a subset of the tests. You can not AlwaysBeReadyToShip if you aren't continuously running the full set of tests for all platforms.

Some tips for cross-platform testing:

When developing a product that has to work across platforms, set up the cross-platform tests immediately--don't put it off until later.
Set up an automated script that will run the tests on all platforms at least once per day (perhaps after each DailyBuild).
Use a cross-platform scripting language to drive automated test scripts, and use the exact same script for all platforms. If that is not possible, find some other way to guarantee that the same tests are being run everywhere.
Whenever tests fail unexpectedly, compare the results on different platforms. If the tests succeed on some platforms but fail on others, it's often a sign that the code is not as portable as is thought.
Whenever a new test is added, the developer who writes the test should be encouraged to immediately run it on all platforms (but this may not be feasible in all environments).
Don't use MockObjects in place of the cross-platform elements. You need to run tests against the real stuff to ensure portability and interoperability. Also beware of using platform emulators for your tests.

See: MozillaTinderbox, CruiseControl

CategoryTesting