Tests As Scaffolding

One of the TestingPatterns

After reading MartinFowler's rejected Chapter 15 for RefactoringImprovingTheDesignOfExistingCode (at http://www.refactoring.com/sources.html#BookSupplements - Long Example), I realized that one of the things he did in there was a pattern I've seen well used in UnitTestingLegacyCode.

page 370: How did I get the $4.84 figure? If I had the requirements document I would have calculated it independently (using a calculator, spread-sheet, or abacus). Since I don’t and I’m refactoring I can do it by running the test, seeing what the answer is, and embedding that answer into the tests. Refactoring should not change the external behavior. If that behavior includes errors, then strictly we don’t care. Of course since we are writing these tests we ought to try and verify them with the business. But we don’t need to do that in order to refactor.

The pattern here is that your tests don't necessarily have to be correct for them to be useful. Often it is enough to simply have the tests check to make sure that the observable behaviour of the system doesn't change when refactorings are made.

On a recent project that was a nightmare legacy project, we did just this. We added tests for the final answers (it was a data-analysis calculator). Then we tried to fix the code. If the final answers changed, we undid the fix and did some debugging to find out what the problem was. Then we added tests to isolate this problem in more detail (rather than just checking the aggregate final answer, we added tests for the relevant intermediate answers as well). We only added tests to the parts we were fixing; we didn't try to have complete coverage of the legacy code.

Often we would find that the final answer was wrong in some way, as we explored the code through debugging and UnitTesting. If we had enough support tests, we would write a test as low-level as possible to test the incorrect intermediate value. When all these tests passed (testing that the incorrect code was still incorrect), we would then change the 'expected value' for that one intermediate value that we were certain was incorrect. Then we would try to make that one test pass. This of course caused all the other higher-level tests to fail since their answers had changed. Simple enough, we just cut and pasted the new answers into the tests to make them pass.

In this way, we used tests as scaffolding; not to make sure that the code worked correctly, just to make sure that it worked the same as it did the day (or hour, or minute) before. Sure the tests were 'wrong', but we were still able to make improvements to the legacy code, something that is notoriously difficult to do. As we went through the bug reports we would add new tests and passing them would cause some of the scaffolding tests to fail, but since we knew they were just high-level scaffolding (by the name of the test method), these weren't true failures. In the mean time, we had the ability to refactor the code without fear of changing the current behaviour, or introducing new bugs.

Anyway, that's a lot more long-winded than I was intending to write. Feel free to chop it up and mix it about.

CategoryTesting