Test Every Refactoring

These patterns are intended to be used by the community so it is the community the best suited for decide their content.

Test Every Refactoring

"I think it is fair to say, that virtually there is no software that does not contain defects. However, we as software engineers still try to make as few errors as possible and we are also always looking for ways of improving the quality of what we deliver" [1].

Pattern Name: Test Every Refactoring

Problem

How do you avoid the introduction of errors that may break the system during refactoring?

Context

You are developing software and you want to make some refactorings before continuing coding. You followed TheFirstRefactoringStep, then you applied RefactoringInVerySmallSteps.

Forces

If you introduce an error during refactoring, it is probably hard to go back without throwing all your changes away.
Trusting in the correctness of applied refactorings will cause frequent mistakes.

Solution

It is absolutely necessary that you test every refactoring. Unit Tests [2] are great for this. Remember that "the tests are your safety net, protecting you from breaking the system during fast evolution" [3].

Resulting Context

Testing systems every time that refactoring is applied will maintain a strict fence on the modified code sections. After each refactoring, if a problem was introduced, tests will alert immediately the causes and the conflicting location.

Rationale

"One way to improve software quality on the functional level is to have good tests in place" [1]. Besides, "work produced must be continuously validated through testing" [4]. Kent Beck and Erich Gamma expose in [5] their development methodology as "code a little, test a little (...)". Paraphrasing Beck and Gamma, refactoring should be guided by "refactor a little, test a little".

Refactoring introduces changes in the source code of the system, so after applying a refactoring, you should test the program to validate its functional integrity. If the tests fail after refactoring, it will be only necessary to verify the code sections involved in the process.

Origins

This pattern come from the WIKI pages about refactoring [6]. The original reduced pattern was: "You don't want to find out after 37 refactorings that somewhere around the 21st you made a mistake (...). Therefore, test every refactoring. Unit Tests are great for this (...)".

Acknowledgments

References

[1] Manfred Lange. It's Testing Time! Patterns for Testing Software. Proceedings of Sixth European Conference on Pattern Languages of Programs (EuroPLoP 2001). Irsee, Germany. July, 2001.

[2] Robert V. Binder. Testing Object-Oriented Systems. Addison -Wesley, 2000.

[3] Ron Jeffries, Ann Anderson, Chet Hendrickson. Extreme Programming Installed. Addison - Wesley, 2000.

[4] Ken Auer, Erik Meade, Gareth Reeves. The Rules of XP. Available at: http://www.rolemodelsoftware.com/moreAboutUs/publications/rulesOfXp.php 2003. Last confirmed: 20081208

[5] Kent Beck, Erich Gamma. Test Infected: Programmers Love Writing Tests. Java Report, 3 (7). July, 1998.

[6] Wiki Pages About Refactoring. Available at: http://c2.com/cgi/wiki?WikiPagesAboutRefactoring. Last confirmed: June 26, 2004.

Author: SantiagoValdarrama?

See Also:

You applied RefactoringInVerySmallSteps. You don't want to find out after 37 refactorings that somewhere around the 21st you made a mistake. It is probably hard to go back without throwing all your changes away.

Therefore, test every refactoring. UnitTests are great for this. Also, if you added or moved code, make sure that it gets called. One way to do this is to introduce deliberate errors in the new code, so that the tests will fail. Another option is to set a breakpoint in the new code and see whether it is reached by the test code.

Run all the tests after the fewest possible edits - say 10 at the most. If you are not adding features, expect every test run to pass, and Undo your last edits if they don't.

That rule forces you to continuously return the code to a passing state. That passing state forces your designs to relentlessly decouple.

Also, of course, make sure you can roll back the refactoring without too much work in case the tests do fail. That is, at least for larger changes, have a backup copy of the state of your program before the refactoring (if your environment doesn't provide this automatically). -- FalkBruegmann

This comment was transformed into BacktrackIfRefactoringFails pattern.

I usually first comment out the code to be removed, and only really delete it at the end of a RefactoringEpisode?. This only gives problems if the tests weren't good enough, the refactoring really broke something, and I find out too late. This approach works best in an environment where you have syntax highlighting, so that you can see immediately what is code and what isn't. -- MarnixKlooster

I think this depends on your environment. Using ENVY, either Smalltalk or Java, I just delete the suspicious code. "Load Previous Edition" gives me a quick way back. -- KentBeck

I'm finding that if I TestEveryRefactoring, and then compare outputs after the tests pass, this technique helps me tighten up my UnitTests. (Though maybe this means I'm a little too lazy about writing them at first.)

For example: I'm currently writing a large report-generating program that spits out CSV files. So when I refactor, I:

Run the UnitTests, and ensure they all pass.
Run the program and pipe the output somewhere.
Save the output in another directory.
Refactor.
Run the UnitTests, and ensure they all pass.
Run the program and save the output again.
Diff the old output and the new output.

Occasionally, the outputs will vary in a way that I don't like, which teaches me two things. 1. that my refactoring broke something, and 2. my UnitTests weren't rigorous enough to catch it. So before fixing the refactoring, I fix the UnitTest, or add another UnitTest to catch a similar bug next time around. -- francis

I think this is a case of DIY VersionControl, with suitable tools the idea of storing your diffs separate from the version control may lose valuable information about what experiments tried and failed, see BacktrackIfRefactoringFails.

I guess the forces operating here are to do with the support of the VersionControl system your using for easy branch creation and 'polluting' the tree with dead branches, so you just need a good method for naming branches, and buy in from the rest of the team that small experimental branches are good.

-- KevinWheatley?

CategoryRefactoring CategoryPatterns