Unit Test To Code Ratio

For a well-factored project with well-factored Unit Tests, what is a typical ratio of UnitTesting code to non-Unit Testing code?

Answer:

I am curious because I have a Python project that I have been doing for a while which has been my first major Unit Testing experience, and I recently ran a quick check. About 100K of the code was unit tests to 600K actual code.

This actually surprised me, because I felt like it was going to have about twice as much on the unit testing side, probably because I spend a lot of time with it.

What are your project's experiences? (It would probably be helpful to mention the language the project is in.)

I always assumed it varied from project to project.My ratio is anywhere from 1:1 to 2:1, not counting mock object classes, which probably reduce count anyway due to easier setup. (C++, slowly adopting STL, smart pointers, and moving away from MFC/MS-specific stuff.)

In the projects I've experienced (XP projects), test code can be up to 2.5 times the amount of production code if the code is well-factored. However, eventually, you start learning how to refactor your test code to eliminate the duplication there. Once that happens, the ratio comes down to somewhere between 1:1 up to 1.5:1. If I had a project where the ratio was 1:6, I would expect that I was missing a lot of unit tests or that I was testing at too coarse a level. However, this ratio could also be different for different languages. My experience is with Java (and a little bit of C#) --MikeCorum

Actually, that's something I (page starter) forgot to ask in my initial question, for the language, which I've now corrected.

Some of it may be that as this is a one-person hobby project I'm not doing strict XP (a lot of CodeFirstTestImmediatelyAfterward? since I'm in a highly experimental domain; I tried TestDrivenDevelopment but I frequently found that without some exploration I couldn't even guess at a good interface!), but even so, I feel I have pretty good coverage (except in the GUI itself, which I did before getting TestInfected; I'm rectifying that now and that won't require much new GUI code so that may get the ratio down to 2 or 3 (code) : 1 (test)). A lot of the Unit Tests I have concisely cover entire multi-dimension matrices of possibilities in very little code. It may just be that Python can factor Unit Tests much more easily. (Also, a lot more of that 600K is in-code documentation; I don't know what the real ratio is, though 4 (real code) : 1 (test) is probably a minimum.)

re: I tried TestDrivenDevelopment but I frequently found that without some exploration I couldn't even guess at a good interface!

I've always seen that one of the better reasons for a SpikeSolution. That may or may not be what you're doing, just in different words.

Something I've found useful in those cases is to do the ugly spike solution, and then refactor it back to test code. Every place where I've written a print or a assertion or some extra code to make something obvious, I replace with a test. This won't give you the same coverage as writing the things first, but it gives a good starting point. Even better, those tests let you write the real version test first while knowing that your implementing the same thing as the spike solution. --WilliamUnderwood

Yep, that's what I've been doing. I find dynamic languages make it particularly easy to refactor a spike solution into the real thing. Sometimes the only thing that survives that process long term is the Unit Tests, but there's no point where the SpikeSolution is just ripped out and replaced with the "real thing". --JeremyBowers

I would suggest the proper measure for comparing unit test to code is time spent not number of lines of code generated. When doing TestFirstDesign, I would look at about half the time spent on test code and half the time spent on production code. Whne things are going well, most time is spent in determining the test to write, the writing actual test code and production code flows pretty quickly after one knows what needs to be done. To off set this, though, there are times where one struggles to make the code pass the test, often due to unknowns one must discover. My advice, don't worry at all about the ratios, but be concerned about the number of times one needs to come back and revisit the code after it is "done." --WayneMack

Actually, on anything other than trivial software projects, I think the ratio is important. The bigger the tests in comparison with tested code, the more sign that refactoring's needed, and the more difficult it is to maintain the codebase. I speak as someone currently suffering from unit test classes (written by a succession of developers) in 5:1 ratio and with 1200 lines. It makes me want to change careeer.

CategoryTesting