Unit Tests Are Production Code

Moved from UnitInUnitIsntTheUnitYouAreThinkingOf?

I have read most of this page, and I believe I have yet a new way to look at UnitTests. I am struggling a bit with how to express this.

Consider the software that makes up a complex software project. Typically, you have a set of libraries at the lowest level that provide the most basic functions. These libraries may be provided by the compiler, a third party or may be written in house. Some libraries build upon other libraries.

You build various components on top of these libraries and components on top of other components. Eventually you get to the top, where you have a piece of software that solves a customer's problem. At each layer in this software, there are various behaviors that are expected to exist, and each layer relies on the behaviors of layers below it, combined with that layer's logic to in turn provide its behaviors to the layers above it.

At the top of all these libraries, components and layers, is the top most layer which provides a large set of behaviors to the customer.

Now, when a programmer wants to make a change to the software to add a feature, the goal is to make a change that implements the feature without impacting the existing behaviors expected by the customer (except as driven by the feature to be added, of course).

But, in the absence of any UnitTests, the programmer has little at his disposal to assure that the myriad of behaviors still hold.

Key Point:

Automated tests (which can run as a complete group and returning a single pass/fail result) transform a system with many complex expected behaviors into a system with one simple expected behavior. By testing this single behavior, you have exercised all the complex behaviors which your customer is paying for.

(Of course, the degree to which the above works is dependent on the quality and coverage provided by your test suite.)

It is the automated, self-checking qualities of UnitTests that are critical at this point. Simply regard the tests as another layer of software (like any library or component in the software itself). This test layer's sole purpose is to transform a body of customer software with hundreds or thousands of expected behaviors to a somewhat larger body of programmer software with exactly one expected behavior -- a successful test run.

The customer software with lots of behaviors solve the customer's problem. The larger body that includes the automated tests solves the programmer's problem.

As developers, we need to build the larger body of software, but we ship the subset without the tests. The interface between the tests and the software under test is no different in concept from the interface between one layer of "production" code and the layer below it that it relies upon.

The more I have thought about this, the less I differentiate between the tests and the "production" code. The tests are production code -- the tests need to be maintained and evolved through refactorings as much as the rest of the code. Of course, the behaviors expected by the customers are typically less pliable.

As I add or change a customer desired behavior of my code, the writing of the test case is simply a user of that behavior which transforms it into a form that the test harness can use to enhance the value of its simple expected behavior.

-- JohnVriezen

I disagree that UnitTests are ProductionCode. "Production code" is the stuff that solves the customer's problem, while unit tests are a development tool. I do agree that unit tests need to be treated with the same level of care as production code, but a different style of development is useful. I update my unit tests as necessary for requirements changes, but I do not refactor my unit tests. My unit tests contain a lot of redundancy and inefficiency, and that's the way I want it. If you futz around with your tests too much, then you have to start writing unit tests for your unit tests, and you start to lose the benefits. My rule is that a unit test cannot change after it is written unless there is a corresponding change in requirements or a change to the production code that forces a change. --KrisJohnson

Ah, but my unit tests do solve the customer's problem. We recently provided our customer base with a "bad" fix and had to retract it. This sparked a new emphasis on assuring developers have written and run proper unit tests during the fix process. Since our software provides High Availability for our customers' systems, they rely on our ability to test and not release bad fixes. Just because the unit test code runs at our site rather than the customer site, doesn't mean it can't solve their problem. - JohnVriezen

Shouldn't all production code have tests? Do your tests have tests? If not, they are not production code.

Yes, our unit tests have tests, but not in the manner to which you allude. As I point out above, a unit test's job is to adapt a piece of software that has many complex inputs and outputs that need testing into a larger piece of software that has a simple expected behavior. In fact, this expected behavior is so simple, that it can stated as "Press this button, and after a while, you should see a success indication." (Usually there is a testing harness such as Junit that aggregates this for many tests into one button, one result.) So the unit test for our unit tests, is to run them, and see that they pass. We could automate that process in a unit test written in code, but that would be sort of pointless.

The main reason anyone writes automated unit tests is to transform a piece of software with a complex set of expected behaviors into a somewhat larger piece of software with a simple (usually trivial) expected behavior which still exercises all the internal complex behaviors. - JohnVriezen

CategoryTesting