Tests Are An Executable Specification

UnitTests specify classes and other closed items, localized parts of the system.

FunctionalTests or AcceptanceTests specify the whole system.

There may be features of the system that a not covered by these ExecutableSpecification?s.

Tests are not ExecutableSpecifications. They test the implementation of a specification, they specify nothing on their own but very. Just as it is a good idea to understand the requirements that drive a design before design, it is a good idea to understand the tests that quantify an implementation before the implementation. This doesn't make the requirements the design nor the tests the specification.

Even with TestFirstProgramming? You code the test and then make the code fit it. Sounds like a spec to me. --TomAyerst

I never really looked at TestFirstProgramming (ala TestInfected) as making the code fit the test. Even so, they are testing the validity of the code, they don't test its overall shape. Only the source itself can be a low-level specification (or blueprint) in the JackReeves (http://www.bleading-edge.com/Publications/C++Journal/Cpjour2.htm) sense. You can't really manufacture a product from the tests.

But executable specification has a different meaning among the formal methods folk. Informally, executing a specification means automatically checking it to see if it means what you thought you wrote. Now ding dang it, why would anyone want to do a thing like that?

Is there really such a thing as a test, or is to test a verb? If there is a thing called a test, exactly what is it?

Time out. Every persistent software artifact is a specification. The machine code for your program is a specification to the machine that executes it. If, when you say test you are talking about some persistent form or forms, such as an executable program and some written instructions on what output it should generate, then you have something that is both a test and a specification. The question is not whether or not it is a specification, but what the specification is for (who will interpret it) and how good a specification it is for that purpose. -- WaldenMathews

The key advantage of a test set as proposed in TestFirstDesign as a specification is that it is unambiguously verifiable against the code. The code either meets the specifications of the test set or it does not. In other words, the test not only incorporates the specification of the desired result, it also specifies the measure and the measurement device used to determine the result. We still may not be happy with the result, but there is little room for debate as to whether the test set and the code are in agreement.

Textual specifications could contain this level of detail, but often do not. In common practice, it is often accepted that the code design and validation design should be two parallel and independent tasks that flow from the specification. In even the best operations this leads to conflict when two different methods for evaluating adherence to a specification are created and different results are obtained. There is usually no compelling rationale to determine whether one interpretation is more correct than the other, hence, all reconciliation effort is wasted effort. There is no way to believe that the result of the reconciliation effort is any closer to the specification than either of the two original interpretations.

Having a specification is not sufficient. Documenting how the specification is interpreted and validated is also necessary. The most effective way to accomplish this is to combine all three into a single artifact, hence the executable test as a specification.

--WayneMack