Enough Verification

I feel there was a lot in common with the way I do software, and some of the descriptions of engineering here. I do have a theory, at least for the components. I can reason about the behaviour of a stack. I use AbstractDataTypes. I use LoopInvariants? and LoopVariants? - generally informally, but they could be made formal. So my software is more theory-based than you might think. Contrariwise, engineering is less theory-based than you might think.

This touches on the major software schism between static analysis and dynamic analysis. XP is very much on the dynamic side. XP'ers seem to prefer testing over theory, doing over thinking, dynamically checked languages like Smalltalk over statically checked languages like Eiffel, UnitTests over DesignByContract. Personally I lean towards the static side, and I occasionally have to readjust my expectations when interacting with this Wiki because of it.

For me it is not really beautiful unless I understand it. It's not enough for me to know that something works: I want to know why and when, and under what circumstances it can fail. The program reached this breakpoint - will it always do so, or might it sometimes go down another path? It passed a UnitTest but that only exercised a finite number of cases - what about all the others?

-- DaveHarris

I don't agree with the static analysis/dynamic analysis distinction. You could do a project extreme and write assertions out the wazoo. I read a paper BertrandMeyer wrote a couple of years ago where he talks about a UnitTest discipline indistinguishable from XPs.

What would make a project not extreme is if you did more assertion checking than you needed, or if you let the presence of the assertions keep you from refactoring. How hard is it to radically restructure code with lots of assertions?

-- KentBeck

Static versus Dynamic is not either/or - you can have both. But IMHO the tension between them is one of the great themes of software.

I think assertion tools could benefit from some work if they are to be integrated with XP. For example, they could do with being propagated from callee routines to caller. This is similar to static type-checking. I would expect a statically typed system based on type inference to be easier to refactor than one based on static but manifest types. (See TypeChecking for a discussion of terms.)

Apart from that, when you restructure code you have to review the assertions. Some of them will survive the restructuring and, like UnitTests, help ensure that the final code still has the original expected properties. This is especially true of the assertions outside the classes being restructured.

Other assertions may refer to properties that the restructuring has removed, and must be changed or deleted. In fact that's one motivation for refactoring: some assertions, like some comments, are a hint that your code is wrong and should be rewritten to eliminate them.

I'm not sure about "more assertion checking than you needed". Assertions test for impossible conditions. I am always tempted to leave them out on the grounds that they are obvious, or that they can never fail. And sometimes (if I put them in anyway), these are exactly the ones that do fail. 50% of the money spent on advertising is wasted, but we don't know which 50%.

-- DaveHarris

If you are using TestFirstDesign, the ProgrammerTests define what is EnoughVerification. Code is only added to pass a test, implying there should be no untested code. The code may have characteristics not tested, but these characteristics can be viewed as implementation details only.

If, sometime later, these implementation details become significant, then a decision needs to be made on how to address it. You may have to split a class into two or more that support different results for a characteristic. For example, you may have to split a date class into a class that handles complete dates and a class that handles partial dates such as year and month only. ProgrammerTests would also be added to explicitly state what the desired operation is to be.

EnoughVerification is determined by when you stop writing ProgrammerTests and modifying code to support them. It is a subjective decision and the decision can be revisited to add additional verification or occasionally prune unnecessary verification.

-- WayneMack See also UnitTestingIsDesign