Guru Checks Output

CategoryAntiPattern: GuruChecksOutput is not a method of TacticalTesting.

Alternately, consider GuruWritesAutomatedTest, GuruChecksChanges.

GuruChecksOutput occurs when only the author of a program or an equivalent guru can tell whether the test cases succeeded or not. It's fine until the guru starts his own company and code maintenance is left to the newbies. -- JohnFarrell

FWIW, I think GuruChecksOutput is not fine at all. Manually checking output is a poor use of a good brain and is too slow. Using automated tests, one can often change a few lines of code and then run a useful set of tests in a few minutes or less to ensure that no new errors were introduced. With manual checking, even a guru just can't go as fast.

Have that guru (and everyone else) writing automated tests using JavaUnit, CppUnit or whatever TestingFramework applies, not as a hedge against future developments but to work better and faster now. --KielHodges

That sounds overly absolute and context-free. Consider CleanRoomMethodology as a variant of this, which has value in some circumstances.

It's neither overly absolute nor context-free to prefer automated testing over human testing. There is no requirement for non-automatic testing in CleanRoomMethodology. Kiel is absolutely right IMO. --RonJeffries

Given the statistical nature of testing in the CleanRoomMethodology, I'd expect automated testing (including RegressionTesting) to be common. The study described in CleanroomSoftwareDevelopmentEmpiricalEvaluation, for example, used automated testing. -- JeffGrigg

I agree that GuruChecksOutput is not fine at all. Sadly it is the de facto standard. If I'd thought it was fine I would have made it a technique of TacticalTesting. -- JohnFarrell

See GuruWritesAutomatedTest:

With RegressionTesting the GuruChecksOutput the first time the program is run, to see if it is correct, but future checks are done automatically. When later changes cause a test to fail, a guru may have to check the output to see if the change is output is a bug, or an intended new feature. This can save a lot of time.

Realize that manual checking of program output is tedious and error prone (in addition to being slow, as mentioned above). Your guru will make many mistakes, missing many errors, until they quit in frustration. -- JeffGrigg

GuruChecksOutput is an oxymoron, IMO. A Guru will automate testing to save time and assure thorough testing, unless Guru has an insane manager who disallows GuruWritesAutomatedTest or test automation is impractical/impossible. For example, rumors say that shock absorbers under buildings inside Cheyenne Mountain have not been tested because the test requires exploding a nuclear bomb over Cheyenne Mountain.

I wrote automated tests everywhere I worked. In all but one case, bad infrastructure assured that my test scripts, test programs, test documentation and other test artifacts were forgotten and/or deleted when I left an organization. GuruWritesAutomatedTest is at best a partial solution for GuruChecksOutput.

IMO languages and IDEs with built-in testing features, enough to support some form of WhiteBoxTesting, might be enough to change the de facto standard for testing to GuruWritesAutomatedTest. At least test code survival should be improved if the application code contains the test code, rather than the two being in different files.

COBOL has a facility to include debugging code within application code. The SOURCE COMPUTER statement has an optional "DEBUGGING MODE" clause. The debugging mode controls whether some parts of the program are comments or active code. This facility can be adapted to insert test code at strategic locations in a program. It is a useful technique but not a complete solution for integrating test and application code.

IDE enhancements should include showing and not showing test code. I for one, do not want test code cluttering the application code when reading application code. When test time comes around, I do want to see the test code. Also, the IDE should warn that test code is active before compiling and/or running a program.

This idea of enhancing languages and IDEs is a brain-storm. It seems incredible that no one else has considered the idea. I have used DEBUGGING MODE to conditionally insert test code in a program, so the idea does work, technically. Why is it not a standard feature of languages?

COBOL slammers, forget making derogatory remarks. COBOL is my least favorite language, but in the mainframe era there were a lot of COBOL jobs.

-- EdwinEarlRoss

CategoryTesting