Call And Check Result

Call and Check Result is a method of TacticalTesting.

This is a very simple way to write a test program that can be easily used by a quality assurance person or someone else whose technical abilities you don't respect :-). It's best if used with ReportBugsSilently. The traditional way to write a test program is to try some test cases and print out the results. If they look OK to you, they must be right. I call this GuruChecksOutput. It works OK while the guru is around.

In call and check result, you effectively wrap the calls to the API you are testing with methods that know the answer. For example, if you have a method boolean Object.equals(Object), you would have in your test program a method void testEquals(Object, Object, boolean).

This method calls .equals(Object) on the first object with the second as the parameter. If the result is not the same as the boolean given, it reports a bug. Using this tactic, your test programs can be interpreted by anybody. Choosing the test cases is the next problem.

This pattern is not restricted to comparing results for equality. It could check that the correct exception is thrown, or apply some boolean function to the result to check that it at least looks right. Do whatever you have to do to make sure your code works.

-- JohnFarrell

In my opinion, serious testing always involves an automated test suite. GuruChecksOutput does not count. When I change a system, I want some easy way to know whether I have broken anything, and the best way to find out is to run its test suite. I may be an expert programmer, but I often change systems that I didn't write, and it takes too much time to figure out how to test them. Moreover, if someone writes an automated test suite then it can be reused over and over. It just takes a few seconds to start running a test suite, then you can go do something else. Running the test by hand and looking at the results will take hours or days, even if the tests themselves can run in a few minutes.

Is CallAndCheckResult the same thing as "automated test suite"? If so, maybe you should change the name. Or is it just a particular way to implement a test suite. In that case, you need to motivate it better. What other ways are there?

It is much easier to implement tests if you have a TestingFramework for them. You need UnitTests and AcceptanceTests even if you aren't extreme. These kind of tests are all standard testing dogma, and have been for a decade or two. The problem, as Brian Marick says, is to get people to write them.

-- RalphJohnson

I suppose what I am on about with TacticalTesting is:

what would a TestingFramework need to look like if it was a way of life rather than a thing the new guy was forced to use two weeks before release to manufacturing?
what new stuff can we invent that would be cool in testing frameworks?
how do we tackle the really hard stuff, like once in a lifetime fluke bugs (HeisenBugs)?

I admit CallAndCheckResult is what you do in an AutomatedTestSuite?, but I am hoping that it is only one technique of many that will come under TacticalTesting. I think because testing is unpopular, the technology to support it is pretty shallow and immature compared to what it could be. In my tactical testing dream, the developers love to write tests ... What I want to know is, HowCanYouCodeWithoutTesting? -- JohnFarrell

Answer: Using the CleanroomSoftwareEngineering, developers write and release code without ever testing it. -- AnonymousDonor

CategoryTesting