Poor Mans Testing Framework
Poor Mans Testing FrameworkPlease allow me to set something straight:
Let me walk through this to make sure I'm clear:
Pretend you're writing multiply() (I don't know why you would do this, but it's the example everyone uses, so bear with me.). Let's work a poor man's unit testing example in CeeLanguage, knowing that we can apply most of the same techniques in other HighLevelLanguages.
The first test looks like this:
int main() {
if (6 != multiply(2, 3))
puts("6 != multiply(2, 3)");
return 0;
}
Attempt to compile and run it. We get a failure, of course (because we haven't written multiply()).
Jot a multiply() down (hard-coding the result to get rid of the evil "Red Bar" as quick as possible):
int multiply(int a, int b) {
return 6;
}
Compile and run. Good. (SilenceImpliesConsent in the Poor Man's Testing Framework).
Now then, before we move on, we look around for duplication. Sure enough, we see it, right there in main(): the "6 != multiply(2, 3)" is repeated once. Honestly, I can't think of a clean way of getting rid of it yet, but since it's only one duplication, I'll just forge ahead and see if I can't figure out a way to fix it after the next test.
Now, we hard-coded the "6" as the return value. Better add another test to bring out the need for real code inside multiply():
int main() {
if (6 != multiply(2, 3))
puts("6 != multiply(2, 3)");
if (42 != multiply(6, 7))
puts("42 != multiply(6, 7)");
return 0;
}
Compile and attempt to run. Of course, it fails. So we flesh out multiply:
int multiply(int a, int b) {
return a * b;
}
Compile and run. Good, it passes. So let's get right to the refactoring.
First of all, I hope you noticed the violation of ThreeStrikesAndYouAutomate in "compile and run". Hopefully, by this point, you would write a Makefile or equivalent to automate this step. From now on, I'll just say "run".
Next we need to get to that icky pile of duplicated conditions filling our main() function. Let's pull it out:
void assertMultiplication(int expected, int a, int b) {
if (expected != multiply(a, b))
printf("%d != multiply(%d, %d)\n", expected, a, b);
}
int main() {
assertMultiplication(6, 2, 3);
assertMultiplication(42, 6, 7);
return 0;
}
Run tests. Pass. Good. However, we're not sure if the tests actually worked the way we expected. So, let's break our implementation for a moment:
int multiply(int a, int b) {
return 1234;
}
Run tests, expecting two failures. Good, we got them. Now, let's hit our editor's undo button until we get back to the correct implementation. Re-run tests, expecting pass. Good, got it.
When we write the test cases and implementation for divide() (again, I have no idea why we need these functions), we'll end up with something like this (skipping ahead a bunch):
void assertMultiplication(int expected, int a, int b) {
ASSERT_EQUALS("multiply", multiply, expected, a, b);
}
void assertDivision(int expected, int a, int b) {
ASSERT_EQUALS("divide", divide, expected, a, b);
}
#define ASSERT_EQUALS(functionName, function, expected, a, b) \
int actual = function(a, b); \
if (expected != actual) \
printf("%d != %s(%d, %d)\n", expected, functionName, a, b);
After I wrote this, I realized that I wasn't outputting the "actual" number anywhere. Fortunately, since we refactored, all I had to do was change the one printf:
printf( \ "Expected %d from %s(%d, %d), but got %d\n", \ expected, functionName, a, b, actual \ )Baby steps, running the tests, refactoring in between. That's it! Eventually, we won't have a Poor testing framework at all.
(Note: Of course, you'd probably want to check out an actual unit testing framework for your language after you've bootstrapped. I just want to make it clear that you don't have to have one to get going.)
Hey poor, you don't have to be poor anymore. PMTF is here! (Sorry, couldn't help myself)
The impoverished of spirit may seek a hand up from OnceAndOnlyOnce.
Too much work. diff expected.output actual.output
I've done this one on some projects. But then you have to output everything. Internal subroutine calls and comparisons give you much better control. -- JeffGrigg
Thanks, you! This is a really good simple example of how you can create simple tests for your language. Of course, you can go to the TestingFramework but this piece of explanation is really good for newbies.
Thanks !!!! -- Fabrice
Good start. But the missing ingredient is ThreeStrikesAndYouAutomate. The code may be minimal, but at error time you still must...
The next step bonds with your editor. VisualCeePlusPlus contains a minimal CppUnit knock-off that obeys both OnceAndOnlyOnce (roughly) and ThreeStrikesAndYouAutomate (aggressively).
I use the C preprocessor 'stringize' operator to remove the "6 != multiply(2, 3)" duplication:
#define ASSERT_EQUAL(e, a) if ((e) != (a)) printf("%s != %s\n", #e, #a)
#define ASSERT(e) if (!(e)) puts(#e)
int multiply(int a, int b) { return 0; }
int main(int ac, char **av)
{
ASSERT(6 == multiply(2, 3));
ASSERT_EQUAL(10, multiply(2, 5));
return 0;
}
-- PhilippeAntras
Way! -- PhlIp
The beef I have with poor man's Testing (not that I don't do it in time of need) is that automation of the entire testing suite must be maintained by hand as you go. When there must be manual maintenance of a list of tests, more effort is required, fewer tests will be written, and mistakes will be made. It's better if the act of writing the test automatically informs a testing framework of the existence of another test to run. For my compromise, see VbLiteUnit. VbLiteUnit may be inelegant in that it keeps all the tests for a test case in one big procedure, but it has the advantage of not requiring (sometimes unreliable) heroics to identify the tests in a language that doesn't do introspection well, and it still does have sufficient safeguards against common copy/paste errors, etc. -- SteveJorgensen
For small projects or for algorithm experiments, separate test files using a full-blown UnitTest framework can be overkill. PythonLanguage's built-in "assert" is handy for creating poor-man's tests in the same file as the code. I often use "assert" to write some tests which run when the module is called as the main program. The tests are designed to pass silently and fail noisily. I can even do TestDrivenDevelopment this way. An example:
#!/usr/bin/env python
def factorial(n):
if long(n) != n:
raise ValueError("factorial arg must be integer: %f" % n)
if n < 0:
raise ValueError("factorial arg must be >= 0: %d" % n)
if n >= sys.maxint:
raise ValueError("you must be crazy to want %d!" % n)
fact = 1
for i in xrange(2, int(n)+1): fact *= i
return fact
# # # # # # # # # # # # # # # # # # # #
if __name__ == "__main__":
import sys, math
for input, expected in (0,1), (1,1), (2,2), (3,6), (4,24), (5,120):
assert factorial(input) == expected
assert factorial(float(input)) == expected
big_arg = int(math.sqrt(math.sqrt(math.sqrt(sys.maxint))))
assert factorial(big_arg) > sys.maxint
for bad_input in -5, 3.14, -3.14, 0.2, -0.2, sys.maxint, sys.maxint+1:
try:
factorial(bad_input)
print "factorial failed to raise error on bad argument value!", bad_input
except ValueError:
pass
except:
print "should never get here!", bad_input
If this particular file is supposed to be called as main, then I encapsulate the tests. I put the tests in a _test() function and call it only when I pass "-t" on the CommandLine.
I HaveThisPattern. Unit tests and wiki share the fact that the concept is more valuable than the codebase. Here is the UnitTest framework I wrote (and I use) for MicrosoftAccess VisualBasic. I throw an error, and use the debugger to find where the error occurs. -- StanSilver
'===========
'Test Engine
'===========
Sub Assert(blnDesiredTrue As Boolean)
Dim intMakeError
If blnDesiredTrue = False Then
intMakeError = 1 / 0
End If
End Sub
Sub AssertNot(blnDesiredFalse As Boolean)
Dim intMakeError
If blnDesiredFalse = True Then
intMakeError = 1 / 0
End If
End Sub
Function RunAllTests()
'Execute all functions that are defined in the "mdlTests" module
'If an error occurs, use the debugger to find the line that makes the error
AllProcs ("mdlDevTests")
End Function
'From Microsoft Access Help
'ProcOfLine Property Example
Function AllProcs(strModuleName As String)
(lots of code here to run all procs in a file...)
'=====
'Tests
'=====
Function TestAge()
Assert (Age(#7/15/1959#) = 46)
Assert (Age(#1/15/1959#) = 46)
Assert (Age(#1/1/2000#) = 5)
Assert (Age(#12/1/2000#) = 5)
End Function
Function TestFormatTime()
Assert (DevClassFormatTime("10 AM", "dummy") = "10:00 am")
Assert (DevClassFormatTime("10PM", "dummy") = "10:00 pm")
Assert (DevClassFormatTime("10:20PM", "dummy") = "10:20 pm")
Assert (DevClassFormatTime("10:30 am", "dummy") = "10:30 am")
Assert (DevClassFormatTime("9:00 PM", "dummy") = "9:00 pm")
End Function
EditText of this page
(last edited June 24, 2006)
or FindPage with title or text search