Extending Perl Unit

Just to be really clear, the following discourse is really old, and no longer particularly relevant. -- AdamSpiers

Those looking for a Perl xUnit framework might also be interested in the [http://search.cpan.org/dist/Test-Class/] module. It's more integrated into the Test::Harness / Test::Builder testing framework so you can use most of Perl's many testing modules with it.

I coded a first cut at PerlUnit, but it needs to be tested and extended. My reason for posting, is in case someone else is doing the same thing. IMHO, a merged program is better than multiple versions chasing each other over the internet. -- CayteLindner

(OK, PerlUnit is not really there yet; but we'd like to encourage the effort. ;-)

ftp://bio.perl.org/pub/katel/UnitTests/

Question: How is this different from the Perl's Test.pm module? -- PeteBevin

Test.pm doesn't even attempt to solve the TestIsolation problem. This is extremely irritating, and in fact when I hacked in Perl and was using Test.pm I had to wrap all my tests in a giant eval {} block to make sure the database got cleaned out. -- GrahamHughes

The code mentioned above should only be downloaded from the bio.perl.org address, the others have old code, which needed the bioperl distro. There is another effort, listed here:

http://www.cpan.org/modules/by-module/Test/Test-Unit-0.06.readme

by ChristianLemburg.

It Would Be Nice If the Test::Unit was integrated better with Test::Harness (the latter being the one you get when building a perl package and doing make test). This could be done by creating an adapter class, with an import method like this:

 package Test::Adapter;
 sub import { die "test adapter needs two numbers" unless @_==2;
              print shift."..".shift."\n"; }
 # platform specific code below. leading path may need remove in real life
 my $suite=$0;
 $suite=~s/\.t$//; # trailing t for Makefile.PL removed.
 eval <<END_SUITE;
 use $suite;
 # this must include code for printing the 'ok N's
 # alternatively something like the JUnit texui goes here.
 $suite::run();
 END_SUITE
 # That's it!

And then use it like this:

 # Entire script follows...
 use Test::Adapter 1,12;

Called from a file called My::Test::Suite.t this would include and run the test suite in My/Test/Suite.pm, and so on. As long as the 'textui' equivalent (which really should go in this adapter class) emits the 'ok/not ok' stuff that Test::Harness depends on, this will make unit tests compatible with business-as-usual perl.

One problem is that Christian has written his whole oeuvre as a single class, and **passes in** the fixtures. Does this not look odd to anyone else? The test code has been separated from the test data. The suites are meant to be run from an (arbitrary) class, instead of subclassed, which makes it harder to run sets of suites automagically.

Another thing about Test::Unit is that it doesn't eval the tests - so the entire program may bomb out for a single test failure. Debatable this one, since it avoids cascade errors while using the same fixture.

-- BrianEwins

Responding to BrianEwins:

1) Integration with Test::Harness: Agreed. At the start, I simply did not want to complicate things with integration to Test::Harness. Your suggestion for sure looks interesting. If you have started to code it, why not make it into a module and put it on the CPAN, too? I gladly offer any assistance needed here.

2) Why everything as one class, and why passing in the fixtures? Two reasons: First, I want the module to be usable in a non-OO-style environment, too (e.g., for simple scripting stuff), second, because a lot of Perl programmers don't understand how to subclass and extend a given class (for example, look at the questions on how to use HTML::Parser etc. in the perl newsgroups). If you can convince me that there are good arguments for the subclassing approach from a user's point of view, I will try to extend the interface to support both OO style use (subclassing and extending) and "standard" use, like the CGI module does. I am not sure I understand what you mean by "The test code has been separated from the test data." (besides the obvious point, that you can use non-OO-style test routines).

3) Test::Unit does not eval the tests: I did this because I was not sure if it makes sense to go on with testing if you have errors at a level that causes a fatal error. It might make sense to alter this.

Anyway - thanks for your feedback!

-- ChristianLemburg

Christian - I'll bundle up Test::Adapter and use it for some simple tests on your own module - it would make sense if they were bundled together.

I wrote a whole pile of answers on continuing after eval failures, so much that I've started a new node: FixtureInfection .

Brian,

I agree on the bundling together thing. How should we do this? Create a little "distribution"?

On the eval issue: I will alter this behavior in the next release of Test::Unit, adding an "Errors" category to the result report.

What about the one-class and pass-in issues? Do you think the effort to provide an alternative interface is worth it? In the beginning of writing Test::Unit, I wondered if I should just carbon-copy-reimplement the Testing Framework, but decided against doing this, since I felt the Testing Framework is too pure-OO-style compared to Perl (if I wanted to program in Python I would program in Python ;-). I tried to stay with the spirit of unit testing, though, so please tell me if you think this is compromised by my implementation.

-- ChristianLemburg

Christian - CayteLindner 's code is much, much more like the 'original'. It may be that rather than mixing programming styles it would be better to provide something like LWP::Simple (have you seen this?) wrapped around Cayte's code. There's stuff not quite the same - the assertions are part of UnitTestCase?.pm, instead of being their own (independently useful) utility class as in JUnit, all of the assertions require __LINE__, __FILE__ put explicitly into the call instead of using 'caller' as you do.

Myself I think the assertion code should be in the TestResult class. What I mean is that the TestResult class is responsible for recording assertion failures. In Cayte's code (particularly UnitTestCase?::run) the TestResult object isn't passed in, and so it doesn't collect the results; instead, the AssertionFailure? class prints out the results (note that changing this behaviour in Cayte's framework means overriding *every* assertion method, so that it calls something else).

Time for me to mail Cayte I think...

-- BrianEwins

Thank you for the suggestions. I plan to investigate them this weekend. The bioperl group would also like PerlUnit integrated with the perl test harness. I may need to ask questions. I just learned perl last summer and I coded up PerlUnit on the Xtreme premise that unit tests are the first step in coding. Time to move beyond the Blue Camel book:) Maybe we can discuss what make's Christian's version more convenient for some apps, and incorporate those features into a common framework. -- Cayte

More stew for the pot: I've just done a fairly literal translation of SUnit 2.3 into idiomatic Perl (see http://views.vcu.edu/xunit/). The main wart is working around Perl's non-object exceptions. It includes the SUnit test class, but not the test runner. I was planning to use it on my next small project before foisting it on the world, but then I checked RecentChanges...

I think it would be reasonably easy to modify to allow Christian's non-subclassing style as well as the SUnit style.

I agree that integrating whatever comes out of the stew with the test harness would be a good idea. -- JeFritz

Folks, let's agree on some joined effort here. After skimming through Cayte's and JeFritz's code, JeFritz's code looks like the most promising of our three efforts to me. I think we should extend his work and throw away the other stuff when we are finished.

For a start, I suggest that our joined effort to provide a real Whatever-it-is-called-Unit should meet the following requirements:

Resemble the interface and behavior of JUnit (to me, this looks like the most accessible and widespread of the "authoritative" implementations of the Testing Framework).
Provide an interface as convenient (read "lazy") for users as possible.
Support integration with Test::Harness stuff (seems to be the most requested feature).
Support subclassing-style use and non-subclassing-style use.
Be as platform independent as possible.
Follow standard CPAN module conventions for installation etc.
Be able to be "distributed under the same terms as Perl itself," i.e. redistribution under either the Artistic License or the GNU Public License should be allowed.

-- ChristianLemburg

Ok then. To move this on, I've taken the liberty of registering a project on Source Forge (http://sourceforge.net/) which will allow y'all to collaborate via cvs etc. It takes 24 hours for them to reply but I'm pretty confident this one will get set up. Its registered under the name PerlUnit, has a project description lifted from Christian's last comments, and is registered as being under the Artistic license (which is what Christian and Cayte were using). As soon as I get notice back I'll upload all the modules. If you want to work some other way, that's ok, I just thought that this might be a useful way to go.

-- BrianEwins

Brilliant! ThankYou Brian. Things happen fast these days ... BTW, would you mind joining in? You seem to be really interested :-). -- after seeing the environment at sourceforge, I am more than impressed. Brian, you have done us a great service here.

And one thing more (also CayteLindner and JeFritz): could you introduce yourself somehow? I like to know a little bit about the people with whom I am working - it just feels better to me. I just added a page about me. Thanks to all for doing this!

-- ChristianLemburg

Wow! Brian, I add my thanks to Christian's (and echo his hope that you'll join in).

With one reservation, I agree enthusiastically with Christian's bullet points. The reservation concerns JUnit - I haven't investigated enough to know if I agree there or not. I have two concerns:

I'd want to make sure we didn't include things that got into JUnit only because of Java's strict typing.
I'd like to be able to say "distribute under the same terms as Perl itself," i.e. to allow redistribution under either the Artistic License or the GNU Public License. I'm not sure that JUnit's license is GPL-compatible. (The GPL is really picky.)

I think Test::Unit is a good name, by the way. --Fritz

I agree completely on the license stuff. We should definitely try to distribute our effort "under the same terms as Perl itself". I have added this to the bullet points above to reflect my support (I always have to remind myself that this is not a newsgroup, but a wiki page, and that I am supposed to change things written before :-).

As to JUnit, I am not sold on it, and I definitely agree with the remark concerning avoidance of Java-specific features (e.g., look at the hoops Cayte's code has to jump through to make assert_equals work).

If you like the name Test::Unit, I am gladly offering to "surrender the namespace" ;-) to our joined effort (sounds funny, eh ... but see how excited people can get about this in any perl module forum).

- ChristianLemburg

CategoryPerl