Automated Continuous Testing

An extreme form of AutomaticTesting for the EclipseIde -- ZeroButtonTesting:

Many modern IDE's provide continuous compilation, which may speed software development by providing rapid feedback about compilation errors as source code is edited. Continuous testing extends this idea to provide rapid feedback about test failures as source code is edited. To support the intuitive appeal of this idea, we are evaluating the potential benefits of continuous testing, using data collected from real developers. We are looking to find the theoretical limit of the productivity gains such a tool could generate, and also the benefits that could be gained from tools built around particular continuous testing strategies.

From http://pag.csail.mit.edu/continuoustesting/

Having read through the PDFs on the above site (quite quickly), I have a question: does continuous testing inside one's IDE help if one is doing TestDrivenDevelopment? One runs one's tests after implementing a new test or refactoring. While one is working, one's compile and tests are quite likely to be broken, so running tests in the background doesn't seem useful. When one is finished with an addition or refactor, one needs to know that all one's tests pass RIGHT NOW, so waiting for a background process to run the tests seems counterproductive. What am I missing? (I'd just try it, but the tool referenced on the above site works only in Eclipse; I'm an IDEA man.) -- DaveSchweisguth?

The continuous testing tool costs nothing and is open source, released under the Eclipse Public License. If anyone's interested in running with an IDEA port, I'm all for it--I looked into it a year ago, and the IDEA plug-in support just wasn't good enough.

As far as whether AutomatedContinuousTesting is useful when doing TestDrivenDevelopment, that's the model that most users (as far as I know) are using it with:

Tests aren't run if the code doesn't compile.
The tool prevents you from incorrectly thinking "that refactoring was so small, I don't need to test." The tests get run anyway, and you might be surprised at the outcome.
In the middle of an edit, you might know that one of your tests fail, but expect none of your changes will affect other tests. With AutomatedContinuousTesting, you'll find out immediately if you're wrong.
The background process that runs tests is no slower than manually starting the tests when "RIGHT NOW" comes.

Finally, in the future, we'll be able to take advantage of the fact that when "RIGHT NOW" comes, the only tests that need to be run are those affected by your last edit, not those affected by every edit since you last thought about testing. AutomatedContinuousTesting is not a cure-all, but I think it's fun to use, and I'm always interested in feedback.

-- DavidSaff

See also ContinuousTesting (not necessarily automated).

CategoryAutomated CategoryTesting