Badly Formed Persistence Layer

Write a beautiful Persistence layer, mapping a relational database to an OO object model. Write a beautiful Business Objects layer, expanding each Persistence object into a Biz object aware of its roles. Then write an awesome Biz Rules layer, mixing and matching these objects with carefree disregard to the fact that they come from a database - they construct fully populated.

Sounds good, so far. Now attempt to instrument the Biz Rules layer with UnitTests (after the fact). See the problem? You can't feed the Biz logic fake data, because each Biz Object had to come from the database, and you really don't want your UT to depend on a live dB somewhere with an exact set of test data in it. The Persistence layer and Biz Objects layer contains dozens of scattered calls that assume the presence of a database, but not one of those calls has any test to see if the database singleton itself has an open connection, nor any graceful failure path.

(The fix is known as a ShuntPattern, and is rarely thought of during development.)

The Anti-AntiPattern fix might be to whack the Biz Objects layer and spoof your own, right at the level of all those objects. This is a total pain (dozens of Biz objects to write instead of just a few Persistence support object), but its better than simulating a database and parsing the SQL sent to it. -- PhlIp

[From ClassicOoAntiPatterns]

What is the objection to having a static test database to test against? Doesn't seem like a big problem. If all you do is load the objects, it is easy. If your tests have to perturb the database state, either copy the tables or re-import before you start - neither of these is usually difficult in an RDBMS, and usually you don't need a very large volume of data for UnitTesting, so it shouldn't take long, and can be scripted. Unless you in-line all your test data in your UnitTests, you have to keep it someplace - why not in an RDBMS?

How many static test databases do you need? One for each developer and one for acceptance testing? How much effort do you need to spend maintaining your static test database to keep it in line with the production database? When you copy the tables, what about the triggers? The constraints? The views? Trust me, you really, really don't want to go this route. You'll end up spending many, many hours setting up simple tests. The testing work will dwarf the actual work by an order of magnitude. -- GeorgeDinwiddie

I thinks its more of a cultural thing, if developers have databases of their own that they can play with it not a problem, if they have to go thru DBA.s the overhead and the lack of skills make it almost impossible to do.

WhyDoYouLetPeopleDoThatToYou?? Good programmers should not work at a place where DBAs lord over all that is relational with an iron fist. DBAs can be a wonderful resource for design questions and troubleshooting, but when bureaucracy gets so thick that programmers aren't allowed to manipulate their own database, its time to pack your bags. ThatsNotNormal? and too many people accept it as so.

Also see UnitTestsAndDatabases -- AndyMorris

WyCash took the database duplication approach and made it work. (See WyCashIncrementallyAddsDistribution.) The key is to write one program rather than just a bunch of independent layers and require any off the shelf code you bring into your program, including the database, to work your way. WyCash started with it's own database and then ported to a networked record manager. (Thank goodness for the tests.) Ten years later it is still satisfying customers without modification. Remember, the larger and more 'complete' a database product you incorporate the more likely it will demand your full attention. Big vendors don't really have your best interest at heart no matter what they say. -- WardCunningham

In Oracle at least you can export a database account to a binary file and check it into source control. That includes tables, views, stored procedures, triggers, everything. When you want to use it, import it into an empty database account and go. You can have as many files as you want. -- BrianSlesinsky

Even if you have some strange aversion to prepopulating a test database, I don't see a problem. Most test programs construct objects needed for the test. What is fundamentally different about persistent objects that the test program itself cannot create them as needed?

Good call, PhlIp.

In my current (but not for much longer :-) contract, some of us finally got the team to understand that separating persistence from business logic is a GoodThing and is not at all 'TakingThingsTooFar? because in the RealWorld there is NotEnoughTime'. We built a pluggable Registry for our persistence layer and an InMemoryRegistry? implementation of it. This served us well.

Alas, certain members of the team didn't see why they had to implement the TopLinkRegistry? methods that they were declaring on the Registry interface and implementing in the InMemoryRegistry? and they got away with this because we decided to put the TopLink implementation in a different Eclipse project, so they never saw the compilation problems. In addition, the InMemoryRegistry? had reached a considerable level of sophistication which caused certain developers (yes, the same ones) to forget over and over again that it was actually just a prop; they kept putting code into the InMemoryRegistry? that was clearly business logic and would be required regardless of the Registry implementation. This code was also often unsupportable in the TopLinkRegistry? (and probably in any implementation involving an RDBMS). This allowed them (they thought) to claim that they had finished their work.

After several months of this behaviour, those of us that were suffering from the pain of having to re-implement lots of code for allegedly 'completed' stories decided that we'd had enough. The InMemoryRegistry? was duly deleted, much to the chagrin of many, causing a significant drop in 'velocity', quite a large dose of reality to be taken and also, unfortunately, the UnitTest run times to become untenable. This did bring about a significant refactoring of the tests and we were able to bring the run time down to 10 mins (2000 tests).

By the way, if some of you reading this are thinking that we could not have implemented XP particularly well in order to allow these things to happen, you are correct. An excellent ProjectPostMortem? opportunity...

-- LanceWalton

The above problems seem to me to indicate a failure in project management to separate and place due importance on the concerns of various subprojects, rather than any conceptual problem with the InMemoryRegistry?. Separate compilation of a Registry interface (with a notification process whenever it was changed, so that everyone affected gets told) and a requirement that any changes to implementations of it MUST be made to ALL implementations would have solved the problem without bloating the run time of the UnitTests. (Of course, if you can't get your developers to understand and follow an agreed-upon design, then you have problems in your company that a new architecture simply won't fix... any improvement will be temporary. Perhaps you need some new developers? Or at least more education of the ones you have.)