The Third Manifesto

Databases, Types, and the Relational Model - The Third Manifesto (Third Edition) by ChrisDate and HughDarwen, 2007

ISBN 0 321 39942 0 Argues that relational database management systems never were fully alive in the first place, since they do not implement the relational model, as handed down by DrCodd, just something quite a lot like it. He's particularly venomous about SQL and its manifold inadequacies. He also effectively fillets pretty much all the current generation of object-relational DB products.

As I understand it, their argument is that object-orientation (by which they seem to mean polymorphic abstract data types) is orthogonal to the relational data model. They call for a new query language, one based on predicate logic [e.g. TutorialDee], rather than sets. And a new family of relational products that use full-blown classes as the domains in their schemas.

The idea is then that a relation is a predicate, rows in a table are tuples of objects that satisfy that predicate. Hence, the advantages of working with objects rather than dumb values are combined with the advantages of the relational model and its implicit navigation.

It's difficult to see though, what impetus there would be for the OODBMS industry to move in this direction, since folks are still buying Oracle, DB2 and such by the skip-load. -- KeithBraithwaite.

It's clear these relational purists are no more pleased with object-relational than the object purists. They offer their manifesto as an alternative union. They carefully define the scope of their manifesto to be dbms. In that sense they are inviting objects into their own house. The rules are clear, though. Objects are to stay on their knees at all times and take on no airs about being an organizing principle. -- WardCunningham

The book does read as if Objects are being included on sufferance. Their view is somewhat better than the "OO data is CODASYL all over again" school of thought, though.

This is an unfair depiction of the book, Ward. The book in fact goes out of its way to reconcile object-oriented theory, insofar as such can be identified, with relational theory. The problem is that there is no single system of object-oriented theory, and many of the common variations have problems (and paradoxes!) that don't gibe well with the simple system of relational theory. So the kind of objects that resulted from Darwen and Date's work seems a pale shadow of typical object-oriented languages. OTOH, their limited form of inheritance avoids some problems that plague many object-oriented systems.

I spent quite a while looking at object-relational systems, and it all just seemed like a bit of hack, obscuring much of the power of the underlying relational system. Perhaps things have improved in the meantime, but I think that TheThirdManifesto heads in a direction that would ultimately be much more powerful.

BTW, make sure that you read the second edition, not the first - there were many, many improvements between the two. -- DanMuller

Objects are to stay on their knees at all times and take on no airs about being an organizing principle

I wonder if it is because objects are often implemented/designed with an emphasis on process over data. In Terry Halpin's book, Information Modeling and Relational Databases, pg 686-687 "Although a rigorous process model is best built on top of a data model, an overview of the processes can be useful as a precursor to the data modeling, especially if the application is large or only vaguely understood. It is often helpful to get a clear picture of the functions of the application first." -- JeffWinchell

It is often claimed by OO proponents that declarative techniques are rigid or don't scale well. However, I have not seen any inspectable evidence for this. It appears to only be a personal bias. -- top

I worked for many years on ExtendedSetTheory databases, which go beyond Codd and Date in providing an even more complete and mathematical set of database principles. This technology was invented by DaveChilds, and is the stuff underlying the programs of Digital Archaeology (http://www.digarch.com) and others. XST is a fascinating approach. Relational math can describe the logic of the data it provides to you, but it isn't strong enough to describe the storage space upon which it is implemented. XST can model the user data, but it can also model the storage, all the way down to the bits.

This is useful because it means that you can define a precise mathematical model that maps any information operation into data operations and then into storage operations.

Surely there's no need for new theory to do that. It's a simple application of representation and abstraction relations.

You wind up with a very powerful way of doing database products. Comshare built a relational product on this technology, and it was quite effective.

The XST math, and to a lesser degree relational math, would seem to be readily implementable in objects, and I've worked on it several times. An odd thing seems to happen, however: it never comes out very nice. I think it's because in objects we mostly think about individual instances, and in RDB and XSTDB we mostly think about aggregates. Informally, objects are about having the one you want already, databases about processing a batch all at once.

The closest thing in objects to the feeling of relational is the Smalltalk collection classes, where you can do an operation over all the elements of a collection. But the RDB XST thing is more like LISP for big objects, where you are recursively applying functions to collections to transform them.

I suspect that there really is a conceptual barrier between object thinking and database thinking. I feel strongly that - at least - we've not found the smooth connection yet. -- RonJeffries

Because TablesAndObjectsAreTooDifferent from a root philosophical standpoint. You can't merge 2 things that are based on fundamentally opposing viewpoints. You would have to give up encapsulation in place of relational operators and accept declarative-based interfaces. It would no longer be OO if you did these things.

I found this book to be fascinating. In the first part of my software engineering career, I avoided database work in favor of system-level programming. (My BS was a combined EE/CS degree, so I got out of college without taking any database courses.) Based on what little I knew about database-related programming, I always turned my nose up at it a bit.

Now, for the past two years, I've been living in an area with fewer employment choices, so I found myself working on a shrink-wrapped Windows application that is heavily database-centric. I've been consuming lots of material on database theory, and Date & Darwen appealed to me very much because of their theoretical rigor. Even though much of what they discuss is not directly implemented by current database systems, it still has helped me understand relational database theory at a deeper level. To my surprise, I learned there's a lot of research still going on in database theory and implementation, and industry has often failed to take good advantage of the research that has been done.

In the second edition of the book, pretentiously retitled FoundationForFutureDatabaseSystems (still subtitled TheThirdManifesto), they have what they believe to be a rigorous and consistent treatment of inheritance, both single and multiple. I find their approach to be extremely thought-provoking. They really aren't talking about objects and polymorphism as implemented in C++ or Java; it's much more like the MultiMethods in CommonLisp. Personally, I suspect this is in fact the direction in which OO has to evolve in order to further simplify the things that are still hard to do in current OO languages.

Date & Darwen have managed to provoke my "language design" itch to the point where I'm working on a general-purpose language that tries to express their ideas in a practical way. I probably won't touch the relational parts of their "TutorialDee" for a while, because I'm more interested initially in exploring the implications of SpecializationByConstraint and MultiMethods.

-- DanMuller

They call for a new query language, one based on predicate logic, rather than sets.

Not exactly. They call for any number of new database manipulation, not only query, languages, all of them adhering to their set of prescriptions and proscriptions. Any such language would be a D, and their specific language is called TutorialDee. These prescriptions and proscriptions are fully fundamented [founded?] on both two-valued predicate logic (True and False, no NULLs or maybes), and on set theory. It is SQL which deviates from set theory, making its tables rather bags than sets, due to duplicates.

I worked for many years on extended set-theoretic databases, which go beyond Codd and Date in providing an even more complete and mathematical set of database principles.

I wonder what is an extended set? Sounds like snake oil for me. A different kind of set most probably is something else. For example, SQL tables aren't sets, but bags, because the have duplicates.

See ExtendedSetTheory for a bit more on this question and the next couple. -- RonJeffries

Relational math can describe the logic of the data it provides to you, but it isn't strong enough to describe the storage space upon which it is implemented. XST can model the user data, but it can also model the storage, all the way down to the bits.

I wonder why one would want to do this. You use the relational model to model the logic level. Than you use whatever you want - hierarchies, networks, files, any kind of graph or list - to map that to implementation. A RDBMS should have all this implementation functionality, but very separate from the logical model; and users should never deal with the physical layer or its mapping to the logical schema, only DBAs and the such should care about this.

I suspect that there really is a conceptual barrier between object thinking and database thinking.

Obviously, because there is no underlying theory behind object-orientation. This is one of the strongest points of Date, Darwen and Pascal. There are no precise definitions, and when there are they are unnecessary. For example, what's an object, a variable or a value? And a class, what's it besides a type?

I found myself working on a shrink-wrap Windows application that is heavily database-centric.

You will want to have a look at AlphoraDataphor. It's been validated by HughDarwen as a valid D.

-- LeandroDutra

Thanks for the pointer to Alphora. I had already discovered it since my addition to this page - actually, they contacted me after seeing something I wrote on a newsgroup. If I were starting an application from scratch, I would definitely consider it, but at the moment I can't get anyone else at work interested in serious study of Dataphor, since we're well on our way to building our own C++ API and database adaptation layer modelled on TutorialDee concepts. -- DanMuller

A Postscript copy of The Third Manifesto paper can be found at http://www.acm.org/sigmod/record/issues/9503/manifesto.ps

-- LeandroDutra

Key claims of TheThirdManifesto are found in DateAndDarwensTypeSystem.

AreRdbmssDead? Well, from that page: They are still extremely useful because their query engines are based on notions from an extremely well-known derivative of set theory. SQL has, at its core a simple, mathematical base.

CategoryBook CategoryDateAndDarwen