Trust The Metastructure

The single greatest failing of every language-neutral binding to date has been the abysmal absence of effective metastructure. The prototypical effective metastructure was demonstrated by Smalltalk80 more than twenty years ago. It is, alas, hard to grok metastructure. It requires work -- e-f-f-o-r-t -- and so the overwhelming majority of developers don't, or won't, bother to understand it. Our entire industry suffers.

Today, with XML/SOAP, we have an opportunity to do it right. I hope we embrace, rather than blow, the chance. The key is to use, effectively, XML as an intermediate language-neutral notation. The semantics of an object-oriented environment can be captured entirely in its metastructure. These semantics can be expressed, elegantly, in XML/SOAP. Once expressed, they can then be emitted in the target language of choice (just like using YACC/LEX) so that the actual code runs in the language of choice -- including the metastructure and binding back to the intermediate-language representation.

The result is that the intermediate language becomes, in essence, the DNA of the environment (the genotype, if you will), with each language binding being a phenotype.

Here's a more specific description of the process. Those who have access to IbmSmalltalk can see this in action by examining -- closely -- the behavior and interaction of EmClassDefinitionDescriptor? and EmClassCreator?.

Start with a full CausallyReflectiveMetastructure: Smalltalk is a good example. Multiple efforts have demonstrated that it is tedious but straightforward to extend the Smalltalk metastructure to specify, for example, a strongly typed environment (it involves adding metastructure entities that model variables, associating a class with each, and adding a return type specifier to CompiledMethod?).

Transcribe a set of instances into the intermediate representation: Write an analog to EmClassCreator? that traverses a collection of objects, emitting XML/SOAP (or any other suitable intermediate language -- this can be done, with more difficulty, with IDL) that correspond to connected instances of EmClassDefinitionDescriptor?. Yes, there will also be EmMethodDefinitionDescriptor?, EmMemberDefinitionDescriptor?, etc.

Map the intermediate representation into the target language: Traverse the network of descriptor objects (I use the VisitorPattern), emitting the target language.

Apply this process to itself: Feed the translator itself through itself, so that the translation is expressible in any of the target languages.

Apply the translator to your favorite tools: Like the Smalltalk debuggers, inspectors, and so on.

Yes, you will need a set of "primitives" that are used to describe the behavior of the resulting environment. Yes, you will need a mechanism for extending these. XML/SOAP provides the necessary magic in the intermediate representation (CDATA, etc). Smalltalk and Java demonstrate how to effectively capture these in an environment (UserPrimitive?s, JNI, PlatformFunction?s, etc.).

The result is an environment with consistent semantics that can run in whatever language environment you like. The next step is to ditch the files and use persistent objects instead, but that is a different rant.

TrustTheMetastructure, Luke.

This could use some more explanations. What are the problems caused by lack of metastructure in language-neutral bindings? Could you explain more about how IbmSmalltalk does things for those of us who don't have it?

When a particular element of semantics is missing from the metastructure (or if the metastructure is altogether missing), then that element must be supplied ad-hoc by the binding. A CausallyReflectiveMetastructure is the most reliable mechanism that currently exists for ensuring that behavior is rigorously constant across multiple bindings. In SmalltalkLanguage, the metastructure includes Class, CompiledMethod?, Process, and Block (plus lots more).

Unlike Java, changing the metastructure in Smalltalk changes the behavior. In the most trivial example, adding a new instance of CompiledMethod? to a class enables every instance of that class (and its descendents) to respond to the new message. No classes need to be "recompiled", "reloaded", or anything similar.

IbmSmalltalk uses a combination of EmClassCreator? and multiple instances of EmClassDefinitionDescriptor? to accomplish changes to the class hierarchy. Sometimes called "SchemaMigration?", this solves a number of problems older Smalltalks exhibited when the class structure was changed. A crucial problem was how the system handled changes that broke an existing instance of Class while that class had instances in the environment.

The general approach of EmClassCreator? and EmClassDefinitionDescriptor? something like the DirectorPattern?. A single instance of EmClassCreator? is created in order to accomplish a particular change. When a new class needs to be created (even if it is a new form of an existing class object), it begins as an instance of EmClassDefinitionDescriptor?, a descendent of Class. The instance of EmClassCreator? constructs a tree of EmClassDefinitionDescriptor?s, reflecting the desired configuration of the new classes. When the tree of descriptors is finished, the director creates a new class object for each one. If any correspond to existing classes, the class creator checks for existing instances. These are mutated into the new class. When all the magic is done, the newly-minted class objects are inserted into the class hierarchy, replacing any old contents. Because the instances of existing classes were each mutated into instances of new class objects (using #become:), the old class objects are guaranteed to be garbage and will be discarded in the next gc pass.

CategorySemanticWeb