Pointer Metaprogramming

OK, I admit it... I'm posting this page mainly to be play the DevilsAdvocate. (The fact that this disclaimer is necessary just goes to show how well-received this topic is likely to be.) But given that DevilsAdvocate is a fun and productive game to play (and I firmly believe that the world would be a better place if more people looked at the other side of issues rather than merely basking in their own self-assuredness), here is my (latest) attempt to shove a stick into the pot and stir the proverbial hornet's nest.

PointerMetaprogramming is the practice, commonly found in languages such as CeeLanguage and CeePlusPlus, of using techniques such as PointerArithmetic, forced typecasts, and other forms of (hopefully) ImplementationDefined behavior in order to engage in MetaProgramming. Not all forms of PointerMetaprogramming necessarily require use of pointer operations; but most do involve pointers at some level.

PointerMetaprogramming is a very powerful metaprogramming technique, as it allows the programmer to manipulate program data in unlimited ways; ways typically disallowed by the facilities present in most high-level languages. Many types of systems programs (including the runtime systems for most high-level languages) require use of PointerMetaprogramming; which is why one seldom encounters (for example) a JavaVirtualMachine which is written in Java.

PointerMetaprogramming is also a controversial technique (like many other techniques for MetaProgramming), it is often alleged that many programmers lack the skill/training to do it correctly. PointerMetaprogramming is often subjected to the following criticisms:

While a properly-written program using PointerMetaprogramming invokes (at worst) ImplementationDefined behavior; many improper programs invoke UndefinedBehavior; in most such instances, the language and its toolchain cannot determine the difference. (As a concrete example - use of PointerArithmetic to navigate a data structure in C. If you do it wrong, it's undefined - and the implementation cannot determine when it is valid and when it isn't).
The reliance on ImplementationDefined behavior makes many such programs unportable.
- However, this can often be abstracted away. The "vararg" facility in C is a classic example of PointerMetaprogramming made accessible to the application programmer. The results of this have been...fascinating.
- Also, in many cases (such as many systems programming tasks) the application in question is highly tied to a specific machine anyway.

AlternateHardAndSoftLayers is a good way to partition a program into those parts which need the power of PointerMetaprogramming, and those which do not (and can be written in a higher-level language).

Key techniques of PointerMetaprogramming:

PointerArithmetic. The use of arithmetic operations on pointers (above and beyond the practice of traversing an array within it's declared bounds) can be productively used to traverse data structures, examine the ActivationRecords of functions, implement dynamically-sized arrays, and such.
PointerSwizzling?.
PointerBitStuffing?. The runtime systems of many high-level-languages use TagBits to efficiently encode and store metadata about objects and object references, such as whether a given word on the stack is a pointer or not. This technique is also useful for coloring the object graph in a tracing GarbageCollector.
Pointer casts. One hallmark of PointerMetaprogramming is the construction of pointer casts which are neither proven typesafe by the compiler, nor verified by a runtime typecheck (yet which are correct). This technique can be productively applied to implement polymorphism, both generic and subtype (see PointerCastPolymorphism).

Several languages support PointerMetaprogramming "out of the box".

CeeLanguage and it's OO cousins (CeePlusPlus, ObjectiveCee).
Many Lisp dialects.

Other languages allow it, but only in distinguished code modules (those marked as "unsafe" or with some other pejorative keyword).

Others disallow it completely, like JavaLanguage. (However, even languages which disallow PointerMetaprogramming do provide a ForeignFunctionInterface which can be used to AlternateHardAndSoftLayers, thus providing an "escape hatch").

In some languages (many of them typeless/single-type languages where the only type provided is a machine word), virtually all programming is PointerMetaprogramming. One classic example is C's predecessor BeeLanguage. Another is ForthLanguage (though that comment ought to be taken with a grain of salt, as there really isn't a single language called "Forth", and some Forth dialects do impose types).

See also PointerCastPolymorphism and some related discussion on FirstClass.