Advantages Of Exposing Run Time Engine

This page addresses the question: what is a RunTimeEngine??

Tentative description: A "database" or data structure containing the variable names and content, operators, function definitions, etc. RunTimeEngineSchema is a toy example of one.

Sometimes, it would be nice if the run-time engine of a program's structure was more easily exposed to the programmer as a data structure or database instead of via special syntax. Here are some of the benefits:

Debugging - Easier to see, filter, and query stack traces
Dynamic Meta-ability - Program or routines could see anywhere it wants in the stack, for example. One is not limited to the language's view. Adding new language or language-like features is a matter of reading or tweaking the right spot in the structure. For example, if a language does not support closures, then obtaining the needed scope is a matter of the proper query.
Easier alterations - Run an update-query to make mass code changes rather than having to parse first. Parsing carries many risks, such as mis-parsing, outdated parser, mixing of multiple languages, etc.
Cataloging - Information about programming units can be added to the structure. For example, programmer ID, classification meta-data, text key-words, etc. could be added to routines, classes, modules, etc. as needed.
Understanding - Knowing the schema of the run-time-engine may help one better understand the language. They could also see how their program is translated into structures. It is like a washing machine with a glass door instead of an opaque one.
Display variations - One can display the information in different ways. For example, I always liked the way Pascal (and OOPascal) puts the modifiers and types after the variable declaration instead of before (like in Java). If the program is in a structure, then whether these labels are displayed before or after is a "report writer" detail. One can tweak their view to how they see fit. See CodeAvoidance and SeparateMeaningFromPresentation.
Multiple languages for the same structure - Turn syntax into an implementation detail. See SyntaxFollowsSemantics.
- Can't be done, language syntax is different for different languages, because the languages are actually different from each other. Syntax is more than simple text parsing, it's a representation of a feature, and those features aren't common to all languages, and can't simply be translated from one syntax to the next. For example, Visual Basic has no concept of closures, anonymous functions, or even first class functions, so you can't just trivially translate Lisp or Smalltalk to VB, you'd have to fake a ton of non-existent features first. C doesn't have objects - same problem, you have to fake the feature first, to even attempt a translation. The whole reason a language is created in the first place is to express an idea or feature in a way that existing languages don't. People have tried this before and continue to fail. If there were a bunch of languages that could all be expressed via a generic meta-syntax, they'd be fairly worthless as languages, since they don't offer anything unique anyway.
- If the internal language is TuringComplete, of course it can be done. I agree that some things are harder than others, but certainly not impossible. Closures are mostly a matter of fiddling with the scope a bit, which is not necessarily hard. You just look at a different part of the stack or a different routine.
- [Not entirely disagreeing with that sentiment, but you're wrong about closures. Closures "close over" a piece of code, taking with them any bindings referenceable by that code. A closure can be saved and used much later - which implies that bindings on the stack either can't stay on the stack, or you have to use a non-traditional stack that's actually a linked list of frames with indefinite lifetimes. That's a very brief description, but introducing closures to a language is not a simple task at all. You'd have to plan for such things from the start. -- DanMuller]
- Perhaps we can play with the concept using the RunTimeEngineSchema toy.
Better chances to do things like MetaDebugInterface.

-- top

Quite a few languages do this already, including SmalltalkLanguage. LispLanguage exposes quite a bit of its internals as well, though you seem to not like Lisp very well. Of course, hiding the runtime engine has a few advantages as well - whether or not these tradeoffs are worth it is application-dependent.

DisadvantagesOfExposingRunTimeEngine

Performance. Many of the optimizations done to make languages like C (as well as statically-typed functional languages like ObjectiveCaml) fast involve severely munging the runtime data structures, including removing lots of stuff that might be interesting to the programmer. Furthermore, many such optimizations are machine-specific, and imposing a strict binary interface would eliminate the possibility of such optimizations.
- It depends on what kind of operations are to be performed based on the information. Generally the information is there. Otherwise debugging, dynamic linked etc. wouldn't be possible. It is just not that they are exposed. Changing the structure may be expensive, but this is a secondary extension to the pure exposition and then e.g. Java seems to manage things like hot code replacement fairly well.
Security. Mainly an issue in Java... there's a reason that Java doesn't allow applets to go mucking 'round with the VirtualMachine internals.
- Perhaps make some attributes read-only. We should explore some UseCases.
  - That suggestion is rather naively optimistic. You'll never achieve security by opening the system then finding places to close it. Consider whitelisting attributes rather than blacklisting them.
- Security is an issue in much more than Java. Anywhere you have mobile code, security will be an issue. If you have a closed runtime engine, you can produce certain guarantees about the operation of code just by looking at it. If not, you'll have a far more difficult time... because one piece of code can affect how other pieces of code will run.
- It is principally impossible to prevent inspection of code executed on the client. So I don't see a security problem here. Though I agree that SecurityByObscurity would be more difficult.
Discouraging ThreeStarProgramming - I've seen far too many "runtime hacks" in my life, where obscure features in the runtime (often intended only for debugging or not for app programmer use at all) are examined/tweaked by application code, often for some minor benefit (or just to show off!), resulting in code that is brittle, unportable, and hard-to-understand. (Of course, many of them are in C/C++, where the runtime data structures aren't "officially" exposed - nefarious attempts to modify stack frames outside of official language facilities are considered UndefinedBehavior - but it happens nonetheless. Knowing how to frob ActivationRecords is often considered a requirement for C wizard-hood, and many OperatingSystems provide libraries to support such frobbery.)
- With dynamic meta-ability often comes opportunity for abuse. This is probably one of those fundamental rules of the universe. Perhaps abuse can be limited by making some things read-only, but even relying on readable values can cause some odd bugs if one is not careful. Perhaps "exposing" is not quite the right term because many dynamic languages already offer ways to "see" internal (and sometimes change) structures if you know enough about the language. I am simply suggesting that the same access be done by treating the internals like a data structure instead of using peculiar syntax to access it. Thus, I am not necessarily suggesting more access, just more comfortable access. This implies that I think data-structure-like or DB-like access is superior than syntactical approaches to the same info, which is basically true. Data structures need access API's anyhow. Requiring a different set of access protocols via syntax is a violation of OnceAndOnlyOnce in my opinion.
- Requirement for C wizard-hood? That was one of the first things I did when learning C!
  - Yes, but on how many different architectures? Just because going 8 bytes past first automatic variable gets you the frame pointer on one architecture, doesn't mean it will work on all of 'em? Plus, all these damn ReducedInstructionSetComputer architectures are such a pain in the ass to deal with. Sticking parameters in registers, where I can't get to 'em without __asm instructions? Whose bright idea was that? :)
    - Oh. . . I just put random values into DS, always set the SP to zero, and set the IP to whatever INT 33 returns. Is that bad? I assume that would work flawlessly on non x86 processors
I disagree. Having a clean exposition of structure is better then inferring undocumented structures and working around them - which will happen in any case - if possible at all.
Constrains the implementation in other ways. Some "features" that look good on paper are hard to implement on certain architectures (or require emulation to do them efficiently).
- An issue of WhenAreStandardsRestrictive perhaps? What is an example of "hard to implement"? Database implementations are usually pretty solid.
  - Good database implementations don't traditionally expose their particular implementation or runtime to the queries they run.
  - I don't see the connection between these examples. Perhaps "run-time engine" is a bit misleading. Run-time "structures" may be a better description.

Languages which expose their runtime engines:

ForthLanguage (however, it is not required in ANSI standard Forth, 99.999% of all implementations do to some non-portable extent)
LispLanguage Well, I'm not sure; almost everything can be reflected in Lisp, but I don't know about what is really the runtime engine part of Lisp.
LuaLanguage? (metatables)
PythonLanguage (how? Well, you can replace the global name lookup function, which allows you to replace the default implementations of most of the basic builtin functions (and keep access to them if you want to), and you can replace the lookup function at various other places too. One of Guido's mottos is "we're all consenting adults here", and he won't try to stop heavy wizardry because it can be useful.)
- JobSecurity in the hands of the evil
SmalltalkLanguage
UserlandFrontier

Related: There are approaches that try to keep structural and consistency information between transformations from the source language down to the machine code. See e.g. TypedAssemblyLanguage. Though this information is only avaialble to the intermediate phases of the compiler it is a kind of exposition or rather a connection of the higher level structures with the lower level ones.

CategorySourceManagement