Mutable Language

Problem:

Domain specific LittleLanguages are of proven value. However, there can be a violation of OnceAndOnlyOnce - each new little language may re-implement features found in other little languages.

Therefore:

Use a language where the syntax can be changed by the user (aka MutableLanguage) to tackle the kind of domain specific problems where little languages excel.

But:

The price of metamorphic languages is *potentially* very very high. If it's too easy to add language features, people will add too many. People will add them for trivial things, and people will add many similar "personal" features instead of adding a single robust and necessary feature.

The key problem is that readability of the source code (for someone not initiated into the new syntax) may not be good at all. (See StumblingBlocksForDomainSpecificLanguages.)

OTOH, it is possible to write unreadable code in any language. It is arguable that political problems shouldn't be solved by technical means, but rather by cultural ones such as social pressures and idealist memes. If one has a highly metamorphic language, it may be best to promote writing code such that "it looks like pseudocode" to the uninitiated - or any other principle or convention to help drive programmers in a common direction.

Discussion:

Separate the mutability bits

I would prefer to reserve a special kind of source file for language extensions. Almost like XML description files (and I think you'll find a lot of inspiration in XML ideas). This is not a technical requirement rather it is imposed by the system. Just as C has conventions like .h files and .c files, the programmer would be encouraged to adopt the convention of having language modifying components, such as new dyadic operators, declared in a single header file. Even a large project would be unlikely to have many such changes.

New Names, not New Operators

The 'danger' of people extending the language in obscure and unhelpful ways is similar to the 'danger' of allowing people to write and choose names for their own subroutines. Which they can already do badly....

Readability is OK when extensions to the language have readable descriptive names e.g:

   typedef sparse_array_of wormholes [0..1023, 0..1023, 0..1023]  is universe;

In this example whilst you may not have seen the type constructing operator 'sparse_array_of' before in your life, it does not act as a big a barrier to your understanding the code.

But, readability does suffers if people misuse operator-like syntax.

In language based on C, using $$ to mean something like (but not quite the same as ) -> would be hard to justify, unless you were entering the obfuscated code contest.

For normal use the *tokenizer* (which should be part of the IDE) would be off limits. This prevents some of the most pathological redefinitions, such as redefining " or ; for another purpose.

It could be handy to be able to define new token types (for example, to have Perl-style regexps in a library). To avoid pathological redefinitions, it should be sufficient to reserve a character for delimiting user-defined tokens.

Summary:

Adding new descriptive keywords is not hugely different from adding new (named) functions.
Adding new operators such as '$$' is likely to compromise readability. You may be better off adding a descriptively named operator, and renaming it only if a widely accepted convention arises that covers it.
Changes to the tokenization process in a language are generally ill advised, unless you are trying to mimic an existing widely accepted language's syntax.

Does anyone have experience with Mathematica's internal language? You can define new operators, prefix, infix, and postfix. You can use any symbol available in its fonts, which track the TeX math extension fonts. You can use its patterns to define any sort of rules you wish. I tried doing things with this, but it gets clumsy and hard to remember.

LispLanguage avoids the syntax problem by not having any. Hence macros are encouraged in Lisp, as espoused in OnLisp.

If you don't trust your people to use the macro facilities wisely, you have an organisational problem, not a technological problem.

Overloading of operators to mean obscure things is a kind of historical peculiarity of C++. A metamorphic language could enforce minimal standards of comprehensability, for instance by requiring new keywords to be of a certain length.

CategoryLanguageFeature