Key Language Feature

A KeyLanguageFeature is a feature in a programming language which according to the BlubParadox in PaulGrahams BeatingTheAverages, has the following properties:

In several parts of the below discussion people also assumed that the feature should be "essential", which is not actually part of the given definition of a KeyLanguageFeature.

The list of candidates for such features below is currently unsorted; as a possible refactoring it might be nice to sort it somehow. Note the term objects refers to any datatype here; and does not necessarily imply what is commonly referred to as ObjectOriented.

The maybe list:

Things explicitly not included; move these above if you think they should be included (and say why):

Ok, I think somebody has an attitude about what a key language feature is. Considering that IO, Linkage, and GOTO are on the list of explicitly not included, while Reflection is on the list of must have makes this list rather off the wall.

You misunderstand the list. The categorization is NOT one of 'must have' and otherwise. It is a list of 'definitely is a KeyLanguageFeature' (in accordance to the definition at the top of this page) vs. 'useful, but either not subject to BlubParadox or not Domain Independent'.

No language will ever get off the ground with IO.

Perhaps. But plenty of languages have gotten off the ground without standardizing IO - i.e. without making IO part of the language specification. That said, I happen to be a distributed systems weenie, and I am all for putting IO and related features (pickling, moving executables, etc.) as KeyLanguageFeatures, but I'm willing to acknowledge that most problem domains really don't require it.

No language can grow beyond the vision of its creator without low-level Linkage available (hint: CompileTimeResolution will not get you this; it is irrelevant to the issue): it is the only to access OS features that do not exist on the designer's platform.

Access to the OS is often well abstracted as a set of services, modules, or procedures in a language whether or not the language is interpreted vs. compiled. Where does 'linkage' as a language feature come into that? In any case, 'linkage' doesn't seem very domain independent as a language feature. I do agree that CompileTimeResolution won't help for OperatingSystem access; it helps more for CompileTime access to foreign resources (be they text files, remote databases, or ELF executables).

A few years ago I would have said unsafe casts are rare but mandatory. I now find that they can be subsumed into the core by adding a handful of core library functions that convert between scaler types and arrays of bytes without changing any bits.

If you think about the rule about writing an interpreter to simulate a different language to simulate a different language, there is no getting out of the need to add linkage to assembly code as sometimes you just cannot do without it. I really shouldn't have to monkeypatch some other process besides my own, but sometimes it is necessary.

Now, dependency information between the various features:

RecursiveProcesses? require Functions. Should be obvious.

RecursiveProcesses? require RecursiveDefinitions?. Also should be obvious.

RecursiveProcesses? require (LIFO Memory Allocation or Enforced TailCallElimination).

Dynamic Memory Allocation implies LIFO Memory Allocation. If you have a heap, you can simulate a stack. Simple enough.

GarbageCollection implies Dynamic Memory Allocation. Doesn't make any sense without it.

LambdaExpressions requires HigherOrderFunctions. Should be obvious. Note that the converse is not true; C/C++ has HigherOrderFunctions if you count function pointers; but it doesn't have lambda expressions. See next rule why not.

LambdaExpressions requires GarbageCollection. It's often said that garbage collection is required for a FunctionalProgrammingLanguage (especially one with SideEffects; without side effects reference counting will suffice). This is one reason why. The result of a LambdaExpression must almost always be heap-allocated, as they do not obey LIFO discipline. Furthermore, the usage patterns of lambda expressions (they get passed around like a cigarette) makes manual memory management of these dang near impossible. For this reason; I claim that lambda expressions require GarbageCollection.

Thread mode stuff

Regarding Preprocessors and Text Macros

This is an interesting entry given the genesis of this page. From a SmugLispWeenies' point of view, this is the poster child for the BlubParadox -- anyone who thinks that macros are an optional feature (let alone evil) is programming in Blub where Blub is a language that either doesn't have macros at all (e.g. Java or Python) or has a horrendously broken thing that it happens to call macros (e.g. C's text/token macros). Lumping preprocessors and macros (in the CommonLisp sense) together is probably not going to lead to clarity.

Good point; I'll happily separate preprocessor text macros (which perform substitutions on the text before the bulk of the scanning and parsing is done) with language-aware semantic macros ala CommonLisp. However, an interesting question still remains: If you accept the viewpoint that macros operate "outside" the core language; they are often are used to automate things that the core language cannot. Which leads to two viewpoints: 1) Macros reveal a deficiency in the core language, rather than using an "outside" mechanism, the core language should be modified to accommodate. This may be more C++ bias (BjarneStroustrup is known to despise the C preprocessor; many features in C++ were put in to eliminate a common preprocessor use). 2) Macros are a legitimate tool; and as they operate using the standard capabilities of the core; they allow the language to be extended in ways which do not compromise the core. Macros are a layer on top, and layering is good. Personally, I tend to fall in between - when I use a macro, I wish there were a better way. Perhaps this is again C/C++ bias; though I've yet to encounter a preprocessor whose semantics were completely clean and neat. -- ScottJohnson

Hmmm. I vote for 2) but I'm not sure I buy your definition of "outside the core". In CommonLisp, macros are part of the core which, by their presence, allow other things to be left out of the core. For instance, in CommonLisp most of the "control constructs" that programmers normally use in are macros on top of more primitive control flow constructs. (For instance, all the structured looping constructs in CommonLisp are built on top of a primitive that is essentially the same as C's goto.) Thus those control constructs are not part of the "core" the way for and while loops are in C/Java/Python/Perl. (They are part of the language standard - most of the language standard is really specifying the standard library, not the language core.) Also perhaps worth noting: CommonLisp macros are not a preprocessor - they are more like a hook into the compiler itself. Basically whenever the compiler hits a "call" to a macro it passes the macro form to the macro code which then returns a form that the compiler compiles in the place of the original form (which might entail expanding macros that occur in the new form). If that doesn't seem clean and neat enough for you, I'd be curious what seems unclean or unneat about it. -- PeterSeibel

In Lisp and Scheme, "macros" are just ordinary code that happens to execute at compile time, and as such they are utterly different in nature than text macros in other languages. #2 is definitely the only one that applies, not #1, in Lisp-family languages. [Nitpick: Scheme has two macro systems, and in one of them macros are not just ordinary code.]

The word "macro" has always meant, as the core of its definition, "something that happens at compile time, not run time", which is why the word "macro" is used in both Lisp and e.g. C, even though the constructs thus referred to are otherwise unrelated. Mentally substitute the phrase "compile-time first-class function" for "Lisp macro" when in doubt.

Thus it is mostly uncontroversial in the C/C++ world that C/C++ macros are mostly a bad language feature (although essential in C and in a few contexts in C++), yet uncontroversial in the Lisp world that Lisp/Scheme macros are mostly a good and essential language feature (although susceptible to misuse, as with anything).

There is no contradiction; the "macros" in question are quite different things in the two language families.

-- DougMerritt

So that's what LispMacros are... ForthLanguage also has the notion of compile-time (immediate) vs. run-time semantics. One can define a Forth word to have either or both types of semantics. As with Lisp, it can be used to good effect to create domain specific languages and to extend the compiler. Also as with Lisp, all the flow control words in Forth are immediate words implemented using a couple of branching primitives. This allows the user to extend the Forth compiler with novel flow control constructs.

[Lisp Lisp Lisp. Macros Macros Macros. Blah blah blah. If macros and lisp are so great, let's see them implement TutorialDee or a similar query language right into the Lisp program using Macros. Good luck. And no, I don't mean just a bunch of brackets smashed together that kind of look like a half assed macaroni/nail clipping based relational language in oatmeal.]

TutorialDee could be done. SQL has been done. Admittedly, LispMacros don't offer considerable manipulation of syntax (Lisp has no real syntax) so it would look 'ugly' by some peoples' standards (and beautiful by others...). What's up with your ranting? Weenie FeatureEnvy?

Actually, I'm envious of Algol style syntax - and using Lisp macros I couldn't implement the most important feature I needed - Algol derivative syntax - which TutorialDee and Cee/Oberon style languages have. In other words, if I was using Lisp - I would be very envious of Algol style languages. If a weenie steps out of first person SmugLispWeenie view for a moment - he can see that all the snobbish arguments for lisp can be used against lisp. Its strength is its weakness. One can always fork a Lisp process from within an Algol program if they need Lisp, too.

It seems wrong to complain that a feature not meant to deliver some other feature you desire isn't delivering it. A bit like saying: "Tables suck! They don't give me secure communications over a network!" Admittedly, macros and syntax are somewhat more related, but they're still distinct facilities. Macros allow compile-time execution of code. Extensible syntax allows manipulation of the parser. Lisp has the former and lacks the latter. Either of them offer mechanisms for embedding DomainSpecificLanguages, and they combine in a rather powerful way, but they are distinct features.

It doesn't provide me with a DomainSpecificLanguage, it only provides more Lisp.

Ah, you must be promoting your strange idea that language = syntax and is independent of semantics.

The idea that it offers a DomainSpecificLanguage is actually just another way of saying it provides more Lisp. Providing domain specific languages can be done by forking processes in most languages - so I don't consider it a feature of Lisp to provide domain specific languages (think about it: since LispDoesNotProvideDomainSpecificLanguages?, it just provides more lisp). Forking an interpreter (such as forking a PHP interpreter from a Cee program, or forking a Python interpreter from a Cee program, or forking some PascalScript? interpreter from a FreePascal program) provides more power. First, someone has already written the interpreter or domain specific language which I can fork and make use of. Second, these interpreters are actual languages - not just more Lisp on top of Lisp. They truly are domain specific languages (consider forking TutorialDee compiler or interpreter).

Another domain specific language is Regex. Lisp is not the only language capable of extending itself - consider a regex interpretter built into an executable (Cee or Freepascal program). Consider I wrote an SQL interpreter inside a program. SQL and regexes are truly domain specific languages - whereas Lisp on top of Lisp, is just Lisp - it isn't domain specific language. It's maybe domain specific Lisp.

SmugLispWeenies go on to argue that it takes too long to write languages that are forked or parsed. They argue that Lisp offers us this power of writing a language inside Lisp. But, Lisp is just Lisp. It isn't a domain specific language - it's more like domain specific Lisp - which is the very flaw of lisp, in that it does not create domain specific languages... but supposed domain specific Lisp (which forces one to program with odd functional syntax, which is not very domain specific). Forking interpreters that are already written and ready to go - is much more domain specific. Parsing a string inside a program, is more domain specific (parsing INI, parsing SQL, etc). And don't think that one has to write his own SQL or INI interpreter - their are already plenty written available as modules.

SmugLispWeenies are contradictory: the domain specific Lisp languages are not languages at all. They are just more lisp on lisp. However, if I write an SQL interpreter or Regex interpreter module for a Cee program or Freepascal program, this truly is domain specific. Actually I don't have to write a domain specific language many times - there are plenty of existing domain specific languages available that can be forked (perl, awk, regex, php, INI parser, pascalscript, JScript, web language could be forked, BrainFuck could be forked, compiler could be forked and launch a program right after compiling, etc). I can reuse a domain specific INI, Regex, or SQL parsing module over and over again in any language that can fork a process or parse a string.

This is truly domain specific: being able to utilize INI syntax, regex syntax, SQL syntax, script syntax, right inside a program on a string (or on a file that is read). One could even fork a Lisp interpreter - if they needed - but I think rather a more domain specific language should be forked. See the irony: lisp isn't very domain specific. In other words, its strength is a weakness - and its claims are jokingly recursively contradictory.

The assertions found in the above italic paragraphs are bizarre - somehow forking an interpreter constitutes embedding, but a language whose syntax consists of EssExpressions isn't a DomainSpecificLanguage? I think PaulGraham would point to RTML as an example of a DomainSpecificLanguage. It's hard to deny that it's domain specific. Is it not a language because it's got Lisp syntax? Isn't Lisp a language?

Incidentally, contrary to the admissions made by the non-italic participant in the preceding discussion, CommonLisp does allow manipulating the parser, through the use of reader macros. It's just that it's far too painful to implement any language that isn't LL(0) that way, and, as execrable as Lisp's syntax is, it's hard to design something better that's LL(0), especially if you also want to be able to use macros to manipulate it. (I would ask defenders of Lisp syntax to consider why all-capitals is hard to read, and compare that to an all-round-bracket syntax.) 

 You are in a maze of twisty little parentheses, all alike.
Actually, I would argue that Lisp is (with the possible exception of syntax) the ultimate interpreted language. Yes, Lisp compilers exist, but Lisp's features can (as noted by Greenspun) be implemented in other languages by writing a Lisp interpreter. Whereas the features missing from Lisp (other than nice syntax) that make Lisp look a little Blubish to, say, SmugHaskellWeenies? can, as far as I can see, only be implemented by writing a compiler (loosely speaking, that is - Haskell interpreters exist, but they do a compiler-like amount of static analysis at load-time).

I would agree that the italic rant is a bit weird. He talks about "just" forking processes, but ignores the overhead that simple forking requires; and I don't think he understands how difficult it is to mingle domain specific languages when you have to "fork out" to get those advantages. And, finally, he's assuming that there exists specific languages for your domains, and that if there isn't, it's a simple matter of firing up your favorite parsing tools, create a compiler (or interpreter) for your new domain, and write your language, all in one go! What are you supposed to do, if you don't want to go through that work -- or, for that matter, can't know the syntax of the yet-to-be-defined language, because you don't know what it's going to look like? Well, I would suppose that you can start with a simple, easy to manipulate and extend syntax, that already comes with an industrial-strength language behind it, and then gradually extend its syntax piecewise and experimentally, until you have a brand new domain specific language that can be optimized to near-C levels if necessary, and can be easily mixed with other existing (and even developing) domain specific languages where those domains intersect...or, I suppose you could just write a compiler, and fork it...

Somehow, I have the feeling that if forking processes really is more convenient than creating new domain specific languages as needs arise, that SmugLispWeenies would have figured that out decades ago. But then again, perhaps they did...you could find C and FORTRAN compilers for Lisp Machines...but then yet again, perhaps that's just a fallback to the idea that "once something is written, you shouldn't rewrite it, but just let it be, in the language that it's originally written in, for all sorts of different reasons!" -- Alpheus

Re: LambdaExpressions:

(Is this different from the definition of LexicalClosures below?)

My distinction between the two is: LambdaExpressions might not have access to the referencing environment in which they were created; and can be used in languages which don't have nested lexical scoping. A C++ FunctorObject (ignoring the GarbageCollection issue noted above) could thus be considered a form of a LambdaExpression. LambdaExpressions need to be FirstClassObject?s to be much use. A JavaInnerClass? object declared within a method is kind of like a LambdaExpression (it does have access to the environment; but only to variables declared final. In reality; the variables are copied into the object, and no reference is kept to the enclosing scope beyond creation).

A LexicalClosure, on the other hand, does have access to the referencing environment in which it is created (including the ability to modify that environment); but need not be first class. LexicalClosures aren't that tricky to implement until you try to have last longer than their referencing environment; in which case they become a royal pain (they tend to require spaghetti/cactus stacks). FirstClass LexicalClosures are nice to have, obviously; as they generalize both concepts. However, they complicate the implementation of a language greatly.

Of course, I may be all wet here.

-- ScottJohnson

See also QuestForThePerfectLanguage

AprilZeroEight

CategoryProgrammingLanguage

EditText of this page (last edited November 8, 2014) or FindPage with title or text search