Domain Specific Tweaks

DomainSpecificTweaks (DSTs) refers to the less extreme and more common form of EmbeddedDomainSpecificLanguages (EDSLs) where, instead of constructing a scoped sublanguage that follows a fresh syntax, one creates smallish 'tweaks' or 'extensions' to the host language to offer more efficient expression of domain-specific concerns. This distinction is useful. When it comes to ExtensibleProgrammingLanguages and mechanisms for implementing them, DSTs require efforts and integration distinct from full EDSLs. For example, DSTs must naturally interact with one another: if you make four DSTs in a given language scope, then each DST must be capable of ready utilization as a subexpression of any of those other three DSTs. In comparison, EDSLs in the general sense need to be aware enough of their resource context to access objects and services and such (to qualify as 'embedded')... but they do not need to be 'aware' (i.e. integrate, even implicitly) the other EDSLs.

To a programmer, DomainSpecificTweaks often seem like extensions consistent with the rest of the language. In the general sense, even libraries of well-named procedures and OperatorOverloading and use of the MetaObjectProtocol can qualify as DomainSpecificTweaks. It is worth noting that RealMacros, while often lauded for their ability to provide EDSLs, are in practice utilized more often for DomainSpecificTweaks. That is, one can cause RealMacros to parse strings or symbol lists and transform these to arbitrary embedded programs... but such 'RealMacros' would not automatically be aware of the arbitrary other RealMacros in the programming scope.

Support for fully EmbeddedDomainSpecificLanguage, especially if it extends to runtime, may be better for scripting purposes and essentially 'embedding' certain scripts (albeit potentially with CompileTime support, typing, and access to other components from the language), as the whole subprogram can more readily be extracted and moved into other contexts (because one can be certain that there is no dependence upon other EDSLs in the scope).

You cannot obtain arbitrary DSTs or EDSLs - it is never "any language you want". With either of these, you are still limited by the semantics of the host language. For example, if a language provides no mechanisms to support AspectOrientedProgramming (e.g. via extension of the post-processor), then your EDSLs and DSTs also won't support it. Due to EmergentBehavior and FeatureInteraction, KeyLanguageFeatures generally need to be supported more 'deeply' than the shallower 'syntax' extensions typically achieved by use of DSTs and EDSLs. This is a truth that seems to largely obviate most 'semantic' benefits one might attempt to achieve by use of EDSLs and DSTs, reducing it largely to (still important) syntactic benefits of more direct and less voluble expression of desired intent - i.e. reducing 'semantic noise' in the code, making it clearer for both those who know the DSTs and EDSLs. That said, there is much benefit in making said intent more directly accessible for refactoring/optimizing independently of the users.

Most of the StumblingBlocksForDomainSpecificLanguages apply to DSTs. It's interesting to note that since DSTs include domain-specific libraries of functions and macros, this can be said also of such libraries.

I hope the above makes some sense of the ThreadMess below, which seems to be fighting a battle regarding the DifferenceThatMakesNoDifference.

There is a big fight about the proper definition of 'DomainSpecificLanguage', wherein (paraphrased here) SamuelFalvo? operates on the assumption that all programming languages are 'programming specific languages', and his opposition (me) operates on the assumption that 'DomainSpecificLanguage' is described in opposition to 'GeneralPurposeProgrammingLanguage' and thus 'programming' doesn't qualify as 'domain specific'... ass/u/me; it was a big fight about nothing. Most of that particular thread should probably just be deleted and left as a summary; if there is no opposition from SamuelFalvo? it can be done in the near future (20080816).

SmugLispWeenie's claim that Lisp provides a way to create domain specific languages inside itself. It mostly provides domain specific Lisp inside itself, and provides ways to mingle other languages in itself like many other languages do. (That is, if you happen to believe that language is distinguishably different that syntax and notation clearly differentiates a language visually.) If it provided a way to make domain specific languages, it'd let me have languages inside lisp. Languages like the (loop) construct, for example. One way to truly obtain domain specific languages is to fork an interpreter or parse a string.

Sure, but you can't really claim that a DSL embedded in a string is a DSL embedded in the language in which the string is embedded. Nor can you say the same for 'forking' an interpreter over a string or file, which is pretty much the same as parsing a string or file, but with concurrency.

Another way is to write such an interpreter in Lisp itself, interpreting the contents of a string, list, or other data structure.

The SmugLispWeenie's claim it takes too long to make a real domain specific language (i.e. one that could parse a string or be forked). This is due to the cost/benefit tradeoff. In most applications, it isn't necessary to provide a lexer, a parser, a unique symbol table/environment management system, type inferencers, preprocessors, et. al. In most cases, you can get by with the definition of a simple syntax, re-using much of the infrastructure already provided by Lisp. As a result, many DSLs in Lisp do kinda sorta look like they're embedded in Lisp.

However, there are thousands of modules out there that provide domain specific languages already - which can exist in side an imperative program, or be forked as process. There are also DSLs for pure functional languages, too, such as Parsec for Haskell, implementing an LL parser.

For example regex interpreters that parse a string inside your imperative program: 

  // find this pattern in a string called "testing"
  pattern = 't.*st.*';
  find(pattern, 'testing');
I will admit, however, that it is hard to discern how this differs significantly from this hypothetical Lisp code: 

 (find (compile-regex "t.*st.*") "testing")
If we want, we can put Lisp inside our imperative program too: 
  forkLispInterpretter('lispfile.txt');
And we can embed C in Lisp: 

 (fork-gcc-on "c-file.txt")
If we want, we can skip the forking and just parse the string using a Lisp module. 

  import
    LispParser?;


  lispstring = '(lisp(inside))';
  parseLisp(lispstring);
And likewise in Lisp too: 

 (fork-gcc-on-string "#include <stdio.h>\nint main(int argc, char *argv[]) { printf("Hello world!\n"); return 0; }")
And there are plenty of lisp interpreters, regex specific languages and parsers, INI parsers and languages, SQL parsers and interpreters. None of this requires using lisp, in order to make use of a truly domain specific language like Regex, Ini, SQL, etc. But, then again, we all know that the concept of DSLs proves independent of any programming language in which an interpreter for it is written. After all, even Forth has a BNF parser DSL in less than 16 lines of code.

One does not have write a domain specific language (roll his own) each time if it is made easy to fork another generous interpreter that someone else has written, or use a generous module that allows one to work on strings inside another language. Even easier would be a consistent Lisp module that already implements the domain specific language via macros and functions on Lisp's lists. That eliminates the need to munge with strings all-together.

I'm fairly certain forking an interpreter has very different properties than use of an embedded DSL. Among other things, you cannot pass parameters in the same manner, and the namespaces become vastly different (i.e. one cannot readily access objects in the program's environment). The concurrency issue is also relevant.

Having a truly domain specific language like SQL, Ini, etc can be much more useful than trying to use (something (that (isn't a language) but just more lisp)). Writing more lisp inside lisp is just lisp - just as writing a DSL in C is just writing more C, and writing Pascal compilers in machine language is just writing more machine language. It is not a domain specific language, but domain specific lisp.

[By analogy, everything you use in computer science is just a domain-specific assembly language.] --AnonymousDonor

Unless there is an exact 1:1 correspondence between source and the exact assembly, independent of context, there is never a leg to stand on in claiming that languages are just syntactic sugar for assembler.

False on all accounts. The whole reason languages other than assembly language exists at all is to avoid the duldrum of having to code in assembly language at all. Hence, all computer languages, ultimately, are domain specific languages for assembly language. C is a DSL for writing operating systems. Pascal is a DSL for writing applications. Lisp is a DSL for lambda expression evaluation. Forth is a DSL for deep-embedded applications. Etc. --SamuelFalvo?

Consider a DSL for business rules that speaks of business services and contracts, legal business entities and agents, transactions, policies, general rules and exceptions to them. How, SamuelFalvo?, would this be a 'domain specific language for assembly'?

Simple. You wrote, in no uncertain terms, that if you code a domain specific language in Lisp, you are just coding in Lisp. You claim, in no uncertain terms, that the result is not a domain specific language.

No I didn't. Someone else wrote that. I was the guy arguing in italics. I said, in no uncertain terms, that there is never a leg to stand on in claiming one language is just syntactic sugar or a high-level representation for another (in this case assembly) unless there is an exact 1:1 correspondence between source and the exact assembly, independent of context.

I called you on your logical fallacy. The reason is that you can #define any language you damn well please in place of Lisp. Why should coding a DSL in C or Qomp be any different than coding a DSL in Lisp? Hint: it isn't. What you ascribe to Lisp also applies to Qomp, whether you like it or not.

Well, that isn't true even if you were directing your comments at someone else. There are two aspects to defining a DSL, and Lisp handles only one of them - manipulation of semantics (via manipulation of the code post-processor - on a rather small scale, too!). Macros are a very limited means of writing a DSL (though they ARE still DSLs).

If you want an uber-powerful approach, use a combination of a modifiable grammar, Explicit manipulation of post-processor (allowing aspecting and code-transforms that have the whole context), and PartialEvaluation (so the costs of these first two features don't accumulate beyond reason). And maybe add a little CompileTimeResolution and FirstClassTypes if you'd like to grab stuff from outside the code. LispMacros are useful, but they are fundamentally constrained to a view of only their parameters rather than their context. There isn't any reason embedded DSLs can't have every feature someone listed. But it is fairly clear that he hasn't a clue how to get there from where he's standing (I have an advantage of having studied this subject in my spare time for about two years for my own language design.)

And since Qomp, like Pascal before it, is a language intended to automate the process of writing software without the duldrums and slavery of using lower-level program representation, precisely as any domain specific language, one must conclude that this extends all the way down to the machine language level.

Here is the inductive proof:

Note that the form, motives, and end results are identical for both levels of abstraction. We can generalize this infinitely higher levels as well. Thus, inductively, I've proven that your claims are generally applicable to all languages, except only machine language. QED.

The point of lisp is providing a protocol to seamlessly integrate all the DomainSpecificLanguages? a big project needs, thus reducing the impact of VendorLockin? through DomainSpecificLanguages?.

Is that right? That is the point of 'Lisp'? Oh, how naive I was thinking that Lisp was created before VendorLockin? and 3rd party AutomatedCodeGeneration was a large issue. I must be mistaken in thinking there was only one other high-level language around at the time (Fortran). I've always thought that the 'point' of Lisp was to create a practical high-level language for List Processing, Math, and other things based on Alonzo Church's LambdaCalculus. But my misunderstanding is corrected. I thank you for it.

The misunderstanding is possibly due to programmers not being used to long-term-evolving languages in these days, where the typical language is embraced and then forgotten when the next hype is bought.

These edits were lost when I submitted my changes to this page. I am really not liking the "Wiki cannot respond to your request" pages. :(

Fully agreed. I'm trying to write a WikiIde with real-time updates...

I declare that Lisp encourages people to make all sorts of Lisps (rather than promoting clear separate languages like say a true SQL or a true non lisp language). And this is the problem with readability - everyone reinvents their own lisp (not a specific clearly defined and structured language).

Most languages offer ways to tweak themselves. Tweaking themselves to be better selves should only go so far - otherwise your language is just a hodgepodge of NothingIsAnything or AnythingIsNothing?. Whether there are truly separate languages existing in a language or not is a LaynesLaw issue regarding the definition of WhatsaLanguage, I suppose. If a language allows modifications inside itself - that is the language you are working with which provides you DomainSpecificTweaks! It's not really a new language, necessarily (LaynesLaw issue).

This page was therefore positively renamed to DomainSpecificTweaks.

Be warned - that too many tweaks, causes complexity and/or unreadability for other programmers. Using too many tweaks is selfish and goes against sharing code with thousands of people. Consider C++. Consider why Paul Graham's store was rewritten (because, apparently, the industry was just not smart enough.. or maybe the code was really complex and tweaked too heavily for real practical world use).

Matz for example says: "A DSL is just an extension of (Ruby's) syntax (with methods that look like keywords) or API that allows you to solve a problem or represent data more naturally than you could otherwise."

Domain specific extensions, or tweaks, in some people's opinion - is a better way to describe the situation than "language".

AprilZeroEight

CategoryLisp

EditText of this page (last edited July 8, 2010) or FindPage with title or text search