Are Design Patterns Missing Language Features

On various places, it has been claimed that use of DesignPatterns, especially complex ones like VisitorPattern, are actually indicators that the language being used isn't powerful enough. Many DesignPatterns are by convention rather than encapsulable in a library or component, and as such contain repetition and thus violate OnceAndOnlyOnce. If it didn't contain at least some repetition, or something that could be Refactored out, then it wouldn't be a pattern.

Discussion on this topic culled from elsewhere on WardsWiki:

Here is an interesting quote from PaulGraham, which leads to the question "Are Patterns a LanguageSmell?"

This practice is not only common, but institutionalized. For example, in the OO world you hear a good deal about "patterns". I wonder if these patterns are not sometimes evidence of case (c), the human compiler, at work. When I see patterns in my programs, I consider it a sign of trouble. The shape of a program should reflect only the problem it needs to solve. Any other regularity in the code is a sign, to me at least, that I'm using abstractions that aren't powerful enough - often that I'm generating by hand the expansions of some macro that I need to write.

from http://www.paulgraham.com/icad.html

PeterNorvig argues in the same vein in "DesignPatternsInDynamicProgramming".

PaulGraham said "Peter Norvig found that 16 of the 23 patterns in Design Patterns were 'invisible or simpler' in Lisp." http://www.norvig.com/design-patterns/

"Pattern" generally implies there is some kind of duplication. Duplication should ideally be factored out (OnceAndOnlyOnce) so that only the differences remain. It appears to me that stronger languages can more easily remove such duplication because sometimes one has to make a kind of sub-language to do it.

A list of DesignPatterns and a language feature that (largely) replaces them:

VisitorPattern .............. GenericFunctions (MultipleDispatch) FactoryPattern .............. MetaClasses, closures SingletonPattern ............. MetaClasses IteratorPattern............... AnonymousFunctions (used with HigherOrderFunctions, MapFunction, FilterFunction, etc.) InterpreterPattern............ Macros (extending the language) EvalFunction?, MetaCircularInterpreter Support for parser generation (for differing syntax) CommandPattern ............... Closures, LexicalScope, AnonymousFunctions, FirstClassFunctions HandleBodyPattern............. Delegation, Macros, MetaClasses RunAndReturnSuccessor......... TailCallOptimization Abstract-Factory, Flyweight, Factory-Method, State, Proxy, Chain-of-Responsibility....... FirstClass types (Norvig) Mediator, Observer............ Method combination (Norvig) BuilderPattern................ Multi Methods (Norvig) FacadePattern................. Modules (Norvig) StrategyPattern............... higher order functions (Gene Michael Stover?), ControlTable AssociationList................Dictionaries, maps, HashTables (these go by numerous names in different languages)

Makes me wonder why we continue to put up with languages that don't have these features!

For the same reason we use the shoddy bug-ridden self-contradicting obsolete language called "English"!

Because we were taught inadequate programming languages as children?

Because we didn't have a choice, at best we can become multi-lingual - but we still have to talk to people who think English is "the best language".

I'm curious: What (natural) language is the best language, and what metric would determine such a thing? I'm a native speaker of English, and I think it a reasonable language. I learned it because that's what my family speaks, and that's what they speak where I live. (That said, my wife is Chinese, and our son is comfortably bilingual). I certainly won't claim it to be the best, it certainly has its flaws (rather irregular spelling and grammar top the list, along with a wide variety of dialects and imprecision in technical matters - compared to, at least, German). It also certainly has its strengths - chief among them is that it's easy to speak BadEnglish and be understood. Also, English speakers and authorities are generally not concerned with the PurityOfEnglish; hence the language is quite malleable and adaptable.

Some of this debate coincides with the debate over whether DesignPatterns should have a symbolic manipulation system. - see the PatternLanguage koan in KoansMetaphorsAndParables

I don't see how language features replace design patterns. Language features can make them easier to use, but design patterns (the good ones) aren't language specific. -- EricHodges

[Are you sure about that, many of those patterns listed above come strait from the GOF, seems like patterns are workarounds for things that can't be further simplified, and that's a language smell. Why would you ever use InterpreterPattern if you have a language that allows you to extend it... you wouldn't, because it's a hack for languages that don't allow that. Personally, after learning a bit about functional programming idioms, it seems many of the GOF patterns are OO hacks to emulate functional idioms.]

You'd still be using the InterpreterPattern, the language would just make it easier. A pattern isn't a set of source code lines. It's a way of doing things. -- EH

If you have double dispatch, why use Visitor?

Why not? Double dispatch doesn't do away with the need to abstract behavior. -- EH

Eh? The Visitor pattern implemented with MultiMethods is indistinguishable from dozens of other mundane multi-dispatched method calls. Yeah, I suppose you could say that the pattern's still there, but it's barely visible against the background - so it's not much use to identify it in that context.

I wouldn't say the GOF patterns are hacks to emulate functional idioms - at least some the patterns are structural. Rather, I think that some are language-specific idioms that have been generalized enough to apply to a class of languages. Visitor is a good example. -- DanMuller

[Yes, that class being OO languages, and Visitor allowing DoubleDispatch, a limited form of MultipleDispatch, I'd call it a hack. If you want to think of having a generic function mapped over a collection as still being the visitor pattern, then I can understand that point of view, but I'd demote it to the visitor idiom, because it isn't a pattern anymore.]

Yes, I agree that it isn't really a pattern at that point. Note that MultiMethods are not a feature specific to the functional paradigm - their defining characteristic is their polymorphic nature, and they could certainly have side-effects. But that's a digression from the topic at hand. -- DanM

Perhaps I'm not making myself clear. If you're applying Alexander's ThickWalls? pattern when building a house and you use a material that makes thick walls easy it doesn't mean you aren't applying that pattern. -- EH

The summary of VisitorPattern (from that page) is: Represent an operation to be performed on the elements of an object structure. Visitor lets you define a new operation without changing the classes of the elements on which it operates. Well, that's how MultiMethods work inherently. VisitorPattern is only significant in languages that restrict polymorphism to methods that are part of the definition of "class".

A better analogy would be trying to apply Alexander's ThickWalls? pattern to the construction of bunkers, which of course have thick walls. -- DanM

Now I think you're not making yourself clear. If you build a bunker and it has to have thick walls then you're not applying the ThickWalls? pattern? I don't follow. -- EH

[If you build the thick wall, then you're applying the pattern... but if the wall was already there, as in multiple dispatch in some languages, using it does not constitute applying that pattern, because you didn't have to build it. I guess I'm saying design patterns are what you call things that haven't become part of the base language. Which is Paul Grahm's point, a sufficiently flexible language has no use for "patterns", anything that looks like a pattern quickly becomes part of the language, thus killing its status as a pattern. Things that were called patterns in assembly, are now just features of more modern languages, design patterns will eventually become the same.]

It doesn't matter if I code the application of a pattern or the language does that for me. The pattern is still being used. I think you're focusing on specific instances of patterns. I'm talking about the pattern itself. Building a language that makes applying patterns easy doesn't make patterns go away. It just makes them easier to use.
- Of course, nobody ever speaks of the "function" pattern, or the "class" pattern, or numerous other things that we take for granted because most languages provide them as built-in features. OTOH, programmers in a purely PrototypeOrientedLanguage? might well find it convenient to simulate classes with prototypes...and programmers in a hypothetical language that provided continuations (or AssemblyLanguage programmers using an architecture with only branch instructions) as the only flow control mechanism (and no macro capability) might well construct code sequences that emulate things like function/procedure call, loops, if/then/else, etc. If one were to study the assembly-language output of a CeeLanguage compiler, without knowing the source of the code, lots of patterns would emerge.
- And what about the AssociationList? What are alists, if not a common Lisp DesignPattern? There isn't any official "alist" library anywhere which performs manipulations on alists; generally most manipulations on alists are sufficiently easy that they are hand-coded in situ, rather than being abstracted in a common place. Of course, modern Lisp dialects now have native associative containers based on HashTables, RedBlackTrees, or other data structures capable of supporting fast key retrieval, so the alist is no longer strictly necessary (though still frequently used). In early Lisp dialects, OTOH, the alist was how associative data structures were implemented. In my mind, the AssociationList does qualify as a design pattern. However, it would be absurd to speak of a dictionary design pattern in Python, or a "map" design pattern in C++, etc.
- I question whether an AssociationList deserves the label 'pattern'. If there's a continuum of concepts from patterns down to trivial idioms, AssociationList's are certainly closer to an idiom. An associative data structure is a higher-level concept that might deserve to be called a (still trivial) pattern, with the AssociationList as a means of implementing it in a particular family of languages. -- DanM

(A more verbose explanation...) Bunkers by definition have thick walls. (A thin-walled bunker would be no use to anyone.) If you restrict the realm of discourse to bunkers rather than buildings in general, then the ThickWalls? pattern disappears. Similarly, if you restrict the realm of discourse from OO languages in general down to OO languages that support MultiMethods, then the VisitorPattern disappears.

Delving more particularly into MultiMethods vs the VisitorPattern: VisitorPattern invents a class to embody the behavior of an operation separately from the multiplicity of classes that the operation applies to. With MultiMethods there are two ways that you can approach this. A GenericFunction can replace the second class, and the implementing methods replace the methods of that class. In the VistiorPattern?, Accept() together with the specific type of the Visitor argument invoke an operation polymorphically on several types of operand objects. With this first MultiMethod?-based approach, the artificiality of Accept() is not needed. MultiMethods trivially embody the VisitorPattern every time you define a MultiMethod? that is polymorphic on a single parameter.

The alternative arrangement, if you need the level of indirection that Accept() can buy you (you don't need to know the type of the operation to invoke Accept(), which is important if you choose to use internal recursion through a hierarchy of objects), you can define an Accept GenericFunction that takes two polymorphic arguments, and specialize the methods on both. Again, this is a trivial case of a MultiMethod? that's polymorphic on two arguments. Although this latter arrangement is more readily recognizable as a VisitorPattern, its most interesting feature is actually the internal recursion, and not the VisitorPattern-like separation of operator and operand (the latter being inherent to MultiMethods). -- DanM

A language can't have everything in it so you will always have missing language features.

That's simply not true. Languages that allow you to add features, Lisp, Scheme, Smalltalk, give you the power to add anything you find lacking, so the only missing features are the ones you're too lazy to implement yourself, because you could if you wanted too. It's just a matter of which ones.

Can you make Lisp or Smalltalk be C without writing an entire interpreter/compiler? Not that anyone would want to, but it makes an interesting test.

You could make a language that had an EssExpression based syntax but the same semantics as C.

You could create some low level extensions that let you do low level things. For example:

  (peek addr)
  (poke addr value)
  (inp port-number)
  (outp port-number value)

In fact, the Lisp Machine's Lisp had operations exactly like this, with pointers and pointer arithmetic. It was necessary for the low level operating system stuff they had running; I don't think any modern Lisp has followed suit here because they don't operate at a low enough level that this is important.

Comments to the contrary at page bottom about modern Lisps pre-date your comment.

[I'm not sure why you'd want to cripple Lisp and Smalltalk like that. I guess you could use macro's to change the syntax of lisp, and leave out most of Lisps features, eventually, it'd be crippled enough to be C.]

You're missing the point. Can you do PointerArithmetic, dubious typecasts, free(ptr), and numerous other nasty things in Smalltalk or Lisp? For most applications of Smalltalk and Lisp, you obviously wouldn't want to, and the absence of these low-level features is not considered a liability. But if you want or need to engage in low-level bit-frobbing - either on memory-mapped registers or on the program or language runtime itself, nothing beats C/C++ for that kind of mud-wallowing. And there are times where mud-wallowing is what you need. :)

[I'm not totally sure, but I imagine I'd wrap assembly or C up into a macro and get whatever feature I wanted, no matter how low level.]

To make it syntactically look like C, yes, you'd need to build a parser. But most Lisps, taking into account common implementation-specific extensions, already have all the functionality of C.

Which extensions? And which are used for the kinds of systems programming crud that C is hated/loved for?

The implementation-specific APIs exist, you can do pointer math, casts, whatever, with them. There's no particular reason that you can't have similar unsafe implementation-specific APIs in most any language, and I've seen them in many otherwise "safe" languages. (Pascal being another good example.) There were entire operating systems written in Lisp, and some of the current Lisp implementations can trace their lineage back to those operating systems, AFAIK. If you're really interested, or really won't take my word for it, you'll need to look at the documentation available at any home site of a Lisp implementation - I'm not conversant with these low-level extensions and won't look them up for you. But I know they exist, having browsed the documentation a couple of times in the past.

Classic SocialProblemsOfLisp attitude. Anyway, see, for instance, CMU CommonLisp SAP (System Area Pointers) and Alien Objects facility, http://common-lisp.net/project/cmucl/doc/cmu-user/aliens.html

Well, I took the previous italicized comment to infer that the author didn't believe what I'd just said. His problem, not mine. Anyway, in the spirit of appearing less socially problematic, I've also spent some time hunting up more evidence, in this case from Franz's Allegro CL 6.2 (which is available in a free non-commercial use version): http://www.franz.com/support/documentation/6.2/doc/ftype.htm. Poking around in the table of contents (http://www.franz.com/support/documentation/6.2/doc/contents.htm) will turn up other things related to low-level programming, operating system interaction, and network communications.

How does SmalltalkLanguage allow you to add features? Anyway, if you add a feature, and everyone has to know how to use it because it's not part of the base language, how is that different from using a pattern? And if you want to add Active Objects to LispLanguage, how would it be any different from doing it in JavaLanguage?

The point is those languages (Lisp, Smalltalk, Scheme) allow a pattern to integrate into the language smoothly so that it looks as if the pattern is part of the language itself. And when a pattern becomes part of the language, it is not as obvious nor of any benefit to call it pattern. Think about "If-then-else" in C++ or Java - is it worth anything to mention the "If-Then-Else" Pattern? Or the "For-Loop" Pattern? We don't think of those constructs as patterns because it is too low-level for the language (C++, Java). However, if you are writing assembly then those If-Then-Else or For-Loop or Polymorphism Pattern become visible because the language's low-level character doesn't provide you with those features. A language that allows you to integrate patterns seamlessly into it allows you to think at a higher level than before.

Lisp systems programming facilities typically support calling external functions (e.g. C functions, Unix or Windows systems calls) and use of C data type declaration and operator semantics. By stealing those C semantics outright, Lisps with this sort of extension can fairly claim to be able to do whatever C can do, although at the price of switching to the CeeLanguage approach to programming - a Procrustrean bed indeed.

CMU CommonLisp SAP (System Area Pointers) and Alien Objects facility, http://common-lisp.net/project/cmucl/doc/cmu-user/aliens.html

Franz Lisp systems programming

38 Foreign Function Interface (foreign-functions.htm) http://www.franz.com/support/documentation/6.2/doc/foreign-functions.htm
40 Foreign Types (ftype.htm) http://www.franz.com/support/documentation/6.2/doc/ftype.htm
59 OLE Interface (ole.htm) http://www.franz.com/support/documentation/6.2/doc/ole.htm
60 Operating System Interface (os-interface.htm) http://www.franz.com/support/documentation/6.2/doc/os-interface.htm
66 Remote Procedure Call support in Allegro CL (rpc.htm) http://www.franz.com/support/documentation/6.2/doc/rpc.htm
68 The shell module interface (shell-module.htm) http://www.franz.com/support/documentation/6.2/doc/shell-module.htm
69 Allegro CL Socket Library (socket.htm) http://www.franz.com/support/documentation/6.2/doc/socket.htm
75 Unix Lisp based shared libraries (unix-shared-library.htm) http://www.franz.com/support/documentation/6.2/doc/unix-shared-library.htm

Corman Lisp (Windows):

http://www.cormanlisp.com/features.html
http://www.cormanlisp.com/CormanLisp/CormanLisp_2_5.pdf
Foreign Function Interface. All functions in DLLs may be called directly from Corman Lisp code, including all Win32 API functions. Most Win32 datatypes and many Win32 API functions come predefined and ready to be used in the WIN32 package. A special C declaration parser is built-in which allows Win32 C API functions to be used with little or no modifications (obtained from Microsoft C header files).
- Emphasis added because that sounds pretty useful.
Foreign Callbacks into Lisp. The Foreign Function Interface supports definition of callback functions which can be used as Windows procedures or for other uses. This allows straight Win32 SDK applications to be built, exploiting the whole power of the operating system. These callbacks can be interactively edited and replaced, even while being used by the operating system or another program

This stuff should be moved to some Lisp-specific page at this point.

The converse is also true you can call Lisp functions packaged as a COM, SOAP or similar object and called from C or other languages, using Lisp (or Ruby, or Smalltalk, or ...) sparingly as needed.

Has anyone ever considered that design patterns are the way that programming languages evolve? In the same way as communicative language, commonly used contractions ( or patterns ) may become standard. In english words like "won't" and "isn't" are contractions, but are more or less considered standard words today. In the same way, 'if-then-else' or 'do-while' could be considered design patterns that have now become standard features in many languages. Perhaps later languages will include many design patterns as standard features... =OdD=

Makes sense, but why does Lisp (a very early language) have so many design patterns as standard features? Are all programming languages evolving toward something like Lisp?

Yes, they are, because lisp recognized early on that a language "must" be extensible. Languages that can't change, die. Programming is language design, and a programs constructs should stand equal with the existing language constructs.

Absolutely. The evidence is ever present. Perl, Python, Ruby all have constructs that are present in CommonLisp. Even the CsharpLanguage has a hint of Lisp-like features. Microsoft is spending time in the area of DomainSpecificLanguages?, which one of strengths of CommonLisp: changing the language to make it work for you. --RalphAllanRice

Sounds like a corollary to GreenspunsTenthRuleOfProgramming: Given enough time, any programming language allowed to change approaches Lisp.

C++ was developed for C-Users, despite there being other OO languages about. Whether other languages are better or not, C-users would likely prefer C++ as it is similar to C. Other, earlier languages may be better, but if you are English it is easier to learn American than Arabic, even if Arabic is the better language. Lisp is probably just a better language that less people use. Either that or there are flaws in Lisp you cannot see.

Actually, a better question is why Lisp has so many GangOfFour design patterns as standard features. The answer, as seen above, is probably due to GreenspunsTenthRuleOfProgramming: OO languages need them to simulate Lisp features. Conversely, Lisp needs its own, different design patterns to simulate OO features. Or at least it did, pre-CLOS.

[Yes, Lisp started out very primitive, and modern Lisps and Schemes embody a lot of stuff that was added based on many years of experience and based on compromises between a variety of problem solution implementations. And then the cycle starts over again with the next language...]

Re: Paul Graham on Patterns

Sthere is a brascket pasttern isn lissp...

  (()()()()())
  (()()()()())
  (()()()()())
  (()()()()())

Tis pasttern seems to bee usterly repestistive.

Psssssp!

Comspare to:

Indeed! Are patterns a LanguageSmell?

See DesignPatternsAreMissingLanguageFeatures, MissingFeatureSmell, LanguageSmell

CategoryPattern