Lisp Is Too Powerful

I think one of the problems with Lisp is that it is too powerful. It has so much meta ability that it allows people to invent their own little worlds, and it takes a while to figure out each person's little world (SoftwareGivesUsGodLikePowers).

Languages are more than just languages, they are a form of culture, and by being culture they tend to enforce (indirecty or directly) a certain way of doing things, i.e. standards or conventions. This means that if you know the language and its culture, there are less surprises and a longer learning or adaptation curve. Lisp is almost an interpreter builder rather than just an interpreter/compiler. This means that one can make it be just about anything they want. Thus, its culture is almost a lack of culture.

Now if the benefits of this power outweighed the drawbacks, I think it would be worth it in a commercial setting. But I have yet to see a good demonstration that Lisp significantly simplifies things beyond what other languages can do (ChallengeSixVersusFpDiscussion). It only slightly simplifies things. The cost of more surprises due to lack of "culture" is thus greater than the ability to make custom idioms that simplify things to a fairly minor extent over what "regular" languages can.

This is why Lisp is a HackerLanguage instead of a commercial language: hackers are generally loners who don't care if others can figure out their code (at least while they are in the mode or role of hacking). Thus, they build their own little world in it that fits themselves nicely so that they can hack fast, but the rest of the world be damned.

Further, to acheive and take advantage of this heavy meta-ability, Lisp does not fit the style of other languages. Thus, it takes longer for most to get "fast" in Lisp.

Sometimes consistency trumps power.

(I may have said this somewhere else already, and I apologize for duplicating if I did. Some of it sounds a bit familiar.)

--top

Above you talk about lisp interpreters but most of the lisps used in the "real world" are compiled. Also there is a lot of shared lisp culture, especially in macros small examples are naming macros that define "things" defthing, macros that lock and unlock, or open and close a resource all follow a similiar style, and lots of macros that are syntatically and semantically similar to defclass. If there is a lack of standardisation in lisp programs it is because there is lack of standardisation in what the programs do.

TomA

I adjusted the "compiler" issue. As far as "standardization", it is not so much language conventions as it is the ability of the language to bend to be almost anything you want. Even with conventions it is more bendable than "regular" languages.

Suppose I created a very ExBase-like sub-language using Lisp. It would closely resemble ExBase, but with more parenthesis. If other developers were forced to read and modify such, they'd say, "What kind of screwy commands are these? This is a weird language with weird conventions! What are you, some kind of table lover? Dude, the 80's are dead." Sure, one could probably do that in many languages, but in Lisp its easier.

I think the problem of lisp is less that LispIsTooPowerful but rather that the power of simplicity is taken a bit too far (GoldenHammer-like). Lisp lacks a clear distinction between its powerful facilities. Maybe the left out syntax facilities could have provided this. ColaLanguage? does this better. The way Lisp now is is that it is TooPowerfulForItsOwnGood.

Are there any real life examples of Lisp developers creating DSL's that can't be deciphered by their colleagues? This argument sounds a lot like speculation to me. -- LarryColeman?
"Can't" is relative in this biz. Given enough time it usually can; but does one want to take that time? And PaulGraham's Y-store lisp project was allegedly rewritten into another language by Yahoo because existing programmers had difficulty figuring out the lisp code. -t
- Actually, "can't" is correct in this case. The way that closures were used in the original Yahoo! Store is just not supported in languages like C++ without essentially implementing a Lisp interpreter. Here's what Paul Graham said about the rewrite:
  - I was only addressing the "can't decipher" issue, not the closure issue. There indeed may be multiple issues involved, but based on the ordering, human groking of the code was an important stated reason for the language change. I'm not sure if the closure issue had to do with co-integration of Lisp and C++, or was only a statement about emulation, but it probably doesn't matter to the key issue. -t

(a) The reason they rewrote it was entirely that the current engineers didn't understand Lisp and were too afraid to learn it.

(b) The resulting program is a new world's record case of Greenspun's Tenth Rule. The Yahoo Store Editor called compile at runtime on s-expressions made on the fly. To translate this into C++ they literally had to write a Lisp interpreter.

Who's going to formally admit to being confused by it? They just avoid it the future rather than document headaches. They vote with their feet. You have a right to ask for such evidence, but it probably won't exist.

But that assumes there's at least one actual person who is confused. On the other hand, you can find people who formally admit to being confused by Lisp almost daily on comp.lang.lisp.
And, you can find even more people who admit to being formally confused on comp.lang.c, by sheer virtue of C having more coders than Lisp. This doesn't prove anything.'
That nothing is being proved is my point. We just have an assertion here of confused colleagues, and I haven't seen a verified example of a Lisp coder confused by code written by a colleague. There was one on comp.lang.lisp who claimed to be confused by inherited code, but that post could have been made up by the resident spammer.
[I developed in Scheme for quite a while, and found myself plenty confused by various macro packages. OTOH, it takes a while to get a feel for any API, be it library, DSL, or framework.]
(It may be an issue of weighing MindOverhaulEconomics.)
There is now a lot of startups using Lisp to build commercial DSLs. I work at one and we can all figure out each other's code
Being useful for start-ups may not scale to general organizations, remniscent of Paul Grahm's Yahoo Stores project. I do in general find Lisp harder to read than the equivalent in AlgolFamily languages, and others have made similar observations (see ChallengeSixLispVersionDiscussion). Lisp may limit an organization's hiring base to those who can read Lisp fast. But maybe the pattern will somehow be broken and IfFooIsSoGreatHowComeYouAreNotRich will finally be "solved". The DSL pattern may merely be an industry-specific "habit" of a kind, meaning they each "do it" merely because competitors "do it" and they may want to poach some of their employees or avoid retraining time.

Not a Language?

Perhaps Lisp is not really a language, but a meta-language with libraries and conventions that almost make it look like a language. It's a nested-list machine language.

[If it's a nested-list machine language, how is it not a language? I feel like I'm reading a direct contradiction...]

Perhaps "not an application language" would be more appropriate. CPU machine language and BackusNaurForm are examples of languages that are not application languages. They are generally used as building blocks or assistance tools for application languages.

Let me put it another way. In a "normal" language when one encounters a new application to maintain they typically are not starting from scratch, but rather at least know the app language. You were likely hired because of knowledge in a particular language (for good or bad). Thus, you can then focus primarily on specifics of the application at hand and don't have to worry much about learning the new application language it is written in (although different shops and programmers may use a different subset). This cuts down on what you have to learn.

However, with a meta-language like Lisp, a language that allows you to "make up" a new language (or semi-language), you then have the possibility of BOTH chores for each new app encountered: learning the app AND the new "language" created via the meta system.

Given enough power, each programmer will eventually shape the language to fit their mind. Unless you think like them, you may have a strange new world to learn. As stated in another topic, the power of standards and conventions is as much in what they don't let you do as they are about what they do let you do. Standards and conventions are about walls and gates. These walls and gates hopefully create consistency in organization and familiarity for faster groking and communication. They are essentially a social tool. Lisp has fewer linguistical walls and gates.

For a stretched analogy, if you stick with Legos or Tinkertoys, then anybody who is familiar with Legos or Tinkertoys can quickly dig into contraptions built with them and make necessary changes. They've learned various Lego and Tinkertoy idioms over the years and are quick and handy with them.

(Tinkertoy computer: http://www.retrothing.com/2006/12/the_tinkertoy_c.html )

But along comes a new building kit called Lispo's which allows one to build different building atoms (blocks, sticks, connectors, etc.) with all kinds of odd shapes and connections. Every Lispo builder makes very different "atoms". Or at least enough of them go "wayward" to create a fair amount of bottleneck projects.

Lispo-built contraptions are often compact and efficient, but they can be very difficult for new maintainers to grok, including other fellow Lispo-ers. Further, Lispo tends to attract those who like to customize atoms such that the probability of encountering a contraption that is hard to grok is noticeably greater in the Lispo community. It attracts more "lone hackers" because it has this meta ability.

In the end, you will admire Lispo and Lispo-ers and maybe even build a few of your own as a hobby, but want nothing to do with others' Lispo projects.

--top

With great power comes great responsiblity. This is why some must avoid it. --Bottom

I've had to stop coding in Lisp, because I'm not wise, intelligent, or compassionate enough to wield such a potent weapon. No human could be. Lisp is reserved for Gods.

Gods without team members and turn-over perhaps.
You work for places that hire brain-dead code monkeys. Sad, that. I always hired good programmers who liked learning new tools, exploring Lisp, and discovering new concepts.

The notion that LISP lets you create "semi-languages" that add another layer of unreadability, or that this is frequently done, is rather exaggerated. Even when it is explicitly done, it raises no more barrier to understanding LISP code than, say, procedures, functions, methods, etc., in any other language. Or, any more barrier to understanding than any DomainSpecificLanguage, whose clarity (or lack thereof) is a factor of the language rather than LISP. Arguably, ForthLanguage is more inclined to host unreadable "semi-languages" than LISP, especially in the hands of an inexperienced Forther. For most uses, LISP is employed as a fairly conventional language, with HigherOrderFunctions being perhaps the biggest conceptual hurdle for those raised on Algol-derived languages. You get used to prefix notation extremely quickly, but only if you use the language. Merely reading it is not enough.

Much of the objection to LISP comes, I suspect, from people who have only (tried to) read it or who have been forced to use it by work or school. People are rarely positive about things they've been forced to use.

HOF are a tool to allow one to build arbitrary loop and conditional statements/operations, for example, creating less standardization and more Picasso-code. Perhaps it encourages Lisp fans to use HOF and other FP constructs just for the hell of it, such as the "cool factor", not because it's an objectively better way to do it. I've asked for some good practical examples of heavy FP helping in my domain, such as ChallengeSixVersusFpDiscussion, but have not seen them, only what sound like excuses. FP may be "fun", but is it helpful? Lisp may encourage programmers to focus on gee-whiz instead of maintenance.

I don't recall seeing HOFs used to build arbitrary loop and conditional statements very often. They're typically used where the alternative would be considerably more complex code -- like having to create a family of static functions or awkward conditional statements, or expose the internal workings of some existing mechanism -- where a single HOF would do.

Imagine a query language that supports summarisation (i.e., GROUP BY in SQL) with the usual aggregate operators like AVG, SUM, STD, VAR, etc. Imagine you need a new one-off aggregate operator, such as a special weighted average whose composition varies with each row. With HOFs (presuming the query language supports passing arbitrary aggregate operators to the summarization operator), it can be trivially added without any other syntax extensions.

HOFs enhance readability by reducing the need for complicated, confusing code.

Can you find an example of the practical use of HOFs where the alternative would be simpler and more readable?

No. I never claimed so. They are generally both competitive with each other in terms of expressiveness. They each may do certain things a little bit better, and thus I'm not against mixing in some FP as needed. But, the key is "some". Algol-style languages do about 3/4 of everything just fine, and is the standing default. Sprinkle in a little FP and OOP where they have a clear advantage for that usage spot. (I'll leave TOP out of the discussion for now.) But FP fanboys say going full-bore FP will greatly improve things, which has not been demonstrated with realistic scenarios. I think they use C as a reference point, and C is a poor language from a human usage perspective. It lacks many features that improve imperative programming.

Algol-style is the current default "base" for most of the app because 1) it's the easiest to learn for most people, 2) has most of the existing code-base, 3) Has not been proven clearly worse as the base default.

I believe LISP's potential power is in it's ability to create DomainSpecificLanguages, not FP and HOF's. I'm skeptical that FP offers offers significant expressive improvement in my domain, custom biz apps, and past scenarios given on this wiki failed to produce significant expressive improvement. -t

Your belief about LISP's potential power notwithstanding, its actual power does derive from FunctionalProgramming and HOFs. Being able to create domain-specific languages is a bonus. Whether or not LISP offers significant expressive improvement when developing "custom biz apps", whatever those are, is something for you to determine.

{It is unclear to which alternative TopMind is comparing HOFs. Many other languages support something similar (FunctorObjects, script + data passing, function pointer + data passing, stack-scoped functions). The main advantages of HOFs above these other mechanisms are anonymity (ability to create a new instance locally without overhead of naming a function) and lexical scoping (reduces need to explicitly gather data), both of which reduce BoilerPlateCode. Every Algol-style language 'reinvents' HOFs with at least one alternative, but rarely achieves the full convenience of HOFs.}

Contentious Discussion Below:

I invite somebody who has at least 5 years of custom biz experience, including maintenance, and is also a Lisp fan to provide a clear demonstration of its improvement.

I have 25 years "custom biz" experience, including maintenance, assuming "custom biz" means developing a variety of bespoke applications -- including employee scheduling, inventory, bookkeeping, and medical records systems, plus development tools from compilers to GUI toolkits to DBMSes -- for use by businesses. I am a Lisp fan and a FunctionalProgramming fan. Internally, my RelProject has a largely functional core. However, I have no desire to expend the energy required to provide you with a clear demonstration of its improvement. What purpose would that serve? I don't need to convince you of anything, and I've little doubt it would turn into a lengthy, pointless, and frustrating debate with you that would end in mutual insults. I have much, much more productive and worthwhile things to do.

Therefore, I invite you to learn Lisp, try it with an open mind, and thereby respond to your own challenge in a far more definitive and convincing manner than anyone could do for you.

If you want to evangelize Lisp into the mainstream, then don't just focus on me. Custom biz apps is one of the largest niches to push it into if you want to expand. "Trust me and try it for 5 years" is NOT a good way to convince others. Otherwise, if you don't want to spend the effort to document betterment, just say so and move on. Rel is more about "purity and protection" than expressiveness, so it's likely I wouldn't be very receptive of samples done in your style anyhow. Further, FP may be of more value in SystemsProgramming? than biz apps because the engineer has more control over the much of domain, and thus gear it around clean, logical interfaces, idioms, and conventions. In biz apps, one is often modeling politics and manager & marketer personalities, which are usually not very logical. You often must bust encapsulation because everything must be able to spill into everything else. -t

It doesn't matter to me whether Lisp becomes mainstream or not. In particular, it doesn't matter to me whether you like Lisp or not.

By the way, I didn't say "trust me and try it for five years." I asked you to try Lisp for yourself, for however long or short a time you like. Then you don't have to trust anyone except you.

My "try" list already has plenty of stuff in it.

You'd get through that list faster if you'd spend more time trying things on it and less time complaining here about languages and tools other people are using.

Where's the competitive testosterone-driven spirit of Lisp evangelists? If it's more expressive, then show it expressing. "Trust me, use/try it for 5 years" is for pussies.

If you're looking for "the competitive testosterone-driven spirit of Lisp evangelists", you'll probably find it on a Lisp advocacy forum. I like Lisp, but I have no interest in Lisp evangelism or bothering to "show it expressing". I'd rather use it to accomplish useful things for my own purposes. By the way, what part of "I didn't say 'trust me and try it for five years'" wasn't clear to you? If you're having comprehension difficulties, I recommend taking a remedial reading course. Or are you just trolling?

When you "invite" somebody to try/use something in order to "get it" like you allegedly do, the implication is a kind of "trust me" in most cases. It's an implied ArgumentFromAuthority.

'Its not an argument at all. But don't let that stop you. Oh well, actually, yes please let it stop you..'

-- Why am I still here?

That's your grumpy interpretation. It's certainly not my intent, and it doesn't explain where you got "five years". One's impression of Lisp, like so many computer things -- plus skiing, skydiving, fast cars, motorcycles, hiking, gourmet food and sex -- gains nothing from explanation and everything from experience.

Your examples are for entertainment, not solving external problems. (Some Lisp benefits may be WetWare-specific, but those are not the ones that are of interest here.)

You're in the wrong line of work if you don't find programming entertaining. My examples are for illustration of the subjective nature of appreciation, which applies as much to Lisp (and many other things) as it does to (say) skydiving and sex.

If you merely want a mental challenge, then try BrainFsck.

You're obviously just trolling.
{He's been trolling from the marked start of this section. And you totally fell for it...}
Well, yeah. So did you, below. :-)
{That's a logical trap, intended to eventually point out TopMind's DoubleStandards, should he bother pursuing it.}
It seems to have worked well. I'm increasingly of the opinion that arguing with Top is like mud-wrestling with a pig: Even if you wind up on top (pun intended), you have to get desperately filthy and the pig enjoys it. (MudWrestlingWithPigs)

{Huh? I've seen no objective evidence that BrainFsck is more challenging for business applications and systems software programming than is Lisp. I invite you to provide clear evidence that it is more challenging.}

Sure. Fine. One pf.-by-example, coming right up...

Lisp example:

    (def func (lambda (i foo)
        (* i foo)
    ))
    (display (func 12 10))
    (display (func 13 9 ))

BrainFsck example: ++++++++++>++++++++++++<[>[>>+>+<<<-]>>>[-<<<+>>>]<]>> . will that work? I haven't a clue [-]<[-]<[-] ++++++++++>++++++++++++<[>[>>+>+<<<-]>>>[-<<<+>>>]<]>> .

Now, let's make a simple change:

    (def func (lambda (i foo)
        (+ i foo) ; the change is right here!
    ))
    (display (func 12 10))
    (display (func 13 9 ))

characters changed (minus the comment): 1.

++++++++++>++++++++++++<[->>+<<]>[->+<]> . [-]<[-]<[-] ++++++++++>++++++++++++<[->>+<<]>[->+<]> . Characters changed (minus the comment): 58.

Conclusion: by at least one one anecdote, BrainFsck is 58 times less maintainable than lisp. Secondary conclusion: This is because BrainFsck has no modules, and therefore every useful procedure must be coded multiple times.

Add more tests, people!

It's a start, but most will agree that BF is difficult to use because of more than just lack of functions. BF with functions would probably look something like:

 ++(+<[(->>+)<<](>[->+<]>++(+)++<[->>+<<]>[-(>)+<)]>+<[-(>>+)<]>[-(>)...

Disclaimer: No attempt has been made to have this example make sense. It's only a rough visual illustration. -t

See also: GreatLispWar, TooPowerfulForItsOwnGood, StandardToolDependency?/StandardToolDependancy (title changes due to an EditWar with GV)

CategoryLisp

JuneTen