Challenge Six Lisp Version Discussion

Continued from ChallengeSixVersusFpDiscussion:

For what it's worth, I was upset by the on-all-sides-argumentative and fruitless conversation on this page, and decided to do something about it. http://github.com/ods94065/challenge-six is my tilt at the windmill. The solution uses a grab-bag of Lisp tricks, including some FP, to shave off somewhere between 9-30% of source code size depending on how you measure it. Not bad for a simple demo app IMO. -- Owen [copied]

Interesting. Very interesting. I'm not experienced with lisp, so I cannot yet comment on many aspects. But a few things stand out in early inspection.

The first is that the lisp version is partly more compact because it relies on positional parameters to reduce the need for key-words. However, I and others have agreed that often positional parameters slow down reading of code. One's head has to "count" parameters and track parameter position to know the meaning of the position. (I couldn't find the commentary I was looking for. WorkBackwardFromPseudoCode discusses the trade-off between positional and named parameters to some extent.)

Fortunately, Lisp cheerfully supports keywords (every symbol that starts with a colon) and keyword arguments to functions in addition to positional parameters, so this is a stylistic choice you are perfectly free to make if you want more readability in your code. You will note I take advantage of keywords when putting together the more complicated tag macros like padded-title and picklist1, for just that reason you state. Because of the HTML library I'm using to define those tag macros, they're signaled with &attribute instead of the usual &key. -- Owen

I used to use an inline-style utility function like this:

  x = iff(condition, a, b)
To replace:
  if condition then
     x = a
  else
     x = b
  end if

But I decided it was harder to read (for my eyes at least) and abandoned its use despite reducing code size. (It can also be less efficient machine-wise in VBS, but that's another issue.) The latter is more visual and the spacing rules of VBS tend to enforce consistent spacing, and consistency is important to readability, especially for team programming. Lisp allows all kinds of different line-spacing or lack of because it's line-space-blind for the most part. Sure, you can create shop standards, but it's just yet another "chore" to remember (and get wrong). And diff shops will do it differently and turn-over means there's always learning curves in action.

I'll cover the questions of spacing and consistency below. There are two important considerations for how Lisp's if compares to your iff: 1) it's short-circuiting, so it follows the convention of most languages; 2) it's absolutely standard, so there's no trouble in fostering consistent adoption. The fact that this out-of-the-box if statement has value-returning semantics, which I would argue is the real advantage of your iff function, is more than just a nice touch, it shows that a functional way of doing things ("everything returns a value") is deep in Lisp's DNA. -- Owen

The following examples are taken from the source samples and are roughly equivalent, allowing us to compare the styles:

 ......select case fmtType
 ......case "T"
 .........if isBlank(rs("comparer")) then
 ...........useValue = "'%" & useValue & "%'" ' for LIKE
 .........else
 ...........useValue = "'" & useValue & "'" ' quote wrap
 .........end if
 ......case "D"
 .........useValue = DBconst_dateDelim & useValue & DBconst_dateDelim
 ......case "N"
 .........' leave as is
 ......case "Y"
 .........select case lcase(fldValue)
 .........case "(either)", ""
 ...........useValue = space(0)
 .........case "yes"
 ...........useValue = DBconst_true
 .........case "no"
 ...........useValue = 0
 .........end select
 ......case "L"
 .........if lcase(useValue) = "(any)" then
 ...........useValue = space(0)
 .........else
 ...........useValue = "'" & useValue & "'"
 .........end if
 ......end select

.(case fmt ....(#\T (if (is-blank comparer) ........(wrap "'" (wrap "%" field-value)) ; for LIKE ........(wrap "'" field-value))) ; quote wrap ....(#\D (format nil "datetime(~a)" field-value)) ....(#\N field-value) ; leave as-is ....(#\Y .....(let ((yes-no-val (string-downcase field-value))) .......(cond ((string= yes-no-val "yes") "1") .............((string= yes-no-val "no") "0") .............(t "")))) ....(#\L .....(if (string= (string-downcase field-value) "(any)") ......."" ........(wrap "'" field-value)))))

(Dots to prevent TabMunging. If I messed up the formatting, I apologize. Note that the letters probably should be commented, such as "N" for "Number". I left them off because they were documented in the data definition comments, which may be a poor excuse. I have an issue with the lisp implementation of the "Y" grouping, but it's minor.)

The first is just quicker to read (for my eyes). Yes, it's more verbose and repetitive, but there is a certain visual consistency to it that reduces the mental work to make sense of the code (LispLacksVisualCues). It's closer to how one would present it on paper in a nested outline form if one were documenting the algorithm in a more presentable way. The lisp one is kind of "bunched up" and requires the "position counting" described above.

Fundamental syntax in Lisp, along with spacing, is probably the hardest for newcomers to come to turns with, and to be sure some programmers never do. However, Lisp programmers do aim for a certain amount of consistency in spacing when writing their code, even if the language doesn't strictly force it--just as C programmers do. Fortunately, I haven't seen debates on Lisp style get as pedantic and silly as the WhereDoTheBracesGo and TabsVersusSpaces debates I've been inundated with over the years in the C community: we've all got better things to do than quibble! I have striven to emulate the style that I've been taught in Lisp textbooks, as well as my own sense of what I think an elegant line of code looks like (informed in this case in part by Lisp's particular brand of terseness), but I understand it takes some getting used to if you haven't worked in Lisp before. Feel free to play with the spacing and style to see what suits you; it shouldn't generally affect the word count.

I think part of your reaction here also stems from the fact that Lisp's most common control structures--particularly if, cond, case, and do--are terse in form, quite apart from the spacing. Do (not covered in this example) is so terse that many Common Lisp programmers now prefer to use loop instead, which attempts to put readability first. (To that point, I prefer dolist and mapcar/mapcan myself whenever I can get away with it, which I frequently can!) It is part of the learning curve of the language, but I think it's a bugaboo. I'm a somewhat experienced Lisp programmer (not an expert), and I can tell you it is no trouble for me to read if, cond, or case statements when written in this form. -- Owen

Thus, I will generally agree your lisp style would be less code, but at the expense of readability, largely due to heavy reliance on positional parameters, and line spacing consistency reduction. Remember, code is mostly for humans and human readers, who may not be the original author(s).

I'm not sure I classify the size reduction as being due mostly to FP. It's more due to the syntax approach that shrinks it. I will agree that FP makes it easier to roll-your-own block/loop constructions to better fit specific situations, but we could probably expand the set of standard ones to do almost the same. But both at the risk of loss of consistency.

--top

I think it's both, as http://github.com/ods94065/challenge-six/blob/master/README.md notes. I think your critique of consistency in adoption is an important one, aimed at the abstractions. Those are things you must think about in any language when you introduce an abstraction: does it in fact lead to clearer code? is it sufficiently flexible? is it easy to use? can it be used consistently? is it documented clearly so others can learn it? It's particularly important if you create an EmbeddedDomainSpecificLanguage like the HTML mini-language I've used in this example. Lisp gives you power to create abstractions at levels (both high and low) which exceed many other languages, which does indeed mean it gives you a lot of rope to hang your coworkers with! With great power comes great responsibility. The questions I would leave you with: if you can put just the right abstractions at your disposal, which dovetail perfectly with the problems you are trying to solve, will you and your coworkers write better code with less effort? If so, why wouldn't you use them consistently? Also: are the abstractions I've written and used heading in that direction? -- Owen

It sounds like it has the trade-offs of a HighDisciplineMethodology. Most orgs are geared to focus on their domain (industry specialty), and not trying to staff a HighDisciplineMethodology IT shop because the owners and/or "top brass" don't know IT enough to consider such properly. They can't tell the flim-flam artists from the real thing. Related: EconomicsOfAdvancedProgramming


Readability is subjective and always heavily influenced by familiarity with the language in question, but I find the second code example more readable than the first. The first says "case case case case case case" in my mind, whereas the second clearly calls out the case strings.

Yes, readability is subjective. As far as language familiarity, I used the "iff" approach (above) for a decent amount of time, but it's readability just plateaued despite time. Having each conditional branch on a different line with clear and consistent key-words surrounding each without the clutter of parenthesis or other symbols is just plain helpful to my WetWare. It allows the visual/spacial processing parts of the mind to assist and not JUST rely on symbol look-up. Thus, there's kind of a parallel processing taking place: visual/spacial AND symbol units. Lisp overloads the symbol processing unit while the visual/spacial "lobe" sits mostly idle.

I know what you mean, but it's an issue that passes quickly if you use LISP for a little while.

Then why didn't the "iff" experiment work?

You mean you used LISP? Or an inline conditional expression in some other language?

Yes, "iff" is a self-rolled in-line conditional made with a regular old-fashioned function, declared by me. Draft implementation:

  func iff(byVal condition, byVal trueVal, byVal falseVal)
     if condition then
        return trueVal
     else
        return falseVal   'Note that older dialects don't use Return
     end if
  end func

(C-style languages often have one built-in with better logic "short-circuiting". [added])

Yes, that's typically used as an inline conditional. Perhaps the experiment failed because it's a relatively rarely-used bit of functional-style coding in an otherwise purely imperative environment. If you used LISP more often, you'd almost certainly get used to employing functional style all the time.

If it plateaued, then time is probably not the problem. Further, the "outline style" conditionals stand out visually. Even if the words are blurred, one can recognize the "look" of a conditional (If standard indentation is used). With Lisp, the forest-level look of conditionals is nearly identical to that of non-conditionals, requiring mental symbol look-up before it's identified as a conditional. With the visual outline form, one quickly knows it's a conditional and then can focus on the details, or skip it altogether if its not what we are looking for. In fact, one is using their peripheral vision to assess the general forest-level structure of the code WHILE also studying individual symbols at the center of the eye's focus. Lisp is less tied to indentation: it's more arbitrary. Again, Lisp bottlenecks up at the symbol-lookup portion of the brain and processing one symbol look-up blocks another because only the center of the eye has enough resolution to identify symbols (for most individuals). Maybe that portion of your brain is more powerful than mine and thus you can tax it more without it becoming the bottleneck. Keep in mind that mental symbol look-up is a necessary skill for just about ALL code styles such that non-Lisp languages are not letting it atrophy, as some may claim. -t

Have you tried using LISP for any length of time? You really do get used to the syntax. Your brain will do a little *pop* one day and suddenly it's all perfectly readable, and you wonder why you ever thought it was less readable than conventional Algol-derived languages.

And if it doesn't click can I sue you? :-) If it takes a while, it may be a case of QwertySyndrome: orgs don't use Lisp because it takes too long to get used to and people don't get used to it because orgs don't use Lisp. But people may start inventing their own conditional handlers, loopers, controllers, etc. and standards go out the window.

They might, but shop coding standards can address that. Give LISP a try on a pet project and let us know what you think after a month or two of serious use.

Shop standards possibly cannot make up for the ramp up time. The average developer turnover period is roughly 3 years. If it takes them one year before the average developer becomes more productive in Lisp than an Algol-style language, then the productivity for the last two years would have to compensate for the first ramp-up year. Plus, they'd need mentoring, taking time away from the more experienced, and most productive, Lisp developers. This would make it at least a wash, if not a net loss, it seems. Further, the developer will be concerned that their skills are not career-enhancing once they leave the current Lisp shop if Lisp is not in sufficient demand outside the shop, and/or limit their location options. Thus, the company would have to pay more for a Lisper than an "Algoler" to compensate for future option losses. Shop-only scope doesn't cut it. Real-world road-testing appears to bear this out: IfFooIsSoGreatHowComeYouAreNotRich.

As far as a personal project, I have enough unfinished hobby projects on may plate right now and don't wish to take on more any time soon. I'm skeptical a month is long enough. Besides, using and mastering my own style/spin on Lisp is not a good team test.

A week is normally plenty, even a day or two of concentrated coding would probably be enough. A month would put paid to it, so to speak. A good developer can be highly productive in LISP in a few weeks, because the syntax is so simple. Once that and the basic concept are grasped, it's just functions. Using LISP for a while would give you a practical, RaceTheDamnedCar perspective, so you could speak from experience rather than speculation.

Simple syntax is not necessarily the same as quick reading. Otherwise BrainFsck would be the cat's meow.

True, but LISP is quick to read once you've used it for a while, just like every other programming language. Even BrainFuck is probably easy to read by those who use it regularly. If there is anyone who does that, of course. The language that is consistently and irrecoverably hard to read is WhitespaceLanguage.

{Nope, you just need an editor that doesn't use whitespace to display whitespace. Perl on the other hand.... :).}

A "GoTo" guru once told me, "Goto's aren't evil, you just have to get experience with them." If frogs ever evolve into programmers, I bet they'd be right at home with Goto's. -t

Anyhow, the industry keeps voting that the first case list is more readable than the second (and/or more consistent per person), and that is the maintenance audience I target. You want to evangelize the parenthesis approach and change their opinion, be my guest. If you can pull it off, THEN I'll change my style to fit the new maintenance audience transformed by you. Esperanto may be more compact or clean than English, but my audience mostly speaks English such that it's more rational to target them. -t

I'm not evangelising LISP, just pointing out that you might understand it better if you use it for a while. That's true of the industry, too.

(Continued at PopularityOfLisp)


More on Case-Block Syntax

By the way, VB/VBS is not the "ideal" syntax in my opinion. There is room for trimming. Something like this could be considered:

  case <expression>  
    :'A'
       handleA()
    :'B','C','D'
       handleBCD()
    :'E'
       handleE()
    :Otherwise
       myDefault()
  end case
At first I had the colon on the right side, but long set lists pushed it out of immediate view.

And a value-centric version could avoid the repeated "useValue =" assignments in the sample, and then code "volume" is quite competitive with Lisp.

  case <expression> to useValue
    :'A'
       handleA() // results to go variable "useValue" if match
    :'B','C','D'
       handleBCD() // etc...
The "to <variable>" clause is optional. Without it, it's like the common version.

Why not simply this?

     useValue = case <expression> ... etc... end case

Leaving out the assignment means it's the statement version; with it, it's the expression version. This approach is used in SQL and TutorialDee.

What's your rationale for putting the colon on the left side rather than the right?

If the set match lists are long, one tends not to see the colon. It's partly because they won't vertically align for longer or variable match lists. Being on the left almost guarantees they'll line up, making it easy to spot the values. It does look "unnatural" on left, but that's just because it's different from typical colon usage. I can "live with" it on the right, with a slight loss in readability.

  case <expression>  
    'A':
       handleA()
    'B','C','D':
       handleBCD()
    'E':
       handleE()
    Otherwise:
       myDefault()
  end case
It's fairly similar to Pascal's CASE syntax.

In practice, existing VB editors give a different color to the word "case" versus the value or variable such that the word "case" is not really the visual deterrent it would otherwise be. Thus, there's little practical pressure to change it. The days where people read code printed from black-and-white printers are gradually shrinking.

Hmmm, maybe something like this?

  case <expression>  
    # 'A'
       handleA()
    # 'B','C','D'
       handleBCD()
    # 'E'
       handleE()
    Otherwise
       myDefault()
  end case


Ideally, if our code editors could transform our view of a code structure into the way we each personally prefer to see it, then the "original" syntax may not matter: you want it outline-like, Zam! it's outline format. Want it compact? Zam! It's compact. We could custom fit the view to our own WetWare and end these never-ending syntax debates. Similar thoughts can be found in CodeAvoidance. -t


Kudos to the WikiZen who actually made the effort to RaceTheDamnedCar and produce objective measurements. It's a rare treat around here.


CategoryLisp, CategorySyntax, CategoryMetrics


MayThirteen


EditText of this page (last edited August 18, 2014) or FindPage with title or text search