Evidence Rants Continued

(Continued from LessonsFromHistoryDiscussion)

"Those are just excuses. You don't even try to figure out why you like X and turn it into a clear, non-psychology-dependent description. As far as making money, since when did employers give a flying fsck about closures etc? It just makes hiring replacements more expensive for them. You live in a fantasy land in which your MentalMasturbation pays off; until you wake up. And you are wrong about me being the lone proof-wanter. The issue keeps popping up: http://lambda-the-ultimate.org/node/893 Just admit that your preferences are a PersonalChoiceElevatedToMoralImperative. Its the psychology, stupid! The truth will free you."

So... At the top of this threadlet you object to someone catching you out on your failure to provide evidence for your position that closures are "borderline MentalMasturbation," but down here you request equivalent evidence of me?

If there is no significant evidence that closures or OOP makes stuff significantly better, then *that* itself is evidence for MM. Not proof perhaps, but indeed evidence. The benefits of two things are "equal or unknown until proven otherwise".

Okay, I withdraw this statement. It needs more qualifiers. I slipped and oversimplified the world here. --top

In the absence of scientific study (which I'll get to in a minute), we do have evidence in the form of anecdotal support of numerous developers who use such paradigms and features on a daily basis. Their claims are usually along the lines that the use of feature or paradigm <x> makes implementing requirement <y> easier, more elegant, more understandable (assuming, of course, a pre-existing knowledge of <x> or its principles), more general, more reusable, or more maintainable than implementing <y> in the absence of <x>. Are we to universally regard these claims as bogus?

It is not so much "bogus", but probably the frequent human nature error of mistaking personal preferences for universal truths. If you can find a way to clearly demonstrate "easier" and "elegant" without reference to personal psychology (or state the psychological assumptions up front), that would be very helpful. There's a lot of people who would love to provide objective and relevant evidence for their pet GoldenHammer. If you show them how, you'll be greeted with flowers.

If I could do that, screw the flowers -- I'd be showered with cash. Debates of this kind have been raging for at least forty years, with no sign of abating any time soon.

This is strong evidence that it's mostly about psychology. Even simple things like GOTO's versus nested blocks that everyone agrees with have no objective evidence not tied to assumptions about human thought and perception. (Some feel a few well-placed goto's are fine, but almost nobody alive suggests going full back to the old way.)

Developers, by in large, aren't stupid. It is rather rare to see a developer obsessively continuing to use a feature even though it is clearly not of (at least personal) benefit. That said, I have seen the occasional developer stubbornly try to turn a simple solution into a tour de force of abstract OO elegance (I have not only created a yes/no dialog box, I have created a yes/no dialog box framework that runs atop my custom yes/no operating system!), or hyper-normalise a database schema to the point of un-usable complexity, and so on. However, these failures are not the fault of the paradigm or feature. Equivalent messes can be created in procedural languages or any other context you care to mention.

We also have evidence in the form of increasing interest, incorporation, and application of alternative paradigms and features in otherwise traditional contexts. For example, the addition of lambdas to C# 3.0. Arguably, some of this is mere pandering to the need for buzzword compliance and the desire to sell lengthy bullet-point feature lists, but such paradigms and features will only be sustained by developers finding them of value, as described above.

Unfortunately, what we largely don't have are (a) controlled scientific studies that definitively verify the value of feature or paradigm <p> in appropriate experimental contexts; or (b) definitive mathematical proof that <p> is superior to paradigm or feature <q>. Part of the problem here is even arriving at agreed-upon definitions of "value" or "superior", let alone the difficulty of defining an acceptable experimental methodology and getting the funding to carry it out. Fortunately, personal preference and anecdotal evidence are -- for the vast majority of developers -- sufficient justification in the absence of such experimental or mathematical evidence. The latter was offered in the days of the GOTO vs. block structures debates, but it's not clear how much impact that had compared to the natural evolution of developer preferences, and we all know where GOTOs now rank compared to block structures. Whether closures, functional programming, lambdas, and so on will follow the same route remains to be seen. (See LessonsFromHistoryDiscussion.)

By the way, I would hardly call attempting to apply a new paradigm or feature mere MentalMasturbation. Most developers are quite rational about use of features and paradigms; if it doesn't suit, it gets dropped. If it does suit, it gets leveraged to good effect. That isn't MentalMasturbation, that's sound experimentation, evaluation, and application. MentalMasturbation is purely unproductive activity.

I disagree. Many developers get bored and try to find ways to justify "toys". Even I have done so on occasion in order to learn new stuff or simply try something different out of boredom. In the longer run it may not be MM because it benefits my knowledge of alternative techniques, but it would not be something managers and owners would have approved of at the time if they knew I was taking the scenic route for a project. I didn't lie, I just withheld the fact that I was taking the scenic route. The same managers were guilty of their own GoldPlating at times also. --top

And in the same paragraph that you ask me to justify my preference for <x> via a "non-psychology-dependent" description, you claim "[i]ts the psychology, stupid!" and apparently find that sufficient to justify your preferences? Why, that sounds fair and entirely un-hypocritical!

Please explain. I don't know what your complaint here really is.

My complaint is that your request for justification is hypocritical. In one place you reject requests for evidence, in another place you demand it.

By the way, when I was an employer I gave a "flying fsck" about closures and any other feature, paradigm, language or approach that might give my company an edge over competitors, and hiring a few highly-productive developers who embrace new paradigms and approaches is actually less costly than hiring more developers -- even if they're cheaper -- who use only traditional approaches.

Then how come more places don't do it? There are many reasons why some places succeed some fail. More on this below. PaulGraham claims that LISP was his GoldenHammer, but there is no evidence that the suggestion works on a large scale beyond his endeavor.
Very true. At best we can say Graham used what worked for him. Unfortunately, neither he nor any statistically-significant sample set of Graham-clones produced an identical infrastructure using language <x>, so we have no objective, scientific means of comparing his Web-based e-commerce solution written in LISP to the same solution written in anything.
Why don't more shops explore alternative paradigms and features? For all I know, many do, but having no statistics on this I'll accept that many probably don't. I can't answer why in general, but from what I've seen it's because (a) management or developers have no interest in investing in R & D, i.e., "it's just a job" or it's considered too risky; (b) "languagephilia" -- the developers or management are heavily emotionally or financially invested in a particular platform or tool (e.g., "We're a Microsoft Solution Provider using Office Products!", typically said with great pride); (c) the shop is heavily invested in maintenance of a product developed fifteen years ago using a tool that was cutting-edge at the time; or (d) skepticism, much as you've demonstrated.

That's why we developed a custom DBMS and associated declarative language, plus C-based libraries, for developing business applications when most of our competitors were still using DbaseIII. It's why we later re-developed the DBMS in C++ and embraced OO when our competitors were still paddling about in Clipper and FoxPro. In the niche in which we worked, we ate our competition. It enabled me to essentially retire while still in my 30s and enter academia, where I can now devote full time to that which interests me most -- exploring and leveraging new paradigms and tools to competitive effect for developing business applications, and encouraging new programmers to do the same. With luck and perseverance, I expect some of them will use the same approach to eat their competition.

I've seen well-run ExBase shops and poorly ran ones. Some had the skills to leverage the advantages of ExBase and some didn't seem to "get it", trying to use it like C. I never claimed using ExBase alone would make everybody magically more productive. I only know it made me more productive. Also, I've talked with people who were amazed at the productivity of FoxPro teams compared some of the up-coming fads. Why should anyone believe your anecdotes over theirs?

They shouldn't. What they should believe from this, if nothing else, is that different things work for different people. Horses for courses, different strokes for different folks, one man's meat is another man's poison, <insert aphorism here>, etc. As such, categorical deprecation of new paradigms and features (aka "up-coming fads", apparently) is as unreasonable as categorically embracing new paradigms or features. Likewise, the same holds for categorical deprecation/embracement of only traditional methods and paradigms. As I've held all along, it seems the most reasonable approach is to stock your toolbox with as many tools as possible, keep an open mind about all of them, and apply the tools (i.e., paradigms, features, approaches, etc.) where they seem most appropriate given the problem at hand.

Anyhow, your claims are merely anecdotes. I know you want your anecdotes to count more, but I'm afraid they don't. Anecdotes have always made for poor evidence and there is no reason to change that. If you can SHOW your techniques kicking ExBase's or P/R's ass using fairly realistic code scenarios and change metrics, be my guest. Until then, stop sounding like an arrogant elitist. It's highly annoying. Hit the code or hit the road! Enough words from your yapper about how fucking great you are.

-top

Yes, my claims are merely anecdotes. I don't expect them to count one iota more or less than any other anecdote, but some people like them, and in the absence of rigorous scientific studies each anecdote may serve as a single point of evidence. Like all evidence, it's for you to decide if it's good evidence or poor evidence.

I agree that anecdotal evidence is better than no evidence. However, you cannot expect somebody to "just accept" them if they are skeptical of your claims.

I'm not sure how I can show that "[my] techniques" kick ass without some working definition of "kick ass".

And that's the problem. If somebody says "X is better", the listener is going to ask "how do you know"? Feelings are difficult to communicate and may not translate across brains. Perhaps you need to think harder about why you "like" certain techniques.

I've thought long and hard about it. ("long and hard"? back to MentalMasturbation? :-) For every <x> that I like, I like <x> because <x> makes implementing requirement <y> easier, more elegant, more understandable (assuming, of course, a pre-existing knowledge of <x> or its principles), more general, more reusable, or more maintainable than implementing <y> in the absence of <x>.

Furthermore, I don't know what "my techniques" are. I choose paradigms, tools and approaches based on what I feel are best to solve the problem at hand. For example, within the past few days I used SQL & PHP to create some purely procedural scripts on a Web server to update a SQL database, and I did some object-oriented programming in Java to further refine a functional-based (impure FP) virtual machine that implements closures in order to provide lazy evaluation in a true relational DBMS that defines a procedural database language that supports nested procedure definitions. What are "my techniques", again?

I don't call what you call them, really. Just show X being better in a clear way. Better-ness is all I care about. We'll worry about labels later.

Remember TopsQueryResultSet? I thought the OO approach kicked the procedural version's ass (and I explained why), but for you, probably not. Horses for courses, etc.

I thought I sort of agreed with you there from the perspective of the effort for the systems programmer (although it may be a tradeoff with regard to which party is favored). Remember, I don't challenge OO much for systems software. It may be better for that niche. It's custom biz apps where OO seems to melt (OopBizDomainGap).

CategoryEvidence