Why Functional Programming Languages Arent Mainstream

Recent Thread from comp.lang.scheme.

Simple question,

Can I do commercial applications with PLT Scheme?

-- Harald

Of course a FunctionalWeenie would point out that Scheme isn't a FunctionalLanguage, and therefore the question is irrelevant.

let's see..

Basis: C has been used in commercial applications

Induction: mzc can convert scheme to C

           http://www.plt-scheme.org/software/mzc/

Ok. If I were your boss and I told you that you could use any language that you can prove was used in a commercial application, and you used Scheme, and you used this as "proof", your ass would be out in the street in a New York minute.

( Not that I am saying anything about Scheme's viability for commercial applications. I just dislike intellectual dishonesty. Most Scheme programmers should dislike this too, because it probably drive more programmers away than it attracts. )

Apparently these same bosses don't appreciate a sense of humor or irony.

I am reminded of a friend who lost his job and decided to start his own company. In the process of writing his flagship software he was in such dire straights that he risked his home ( by coming close to missing the mortgage payments ).

Do you think he would would appreciate the humor?

How about the IT manager who might lose his job if a project fails?

Until FP advocates come to realize that many professionals risk their futures should they try out their language du jour. In the mainstream the knowledge that you will finish is more important than the knowledge that you might finish fast.

The reason is simple; and it has nothing to do with the merits of FunctionalProgrammingLanguages.

Ignoring projects with a legacy code requirement, most IT managers, SW engineering managers, etc. want to use languages for which an ArmyOfProgrammers is available, and which have been shown to be adequate (not ideal; adequacy suffices) for the task at hand. This means C/C++, Java, VB, WhateverDotNet?, Perl, etc.

Other languages, including functional languages such as LispLanguage and imperative languages such as SmalltalkLanguage, may be better suited to the job and easier to develop for. I won't argue this point. However, neither language has an ArmyOfProgrammers proficient in its use. PointyHairedBosses also seem to be convinced that the tools, libraries, and other infrastructure available for these are not as well-developed as for the aforementioned MainstreamLanguages.

Or to put it in another way.... suppose the project fails and the ProjectManager has to explain why to the CIO. If a MainstreamLanguage is used, one that IT shops round the world can use successfully; that choice won't be questioned. If, on the other hand, a project written in LispLanguage fails, regardless of the reason for the failure some PHB will likely point a finger at the language choice. And whoever specified that language may be held accountable.

In other words, NobodyEverGotFired? for using Java.

This "logic" is why virtually no IT shop is able to make anything work today. The BioPharm? world is collapsing because, among other things, the army of "programmers" can't keep their hodge-podge of Perl, Java, C, C++, shell scripts, Unix, DotNet, and everything else running reliably enough long enough for the scientists to get any research done. We spend two weeks hearing breathless hysteria about worms and viruses because all those "IT managers, SW engineering managers, etc." can't figure out that a monoculture breeds disaster, that security patches are probably a good thing, and that building enterprise-critical applications on a Microsoft operating system is begging for trouble. Maybe if a few more people were fired for using Java, and maybe if a few more managers would listen when a senior developer attempts to explain why Lisp, Smalltalk, Scheme, and similar languages would be a better choice, then our industry might improve a bit.

(Offtopic OS flamewar deleted. Take it elsewhere, guys, will you?)

Keep in mind, when PointyHairedBosses talk about avoiding risk; often times the risk that they are trying to avoid is that a project failure might be traceable to them; not the risk that the project might fail. Doing something bold and original is a great way to get blamed. 'Tis not good to be the TallestDandelionInTheField? when the lawnmower is bearing down....

-- ScottJohnson

Success is also a deterrent to using less popular languages. Suppose the project succeeds. Obscure language programmers are in a position to demand higher salaries to maintain the software. -- EricHodges

Which is why managers want to have an ArmyOfProgrammers available. SmugLispWeenies don't come cheap, you know...

This "logic" is why virtually no IT shop is able to make anything work today.

I've seen more projects fail because they picked Smalltalk than fail because they picked Java. And most of the IT shops I've seen make most of their projects work. (By fail I mean can't be made to perform as desired within a reasonable budget.) -- EricHodges

I suspect that we're each in mostly non-intersecting and at least temporarily self-sustaining environments. I know of no projects that failed "because they picked Smalltalk", and I know of lots of projects that failed because they picked Java, C++, or Perl. But in my case that's because most of my engagements have been Smalltalk engagements, where I was brought in after the other alternatives had crashed and burned.

In all cases mentioned - how do we know that the language was the cause of the failure? And if the language is implicated, is it because the language was inappropriate to the task, or because the developers weren't sufficiently skilled in the language to use it correctly? As you mention below, a C++ project failed due to COBOL programmers being used to develop it. Is this the fault of C++, or the fault of the inexperienced team? I'm not trying to defend CeePlusPlus necessarily - it has quite a few glaring shortcomings when it comes to writing enterprise application code - but blaming the language (or any other technology) for a project failure is often a cheap and easy thing to do. (Sometimes it is the correct conclusion, of course). But just because project team X failed using Y and you succeed using SmalltalkLanguage doesn't necessarily implicate language Y.

The logic failure that I most object to is the ArmyOfProgrammers argument. In my view, it is analogous to the perhaps apocryphal story of the legislature that voted (allegedly in all seriousness) to make the acceleration due to gravity be exactly 10 m/per sec per sec, because that was a more convenient number and would therefore reduce the expenses of government contracts. The fact that a gazillion eager programmers believe that a language is "better" or is "mainstream" cannot change basic shortcomings of the language - as our industry learned to its great dismay after we attempted to standardize on C++ because the pointy-heads decided it was better (without bothering to actually consult the trenches).

I agree the ArmyOfProgrammers argument is a bit flawed; but it isn't anywhere near as absurd as defining gravity to be 10 m/sec^s instead of 9.8. (or defining pi to be 3.00). However, it does reflect reality. Smalltalk programmers are quite a bit more expensive (and quite less readily available) than are folks who can pound out C/C++, Java, or Perl. Likewise with programmers in Lisp or any other non-mainstream language. The PointyHairedBosses don't seem to believe that the extra expense is worth it. Of course, the fact that there aren't major corporations with million-dollar marketing budgets promoting the likes of Smalltalk and LISP probably has a lot to do with it as well.

Not to put too fine a point on it, but I made a very successful living during 90's cleaning up the messes created by IT departments in the financial services industry who adopted C++ because it was "mainstream", only to find (two or three years and many millions of dollars later) that it just couldn't be made to work by the people at hand (usually COBOL programmers) for the problem at hand (like a loan syndication system, for example). You haven't lived until you've tried to explain to a C++ vendor (in, say, 1992) what the phrase "two-phase commit" means and why you care.

C++ in 1992 was still quite abysmal. Of course, many say it's gotten worse since then. :)

See DefinitionOfProjectFailure

I personally think that having an ArmyOfProgrammers is a good reason to avoid that language. If I ever start a software company, I will choose my OperatingSystem, ProgrammingLanguage etc. in such a way that the run of the mill programmer won't even apply for a job. This has nothing to do with the fact that one ProgrammingLanguage might be better than the other. If I look for a Java developer today, I will get hundreds of applications from candidates. Some of these will be good programmers, but there will be many who don't like to program and are in only for the paycheck. There will also be many who are Java programmers because they took a three month course, but have no fundamentals. It is a very hard task to separate out the wheat from the chaff! And we all know that one bad programmer can create such a big mess that a lot more effort will be needed to clean it up. On the other hand, however you might fault SmugLispWeenies, if I look for a lisp programmer willing to develop in Linux, I will get a much shorter list of applicants, most of whom will actually be passionate about programming and developing good software. -- SriramGopalan

The ArmyOfProgrammers argument is undermined by another argument too; Lisp, Forth, Smalltalk, Scheme, Haskell, . . . all are vastly more productive languages in their domains. A single programmer can produce more functionality per unit of time in these languages than any other language. Consequently, given equal project velocities, you need fewer coders in a more productive language than in a less productive language. While an accomplished Haskell SoldierOfCode? may cost more than a CeeLanguage soldier, the fact is, you'll need fewer of them. All things considered, you may even end up spending less for overall project completion using a Haskell or Lisp coder than a C coder. --SamuelFalvo?

Do you have any evidence for the productivity argument?

[Exactly - let's see it.]

[And comparing to CeeLanguage? Any language with higher level features will be more productive than C. Regarding "in their domain". What is the domain then? Robots that have AI can be powered by Cee, Ada, Oberon, or other procedural languages. Video games have extensive AI algorithms in them.. consider how smart the computer is when you play NHL against it or when you drive your car off a concrete jump in Grand Theft Auto and it does all sorts of cool flips and the car lands and gets damaged. ]

[Would all this artificial intelligence in video games be better off if it was done with a functional language? Most often the AI advocates (usually the domain of functional languages, although please mention more domains if you have any) miss the fact that AI can be done well in other languages - as proven via video games, robots, restartable rocket ship programs via Ada, etc. So once again - please do specify this domain that the languages excel in.. Often, we see functional programmers using functional languages for imperative usage - such as web programs that output data, and tweaked lisp that looks iterative and imperative more than functional.]

I have already tried to get something more concrete and dissect-able without success:

I just get back what seem like excuses, nebulosity, and MovingGoalPosts. There is NoSilverBullet on the global level, so use what best fits your head.

--top