Individual Experience Shapes Perceptions

The title may strike one as being pretty obvious, but perhaps we underrate the extent to which it influences our perceptions about the general IT world or niches/specialties outside of our own.

Regarding the "staff-confusion-vs-FP wars", It seems fairly clear that our work environments are different based on multiple discussions and this appears to be large reason for our different perceptions about "typical developers". I usually work in "decentralized" IT where each department/section/group/region has its own set of developers for custom projects for non-IT companies. These developers are chosen for people skills, business skills, domain skills, writing skills, and often general IT experience; not JUST technical knowledge. All these types of skills are weighed on. This also means that such employes are probably not the "high end" of the developer pool from a purely technical standpoint because balancing for the others often requires sacrificing some on the technical side. It's rare an employee is "A" grade in all 6 of those categories, unless a company is prepared to pay a premium. Even if they are lucky to get an A-grade on the technical side, they cannot rely on all future code maintainers to also be A-grade and thus want code that caters to the average hiree.

Often they don't have a degree or have a degree in a non-IT field. And, often non-technical managers make the final hiring decisions. One long-term-contract placement center inadvertently relayed what the interviewer told them about me: "He seems to know his technology well, but he's a stick-in-the-mud personality-wise. We'll pass".

On the other hand, my FP-related detractors seem to be "machine-centric" specialists who are brought in when high machine performance or high algorithm optimization is required. Thus, you are expected to be "high end" from the technical side of things and are surrounded by similar colleagues. You probably have a Masters or higher, which means you are being paid a premium for your technical skills and are expected to know it well. Thus, the learning or confusion-related issues are not seen as a bottleneck from that perspective because the staff is expected to know many coding idioms and paradigms well because they are hired at a premium for that very reason. Specialists are typically paid more. They are not brought in to be general business analysts or wide-view domain experts. They are to know code and computers well above all else. (This may make them a little naive about the business side, though.)

Personally I prefer direct interaction and feedback from users. I find it much more satisfying building tools to help people do their job better, and seeing it directly help their work. That motivates me more than money. I do not want to be a "heads-down" pure coder and am usually partially also an analyst (programmer/analyst). I have to pay close attention to what managers and owners expect in terms of trade-offs, and this means maintenance staffing also. This has shaped my view of what tends to confuse or choke up typical developers. I've even altered styles I personally prefer to match expectations. I don't guarantee my observation-related conclusions are 100% accurate, but we don't have formal studies to settle such either way: I just report what I have observed, heard, and surmised based on a wide variety of conversations and clues (including their source code).

--top

But for minor and inconsequential details, I generally agree. I would add only that "typical developers" is highly subjective, per CompaniesHireLikeMinded. What I call a "typical developer" may well be different from what you call a "typical developer".

However, my interest in FunctionalProgramming and related "higher order" programming does not come from a "machine-centric" background but a user-centric one. In my experience, FunctionalProgramming makes it possible to make more reliable software in a shorter time-frame than using conventional imperative programming languages. That means happier users, and it comes down to one thing: managing state. Managing state is complicated -- though simple languages can deceive the naive programmer into thinking it's easy -- and error-prone, especially in concurrent environments and/or using unreliable networks. Languages, paradigms, approaches and tools that make it easier to manage state -- like FunctionalProgramming -- make it easier to make better programs, and better programs result in happier users. Happy users make me happy.

Of course, if the work you do isn't highly concurrent or subject to an unreliable network, this may not be an issue for you. As a result, conventional imperative programming may seem entirely sufficient to keep your users happy.

I'm not sure what you mean by "highly concurrent". Typically concurrency is handled using RDBMS ACID capabilities in apps I encounter, and the web server to a lessor extent.

Sure, that works for some stuff. However -- as an example of where these technologies are not sufficient -- using an RDBMS's ACID capabilities (or the web server), if process A updates table T how does process B get notified immediately and get told which rows of T have changed?

Define "immediately". Pulling on date/time stamps can get roughly a 5-second turn-around time which is good enough for typical manage-information-systems business use. Is this back to your pet example: the dashboard/brady-bunch example? Haven't we beat that to death? My summary: you used shitty languages and/or API's as a straw-man.
"Immediately" implies "not via polling". It has nothing to do with the "Brady Bunch example" or any other example shown here. Whilst you may not have to do it, many of us have to implement highly concurrent, event-driven, non-polling, no-SQL-DBMS-is-fast-enough systems on a daily basis.
Sorry you are stuck with JSshit as a language. My condolences.
Huh? Do you mean JavaScript? I rarely use Javascript, and what does it have to do with anything? Language, here, is irrelevant.
I don't know what your point is then. I suggest creating a well-described example/scenario.
Sure. Imagine an instant-messaging application.
How are HOF-based API's for it better than all possible OO-based API's for it?
It isn't about "HOF-based APIs" vs "OO-based APIs", but about FunctionalProgramming in general which deprecates stateful programming in favour of mapping input to output in side-effect-free functions. That makes handling concurrency simpler than imperative programming because it minimises concurrent access to shared mutable data. However, in general HOFs are simpler than OO equivalents because they implicitly close over their defining environment, whereas the defining environment must be explicitly passed to FunctorObjects.
Every paradigm/technique probably has a niche or application category where it shines compared to others. I don't dispute that.
- Note that if custom instant message applications became popular, then API's that rolled up concurrency-related issues into an OOP interface may emerge that takes care of most of the tricky parts of parallelism and concurrency. It may not be as machine-efficient as a carefully-rolled FP version (or assembler), but often many orgs find its cheaper to throw hardware at the problem instead of hiring more costly developers. -t
- It's only one example. If custom instant messaging gets replaced with commercial instant messaging, there will still be an unlimited number of highly concurrent problems to solve. Some companies will throw hardware at their problems, but they'll be less competitive than companies who can solve them more cheaply with software. A good programmer who can write that software is paid about the same as a bad one, but costs less in lost productivity and is worth more in gained productivity.
- Cutting edge technology is indeed an area where a HackerLanguage may give you an edge because being first-to-market gives you an advantage above longer-term staffing issues. In general, though, most orgs do seem to throw hardware at problems instead of paying for finely-tuned software. If it's hurting them relative to those that don't, it's not making itself obvious. (IfFooIsSoGreatHowComeYouAreNotRich) And like I said elsewhere, raw code writing ability is often only one of many factors used to select, judge, and pay software employees.
So why do you feel compelled to criticise HOFs, lambda expressions, FunctionalProgramming, etc., whenever we mention these things? If said paradigms/techniques aren't appropriate to your niche or application category, why not simply note the fact (to yourself, ideally) and move on?
I don't know what mention spots you are referring to in order to review and possibly repair my response. Insufficient information.
I'm not suggesting you go back and MindWipe what you've already done -- that would be a bit disruptive. In the future, perhaps you could consider whether commenting on some programming paradigm/technique adds value, or whether it's an opportunity to recognise that it's not applicable to your niche or application category and merely note that fact to yourself?
If somebody makes a universal-sounding claim, I will add a note about possible scope issues.

Re: "FunctionalProgramming makes it possible to make more reliable software in a shorter time-frame" -- I don't disagree in terms of "possible", but whether typical developers (as I encounter them) can or will leverage such is another matter. Tools that work best with average practitioners may not work the best for cream-of-the-crop programmers (or perhaps lone developers who only share code with themselves.) AlgolFamily languages with OOP for GUI and networking-related API's currently seem to cause the least confusion among typical programmers I encounter. And because of this, I focus on perfecting and documenting these techniques rather than FP. If you are stuck with Qwerty keyboards at work, then master Qwerty instead of evangelize Dvorak keyboards etc. It just causes friction.

"AlgolFamily languages with OOP for GUI and networking-related API's" probably cause the least confusion among your typical developers because that's what your typical developers have been taught. When your typical programmers are taught FunctionalProgramming -- and increasingly, we do teach FunctionalProgramming -- they'll find it familiar. Not so long ago, the same arguments you're making now are what were made against ObjectOriented programming, and before that, exactly the same arguments were made against StructuredProgramming. We know how that turned out.

The OO fanboys of the time were using OO for the wrong things, namely domain modelling. Maybe a similar culling will happen with FP and the industry will eventually settle on when to use FP and when not to. Based on the pattern of examples FP proponents seem to use, I suspect that's still several years off and we are seeing a lot of junk claims right now.

That's pure speculation.

So is any counter-view. Touche!

The only "counter-view" being presented -- other than to point out the lack of evidence behind your assertions -- is that HOFs and lambdas simplify coding. That has been amply demonstrated. Claims that HOFs and lambdas confuse programmers have not been demonstrated, nor has any other evidence been presented.

I disagree with your usage of the word "evidence", but please re-bicker it on EvidenceDiscussion instead of here.

(Moved to EvidenceDiscussion)

One of my primary detractors has roughly painted their past as this: Early in his development career Mr. X got together with some recent graduates and formed a small company to take existing DOS-machine ExBase applications and give them more attractive graphics, perhaps what could be called a "proto-GUI" (see ExBaseRant). Top students and/or carefully chosen students from a local university were recruited as C/C++ coders. The company was (allegedly) quite successful, allowing Mr. X to semi-retire into being a university instructor.

There are a mix of truths and untruths. It's true that the company was quite successful, allowing me to semi-retire into being a university instructor, and that it was formed with some recent university graduates. However, whilst we did port some existing DOS-based ExBase applications to our C++-based tools, most were new applications. Our main goal wasn't "more attractive graphics"; they were text-mode so they didn't necessarily look any different from what ExBase could do. What our C++ tools gave us was superior facilities for rapidly creating complex forms, reports, and menus; easier development of multi-user applications; much higher performance; and better support for extensibility and code re-use.

(I've seen C/C++ coder teams struggle to be competitive in custom business applications. While their software was often visually "nice" and ran quickly, it often took a long time to create or change. And if they rushed it, it was crashy.)

Weak programmers are weak in everything. Good programmers are good with everything, but their capability is increased by using better tools. Note that C/C++ (or ObjectiveC) underpins every modern, popular operating system and modern operating systems are anything but "crashy".

Like I've pointed out elsewhere, there are multiple factors (people, presentation, domain, multi-hat, etc.) typical developers are judged by such that high abstraction and/or multi-paradigm is considered one of many skills, and often limits future staffing options. Businesses typically end up catering more or less to a kind of least common denominator for the staff-flexibility reasons described in GreatLispWar. That's what I observe and that's what I am reporting.
It depends entirely on what the given IT department wants to achieve, and its overall vision.
Agreed, but my experience does not match yours at all in terms of what typical/common departments do. The fact that few other companies have matched your C/C++ experience for custom biz apps is evidence of this. (Although we seem to dispute that mix also. I invite readers to study typical job requirements on recruiting sites for "custom (internal app) business programmer/analyst" or equivalent, and am confident they will see that C/C++ is not dominant.) -t

I see three main problems with this scenario (outside of verification difficulty). First is that most developers come from a wide variety of backgrounds and degrees in my experience such that the above is probably not representative of "typical developers". Just because YOU were able to create and leverage a university-centric staff does not mean every shop can. For one, new graduates often lack business and "office" experience. If your team found a way to avoid that bottleneck, that's lovely, but may not scale to other teams. I suspect your team was hand-picked to have a similar mind-set such as to make communication of business needs relatively smooth and seamless between the analysts and the newbie coders. This may reflect a kind of recruiting and/or managing "social ability" more than education level alone, or even luck of personal encounters. In a small or start-up situation, this is generally easier to do, but may also be hit-or-miss. While I applaud Mr. X for (allegedly) being able to pull this off, it does not appear to scale or extrapolate to other shops (IfFooIsSoGreatHowComeYouAreNotRich).

Whilst "typical developers" may have, at one time, predominantly come from non-IT origins, a significant proportion -- the majority, in many companies -- are now degree educated. It's why university ComputerScience degrees are so popular.

That's not what I observe, at least outside of an IT-domain company.
It sounds like you work in an organisation dominated by end-user computing. That's not unusual, but it's certainly not a prevailing idiom.
I'm not sure what you mean by "end-user computing". If you mean user-written-programming, then NO. I don't know where you get that impression. If users make their own contraptions, they don't normally contact IT staff, including programmers. Programmers are only contacted if such apps grow beyond something user-managable and need more formalization, which is not uncommon.

Second, if you retire from production work fairly early in your career, you are not exposed to as many different organizations and IT staffing patterns. I'm not a teacher and none of my career time has been spent on formal teaching (beyond some How-To meetings). That difference has been spent in production with regular developers and analysts and users.

Good thing I have continued doing production work on a consulting basis, and regularly work with the companies where our graduates are placed.

You'd still be exposed to "university-centric" domains or customers more than normal.
Define "normal". Most cities are university towns. Very few of our customers were "university-centric".
They don't normally look to a university first to do any hiring. The impression is that recent graduates lack practical experience. With explicit specifications, graduates can often build great stuff, but often a programming-decentralized company wants a programmer/analyst hybrid to do both analysis and coding in one shot because it avoids the lossy communication step between programmer and analyst.

Third, "hanging out" at a university is probably going to bring you to a certain kind of "problem type" and co-workers when do you encounter and work on side projects (which has been implied). Your university setting is going to filter your experience.

Possibly, but having worked with a wide variety of companies, I doubt it's a significant factor.

(Another potential issue to ponder is future maintenance. If Mr X's company disappeared or charged too much for followup maintenance, can a given customer find a decent alternative, or will those outside of Mr. X's company find the special-built C++ UI libraries labor intensive to figure out? Customer lock-in may have been part of the reason for the "success". After enough customers get burned, the lock-in strategy may no longer pay off. It may be why people don't want to buy Microsoft unless they have to.)

Given that C++ is still popular, and ExBase has all but vanished, I suspect my company's approach was a better strategy in terms of "future maintenance" or "followup maintenance" than sticking with ExBase. In general, niche languages are more likely to be orphaned than mainstream languages, and whilst there is an inevitable overhead associated with learning a particular library, it is arguably smaller than learning a different language and its (inevitable) library.

C/C++ is still relatively rare for typical custom biz apps I encounter. I'd estimate 5% to 10% penetration. And Clipper, dBASE, and FoxPro are still for sale, although I agree their penetration has declined, but for reasons described in ExBase.
How much Clipper, dBASE and FoxPro do you encounter? Whilst C/C++ has peaked for custom business applications, it's been replaced with C#, VB, Java and PHP, which are general-purpose programming languages in the C/C++ idiom. C# and Java were defined specifically as successors to C++ for business application development, thus demarcating an evolutionary path from C to C++ to Java/C#/PHP/VB. Clipper, dBASE and FoxPro were a dead-end, and linger on only as niche, legacy products. None are still sold by their original companies.
I bet your C/C++ apps from the late 80's and early 90's have similarly been mostly replaced by something else. ExBase wasn't intended to be a general purpose programming language and thus had more industry-change risk. VB-classic was similarly the right product at the right time that fit a certain niche well (internal RAD desktop GUI's) but wasn't a good fit for the web or large-scale and/or packaged applications. (VB.Net has been characterized as encouraging or requiring bloated/verbose coding and is thus arguably not comparable and not as popular as VB-classic, at least for smaller projects.)

We can argue about what is the most likely scenario and/or how recruiting and staffing "should" be done. But neither side has solid studies to demonstrate either scenario. I'm satisfied to LetTheReaderDecide which shop scenario best represents their staffing situation rather than bicker over different views of staffing reality, but Mr. X seems rather insistent that he is right and appears upset when I add an alternative viewpoint into Mr. X's wiki writing. -top

You're the only Wiki participant I know who frequently descends into cursing and general vulgarity. You are the one, therefore, who "appears upset" when his views are challenged.

There are other ways to display "upset". I would note that truck drivers are known to curse even when happy such that cursing by itself is not necessarily a sign of "upset". But I do not wish to get into a LaynesLaw loop over the word "upset".