Development Pyramid

In a 28 Feb 2005 article (http://www.codinghorror.com/blog/archives/000224.html), Jeff Atwood asserts that the programming world has a pyramidal structure, with a handful of elite kernel developers at the top, a second tier of a few hundred highly skilled OS developers, a third tier of thousands of framework and core-application developers, and finally a base of several million small-app and web-site developers built around the frameworks created by the third tier. From this he argues that the majority of programmers should be encouraged to learn how to use the frameworks efficiently, and that teaching low-level programming at the undergraduate level is counterproductive, as the majority of programmers do not need to understand it to work with the frameworks which he assumes will dominate the careers of all but a very few.

More generally, he argues that optimization should be done at the level where it improves the performance of the most ClientProgrammers?; thus, it is inappropriate for a base-tier programmer to put a lot of effort into the kind of one-off specially-optimized functions which tend to proliferate in most coding shops, who instead should trust that the developers higher up the food chain have already done a better job of optimizing the code than they can.

Opinions?

I'm rather divided on this myself. His point about one-offs is really an argument against PrematureOptimization. His assertion that frameworks are both necessary and inevitable, and that application level programmers should just keep their heads down and not try to emulate their betters, smacks of elitism, yet for better or worse the truth is that most EnlisteeCoders in the business world don't ever need to work on low-level programming. While the leverage argument has merits, the blind faith in the power of frameworks and their developers which he advocates seems misplaced. - JayOsako

Having given more thought to it, I have concluded that the premise of the article is flawed. It grossly oversimplifies the relationship between expertise and specialization, and reasons ex post facto regarding the causes of the distribution he perceives; most of the flaws are rooted in the false assumptions of the CultOfSystemsProgramming.

First, it assumes that the programming field as a whole can be described as a singly rooted hierarchy, with experts on top and non-experts on the bottom. Implicit in this are the assumptions that a) systems programming is the most difficult of all programming fields, and b) the only difference between a business applications programmer and a systems programmer is overall skill. Both of these are demonstrably false; while the field as a whole does share a broad common base of practices, there are a wide number of specializations in programming (as well as non-programming specializations that involve some programming as an adjunct, such as MIS and system administration). To master any one field takes both study and inclination, and there is no reason to believe a given programmer would be proficient in all of them just because he or she is a talented systems programmer. I have known many very talented systems programmers who have no comprehension of databases, and conversely, equally competent DBAs who could not design a web site or write a device driver. While some of the best programmers are generalists, most of the true experts are proficient in a relatively narrow field or handful of fields, and even the broadest polymath cannot master everything.

The article incorrectly concludes that, since commercial OS development is dominated by a handful of extremely competent programmers, systems programming must be the realm of the most rarified of wizards. This reverses cause and effect. The truth is, the niche for commercial operating systems is very deep, but very narrow, and it is the natural inclination of the market to settle into a near monoculture based on the lowest common denominator. Thus, the field for professional systems programmers is small and highly competitive, with the majority of systems programmers being shunted off into fields which they are less adept at. This by no mens implies that those handful are the absolute best of all programmers, nor does it imply that their work is more difficult than that in fields where the demand for programmers and operators is very large (e.g., client-server integration). Indeed, the more advanced enterprise applications are probably more demanding than systems, not less; in the words of The Tao of Programming:

When designing an accounting package, the programmer operates as a mediator between people having different ideas: how it must operate, how its reports must appear, and how it must conform to the tax laws. By contrast, an operating system is not limited by outside appearances. When designing an operating system, the programmer seeks the simplest harmony between machine and ideas. This is why an operating system is easier to design.

I specifically emphasized the word 'professional' earlier because there is in fact a very wide and active, if not always terribly productive, community of hobbyist student system programmers, who write operating systems in vast profusion; these range from the ubiquitous Unix and MS-DOS clones, to various experimental systems which resemble nothing else ever seen. Also, in the field of embedded real-time systems, where compatibility is of minimal importance, there are a large number of competing commercial and open-source systems. Thus, it should be clear that systems programming, like the rest of the field, includes many individuals whose skill and experience covers the full range. - JayOsako

I'm not sure about this. If you view the university as just a vocational training program for industry, then yeah. It makes no sense to worry about low level concepts when almost everyone needs to program with high-level frameworks to make ends meet. Better to teach how to get up-to-speed with libraries, APIs, protocols, and frameworks quickly, because that's what everyone will be using.

But I'm not sure I buy the premise that the university exists to supply talented programmers to the workforce. A lot of major productivity gains come about because someone has a radically different way of doing things that yields order-of-magnitude improvements. These types of revolutions are essentially unpredictable, but people who come up with them need a certain basic level of understanding. I think a large function of the university is to raise people to the point where they could conceivably make a breakthrough, even if 99.99% of them won't. It's like how venture capitalists scatter their money across 20-1 longshots in the hopes that one of them will make a 100-1 payoff.

If you just learn how Microsoft .NET or J2EE works, then there's basically no chance that you'll think up some way of doing things different. But if you learn how Microsoft .NET or J2EE works and you learn how the people who designed them built upon the people who designed C built upon the people who designed assembly built upon the people who designed digital logic, you might conceivably be able to build something better.

As someone who wants to be an entrepreneur-who-knows-how-to-program, I don't know where this DevelopmentPyramid would place me. Essentially all my non-academic work, even in projects I've initiated myself, has been in the base tier (with the possible exception of an API I'm working on now). But I need to know all the fundamentals, the theory, the academic publications so that I'm aware of new opportunities. And I find that when I'm starting a new project, frameworks and libraries and languages are all irrelevant. The important stuff are specs, user feedback, and sometimes a good deal of hard math.

It's why I study the history & people of computing and not just the latest technologies. Sure, I ain't gonna invent Netscape or UNIX or Lisp or semaphores. But I want to know how they did it so that if I ever have the opportunity to invent the next whatever-it-is, I'm not left in the cold. -- JonathanTang