Do Coders Need To Understand Why

I've been causing myself quite a bit of trouble by uttering the three-lettered interrogative, and this is something, I think, tied with an averse reaction to StaticThinking?. Anyway, the boss comes up to me, after talking to the client:

: Boss: We need to move all of the addresses for organizations and contacts into a locations table.
: Me: Why?
: Boss: [stunned silence]

: Boss: The client has asked that we use a different WYSIWYG widget in the webmail client, how long will it take to plug in FCK?
: Me: What are they trying to accomplish?
: Boss: They say the webmail agent is clunky. But how long will it take?
: Me: What sort of thing are they reacting to that caused them to conclude this?
: Boss: [piercing glare]

: Boss: We need to build a new mail server.
: Me: Why can't we use the one we already have?
: Boss: That one is having performance problems.
: Me: Why don't we invest the effort in fixing our existing server?
: Boss: We just need to do it.

I will admit that I do appear to have an incredibly averse reaction to having someone come to me with a solution for me to code and not a problem to solve. Why am I starting this page? Hmm. I guess I was just looking for empathy. Perhaps a few "Of course you need to understand why!"s.

If you've got a technically unskilled PointyHairedBoss and/or you have a good understanding of the business problem and the client's needs then, yes, you have my sympathy. However, if your boss is competent, technically skilled and client facing and you're merely a hired CodeMonkey, then your boss has every reason to expect you to ask only the questions needed to get the job done. Otherwise, you're wasting your client's time, your boss's time, and your time. It's his or her job to know why. It's your job to know how. You don't need to understand why, you need to shut up and code the solution. -- DaveVoorhis

Wow. I'm curious about your emphasis - are you speaking from experience? If so, it would be interesting to me if you'd be willing to expand on that. You've practically quoted him in the last sentence, and I don't have the skill to inquire about what forces are influencing him without making my job unpleasant. (Because it would be asking "Why?" of course.)

My contention is that the coder always needs to know why; however, that extreme could be an overreaction to forces particular to my situation. One saying of mine: "It is entirely possible to implement the spec and not solve the problem."

I am speaking from experience -- as the boss. When I was responsible for managing a small, very busy consulting and software development firm, the last thing I had time for was being second guessed by a wet-behind-the-ears, green-as-grass, newly-graduated code jockey. If you are my employee, and my decades of IT and business experience tell me it's time to replace the mail server, then you'd better replace the bloody mail server and not expect me to take time away from working with clients to explain why it should be replaced. If it turns out I've made the wrong decision, that's fine -- I'm responsible for my decisions, not you. If you think I'm about to make a major blunder, then I certainly expect you to warn me, but I don't have time to explain the "why" of every decision. I pay you to implement my decisions, not question them. When I promote you to consultant or make you a partner in the firm, then you can discuss it with me.

If your boss is at all like me, the forces in question are the competitive nature of business. In a cut-throat sector, a business that expends time explaining every decision to its employees will be beaten by a competitor that uses the same time to code solutions or address client concerns. -- DaveVoorhis

[That's all very well, but such a business may really suffer if the boss gets influenza and can't work for a while.]

That's a separate issue. The above is about effective use of limited time in order to achieve the greatest competitive advantage, and trying to get the best return on wage investment in employees. Note the last sentence of my first paragraph directly above -- the "whys" were discussed with partners and senior employees. The code jockeys and such, however, were expected to code. That may sound draconian and militaristic, but it wasn't. Explanations were freely given when time permitted, but obviously I didn't want to waste valuable time discussing whether or not to replace the mail server when I needed a new mail server by 5:00pm and clients needed pleasing or appeasing.

A partner or experienced employee covered for me when I was sick. However, this can certainly be an issue for a small business. On one occasion in the early days, when no one else was available and I was down with food poisoning, I received calls on my cell phone while hunched over the toilet. At one point, I had to politely ask a client to hold so I could puke. -- DV

In the age of OffshoreOutsourcing?, the answer is Yes! If interacting with the users in order to understand the requirements is not needed, then bean counters will offshore the task in a heatbeat.

I'll give you my reaction to your three examples.

1, move all addresses to a locations table. In this case, I would beat him to "stunned silence" by one step. The boss shouldn't know anything at all about tables or databases, especially the ones used in our products. Same goes for the customer.

2, different WYSIWYG widget, how long will it take to plug in FCK. My first thought here is that the customer doesn't likely know what FCK is. If you're getting requests like this, it means that the boss is not simply communicating customer requests to you. He is putting himself in some sort of architect or designer role. If he's not qualified to do that, then I'd document each occurrance of this and wait for one to backfire. Then, take the issue up with his boss.

3, build a new mail server because the old one is slow. I'd ask to see the results of the study that concluded the old software is slow. If I don't know what specific performance problems it has, then I may just end up writing the same ones into the new sofware. If there is no study, then I'd insist on performing one myself before starting work on a new implementation.

-- MichaelSparks

Your answers are reasonable, but seem to assume the boss is technically incompetent. Why, for example, shouldn't the boss know anything about tables or databases? What if the boss is an experienced data modeller who takes direct responsibility for maintaining the product's schema? Isn't it possible that the boss is highly skilled, has done his or her research, and is taking precisely the right course of action in each case? -- DaveVoorhis

Not necessarily incompetent; but in some organizations (including where I work), the boss oversees quite a few projects and hasn't the bandwidth to know the architectural and technical details of them all. (Fortunately, the boss here doesn't dictate technical details he has no business deciding). It can't be assumed that the boss is also the architect, and has the responsibility for making architectural and technical decisions of that nature. Certainly, in organizations where responsibilities are divided cleanly into coding and designing, it may well be the coders' lot to do as they are told. But in many organizations, programmers often need to understand the context of their designs and/or code, in order to better serve the customer.

If there is a technical lead on some project, I think it is universally a bad idea for it to be the same person as the "boss". Bosses hire and fire people, help you put in paperwork for sick days, etc. Why on earth would you want to combine that role with the role of a technical leader? They are completely different skill sets. It's very unlikely that you'll end up with someone who is good at both jobs, and even more unlikely that they would stay that way after a few years. It also leads to all sorts of subtle power struggles. What happens if there is a technical disagreement between a manager and a non-manager? It might not even be brought up for fear of retribution. It's much better to have all technical people on a level playing field where their ideas can compete based on merit. -- MichaelSparks

In part, this may be an issue of definition. To my mind, a technical lead is a "boss", but for the sake of discussion I'll assume we take "boss" to mean a managerial lead separate from a technical lead.

Where the budget exists to employ separate technical and personnel management roles, that may well be ideal. Small to medium size enterprises -- or under-budgeted departments in larger ones -- often cannot afford such luxuries. Among my clients were plumbing firms where the owner unplugged drains and managed employees, machine shops where the manager spent half the day in front of a lathe, nursing concerns where the head nurse alternated changing bedpans and reading resumes, manufacturing firms where the IT director also wrote code and wielded a wrench, and one dairy products company where the in-house programmer started each day driving a delivery truck. Out of necessity, small business operators (and, often, their employees too) are used to wearing many hats. In my small consulting firm, that meant the "bosses" (in a non-technical sense) were also technical leads. In small business, you learn to take on many roles or you don't survive, and it can be a very good thing -- it keeps the job interesting, and it keeps you in touch with many aspects of the business, including the nuts'n'bolts stuff where the money comes from.

Personally, I enjoy the challenges of such diversity, and wouldn't have it any other way. Perhaps I'm not as good a personnel manager as I'd be had I concentrated on just that, or as good a technical lead had that been my sole role, but I think I've acceptably learned to do both in those circumstances where doing both is warranted. -- DaveVoorhis

I agree that sometimes you could be forced into the combined role due to budget concerns. But that's different than setting out thinking the combined role is the best approach. It changes how people interact with someone in the role, and hopefully how someone fills the role. In my experience, teams work better when they can arrive at a design together. It's bad when there is a designated technical lead who dictates the direction to the rest of the group. It's worse when the lead is also a manager, because that means his decisions can't even be challenged. The team tends to get demoralized, viewing their roles more as secretary or technician than engineer. Of course, this assumes that you have competent people on the team. I must admit that I've never found much use for entry-level programmers. -- MichaelSparks

Entry-level programmers are great. They work long hours for minimal pay, they endure streams of unholy abuse while sucking up to Boss in an endearing manner, and their bones crunch satisfyingly under my hobnail boots. -- DaveVoorhis

Heh. Your humor works as empathy for me.

A coder who is going to DoTheRightThing needs to understand why. When problems arise and the boss isn't there to ask, the coder will make a better decision if he or she understands why. A boss who has no time to help a "wet-behind-the-ears, green-as-grass, newly-graduated code jockey" understand why will always have a "wet-behind-the-ears, green-as-grass, newly-graduated code jockey." As an entrepreneur who's been running companies for something approaching two decades, my "decades of IT and business experience tell me" that I love competitors who are "too busy" to help their coders understand why. Just ask if you'd like to understand why. -- TomStambaugh

You may be right. Had my company and yours been in competition, perhaps you would have mopped the floor with us. I make no claims that our methods were ideal, but they seemed to work. The bulk of our development was in a pre-ExtremeProgramming era, with several key elements:

We adopted a "chief surgeon" development approach. For each project, the chief surgeon -- a senior developer/architect/consultant -- was entirely responsible for making the client happy. If necessary, the chief surgeon could use code jockeys to implement portions of the project. At various times, our code jockeys were students, new graduates, local programming experts used offsite, local programming experts hired to work onsite, and Russian outsourcing firms. These weren't expected to ask "why," because they were paid to be human code generators for the chief surgeon. Do you want your code generation tools questioning your decisions? I thought not.
If appropriate, the chief surgeon was encouraged to work at the client's site, including (if appropriate) even performing client's employees' jobs if needed to gain an insider understanding of the problem domain. On-site work employed a requirements-gathering methodology we developed for the purpose. (I co-authored and published a paper on the methodology in the mid 90's, but it sank without a trace.) Instead of OnsiteCustomer, we used an inverse concept -- OnsiteConsultant. As such, the chief surgeon was expected to understand the "why" in detail. From the code jockey's point of view, he could be treated as the client.
The chief surgeon was supported by the company Tool Developer who maintained and developed the in-house software development tools and libraries. Refactoring was encouraged, particularly with an eye to finding code that could be integrated into the tools for subsequent re-use. From the Tool Developer's point of view, the chief surgeons were his clients, and his overall goal was to make software development easier for the surgeons. However, if a surgeon said, "I need tool <x> to do <y>," he'd be expected to provide it with minimal or no questions asked. The overall philosophy was that mutual expertise was to be trusted and facilitated, not questioned.
As the boss of bosses, I was ultimately responsible -- and willingly took responsibility and blame -- for everything; in particular several key long-term projects for which I was the chief surgeon. No code jockey every took the fall for anything. I was also the Tool Developer, though the nuts-and-bolts coding was often farmed out to the code jockeys under my direction. The code jockeys had no responsibility other than to produce the requested code, configure servers, and do other similar jobs. Clients, by in large, didn't know they existed.

The system was not perfect (note, for example, the TruckNumber implications of having a chief surgeon), but it was relatively successful. At some point, I'll devote a page or two to dissecting our approach in more detail. Will I use the same approach when I start my next company? Will I treat code jockeys as code-writing machines, or will I be willing to devote time to explaining the "whys?" Will I use the same methodology, adopt ExtremeProgramming, or use something else? This I don't know -- I'll decide when I do it. -- DaveVoorhis

I believe that the answer is ItDepends. And it depends on the responsability for the actions. If the boss takes full responsability, then the programmer should do it anyway. But, the boss should at least understand that the coder is forced to PlayHurt in both short and long term with this way of working. (S)He is not able to offer the best solution because (s)he does not have all the required information. And, even worst, (s)he is unable to learn from it. -- AurelianoCalvo

All too often, it is the coder who will take the fall when s**t hits the fan. Suddenly the boss is nowhere to be found. Few bosses will actually take blame along with responsibility when such an approach goes wrong (as it often does); most bosses who WILL take the blame and responsibility want their coders to understand why. -- TomStambaugh

I know, I've been there (as a coder :-( ). Life is tough! This is what I try to do: 1. Establish the responsability before any problem arises in written form. 2. Never work over time without pay. 3. Document why I'm forced to PlayHurt (usually an email with several cc:). -- AurelianoCalvo

You may find that asking why? is a little combative. Asking can you help me explain the problem we're solving or something similar may be heard in a more constructive way. If someone pushes back, you can say that it's really easy to make good software that does the wrong thing. It's error-prone, bordering on unethical, for coders to work on something when they don't understand the big-picture problem that's being solved.

Thanks for this reminder. This is something that I'm normally more aware of, but I see that I haven't been doing my normal "reveal my motivation first so it becomes a mutual problem instead of a challenge" sort of thing. (Part of the difficulty here is, although I know I've got a long record of misimplementing and other tragedies from not knowing "Why?", I can't predict a particular tragedy from not knowing why, so I end up speaking in the abstract, which is off-putting.)

In my own experience, I find that a brief statement of "why" can help steer the thought processes of those seeking to do the "how" of a thing.

In coding the implementation of a customer service logging feature, the "why" of "we need a way to allow the operators to record customer comments so that we can rapidly retrieve said comments based on content for follow-up calls." allowed us to make the comment logging feature index-and-search-friendly.

On the other hand, the omitted "why" of "so that we can extract the data for export" led us to completely recode a module that would "allow us to query the recent payments by client," as the query module was written to handle the "reasonable" case of show it on the screen, formatted for easy reading when what was needed was formatted for easy export/import. Yes, yes, the spec was lousy and assumptions were made, but it sure would have helped if "so they can send it to a collection agency" had made it into the description.

I have seen whole large-scale engineering efforts wasted because the "why" was given simply as "because the customer wants it" or "because Bob (an Exec VP) says so," only to find, later, that the driving force was that our competitors had claimed to be able to do [foo], had sold the customer on the idea, our management had mis-interpreted the concept, and passed it down to Design as an edict, "thou shalt do [phoo] for the customer." We overcame all kinds of practical barriers, waded through swamps of "why the hell would anyone want [phoo], anyway" and eventually got the concept working. The customer observed, "that's not what we were told it would be" (foo != phoo) and when we chased that rabbit we found that Acme had lied, said they could [foo], customer had used this as a lever to boost a feature upgrade from our sales staff, sales and management said "well, if Acme can do [phoo], then so can we;" and months of effort went into an unwanted feature.

I have seldom seen large-scale effort wasted when the "why" derivation was adequately presented. I have seen massive waste when "why" was denied ("you don't need to know that").

While I concur with Dave's observation that a seasoned, competent chief should not have to rationalize every instruction to a green, unblooded indian brave, I find that a succinctly stated "why" can save you a lot of grief.

-- GarryHamilton

This is exactly the reason I ask "Why?"

MayZeroSix