Why Do People Make So Many Mistakes
Why Do People Make So Many MistakesA common thread in many threads is to adopt strategy X because we make many mistakes. Why is that? How is that we are so bad at our jobs that we fail so often?
We can't do any design up front because we'll be wrong. We can't write code correctly, so we need to double the code-writing effort to create unit tests. We can't write code properly, so we need someone to pair with us to smooth our failures. We can't create frameworks, because there is a chance we'll fail.
Why do you fail so often that you need to take these steps? If I had failed this often, I would wonder what was wrong with me. I couldn't blame the methodology or the tools.
We make "honest mistakes" for the following reasons:
The original author didn't sign. But I'll bet the author is either in management or in school.
-- KrisJohnson
I've been out of school for a while, and I'm not in management, but ChangingConditions? happens a lot in real-world software development. When someone says, "Oh, yeah, and the client needs X changed," the usual response is, "Of course."
I would suspect 'complexity' is the main culprit, and would perhaps also list 'unforeseen circumstances'. -- VickiKerr
The original author didn't sign. But I'll bet the author is either in management or in school. -- KrisJohnson
Neither. I've had very good experiences with developing frameworks, doing up-front design (no big up-front design, though), and producing working code in over 16 years of development. It's seldom that a unit test actually catches an error, but I still do them. System tests might be more economical, given the error rates. So I really don't understand where all these horror stories come from. I don't understand why your code doesn't work, when it should if you were skilled at your job. I don't understand why your frameworks are so horrible that they need to be trashed after years of work. It sounds like incompetence to me. So rather than argue each methodology strategy point by point, it occurred to me in a flash that I really don't understand why skilled programmers would be having so many problems.
I'm not being jerky, mean, or purposely argumentative. It is an honest question. Skilled professionals should be able to do far better than it seems they are doing. Maybe the problem isn't the methodology or anything other than the programmers aren't very good to begin with?
If you don't make mistakes, then you are a very uncommon person. Most people do make mistakes. But the methodologies promoted here aren't really focused on programming errors. I agree that a competent programmer should be able to write code that works. But writing working code is a very minor part of the average developer's job. These methodologies are focused upon not making mistakes when assuming or "guessing" that you know what the user/client wants. So by "mistake", we mean "failure to predict the future" or "failure to read people's minds to find out what they really want." Software development happens to be one of the few areas of human endeavor where a very minor mistake can have major consequences. So it is good to have methodologies that are tolerant of mistakes, rather than methodologies that assume perfection. -- KrisJohnson
The empirical evidence is pretty much unanimously against the original poster. Even the very best programmers make a lot of mistakes, at least on large systems. The vast majority of such systems fail at least in some respects, and many of them catastrophically. I can't think of a single order 10 Mloc system (and I have worked on quite a few) that isn't in significant trouble of one sort or another. I would be quite surprised to find one that wasn't. The problem isn't bad programmers or bad management. Fundamentally, the problem is that we don't understand large software systems, let alone understand how to build them predictably. There isn't a single proven, tried-and-true method to build large systems successfully. Most methods fail predictably, and those that don't often seem to be irreproducible. If you really feel you don't fall prey to these problems, you are either the prophet this industry is waiting for, or you are deluding yourself. There is a third possibility: that you spend your time with small and or simple projects, and lack experience with larger systems. Or were you merely talking about the trivial details of putting together small pieces of software? In this case, sure. I agree that any 'skilled professional' should be able to do this with little difficulty. But that is not the real work, and it is not where the problems lie. -- AnonymousDonor
Don't know about you other folks. I've been doing software for nigh on to 45 years now, man and boy, and I'm thought to be pretty good at it. But I make lots of mistakes. That's why I try to adopt practices that help me catch those mistakes before they cause me trouble. When you get really good at it, you can make mistakes and catch them so fast that to other people you look really smart. -- RonJeffries
It's not just that people make mistakes. That is trivially true, and shows a certain lack of depth of thought to assert that's all that is being said. We are talking about systematic failure whose reach into every part of development is truly breathtaking and may have no parallel in any other profession.
It is interesting that the reaction is oh this person is merely a student, academic, or someone who has only worked on small simple projects. The thought that people just may not be very skilled is somehow not acceptable. I've noticed that people who are good at their jobs produce very few bugs. Even on very large and very complex systems. How is this possible? How is it possible that some people are so much better than others? What do they do differently?
The thought that people may just not be skilled is fine. But the evidence I have seen doesn't support it. What you have claim to have noticed, I don't believe is true. I don't think there are any people who do not make design and implementation errors on large complex systems. I certainly haven't met any, and I have known some brilliant programmers. The problem is that *nobody* understands these systems, and *nobody* has a repeatable method for tackling them successfully. If I suggested (above) that the original poster might only have worked with small systems, it is because that was one hypothesis that fit the discrepancy between my (and many others') experiences and the view espoused at the top of this page. -- AnonymousDonor
So it is good to have methodologies that are tolerant of mistakes, rather than methodologies that assume perfection. -- KrisJohnson
A mistake means code has to change. This is methodology-independent. Well-built code, regardless of methodology, should be equally easy/hard to change. I'm not aware of any methodology that requires perfection, that says don't test, that says spread duplicate code everywhere, or that says spend three years developing useless code. Developers may do this, but that's a lack of skill on their part. It's not required.
My first reaction is that if you don't make many mistakes, you must not be challenging yourself.
Many of the techniques you mention also cover for your team members' mistakes, and you cannot guarantee that management will always bless you with teammates as skilled as yourself.
So it's not possible to challenge yourself and have high quality? It sounds like another excuse for sub-par performance.
It may just be an excuse. But that would be sad, because it is a desperate excuse for anyone (and much more so for programmers) to claim a lower level of skill than he or she actually has. I'd hate to think what kind of environment I'd have to be in to produce low-quality work and say it was because the work was overly challenging, when in fact it was well within my capabilities.
It's possible to challenge yourself to produce high quality work. But if you're going fast and making no mistakes, you're in your comfort zone and not learning anything. If you're learning a lot and making no mistakes, you're probably not going very fast. Either way, you'd better hope your competition is as perfectionist as you.
Software development is a young trade. It is going to make a lot of mistakes. When somebody complains that there are so many errors in software, they often compare it to engineering. "If builders built buildings like people wrote software, then TheFirstWoodpecker to come around would destroy civilization".
But engineering (and law, and carpentry, and medicine, and probably most of the trades that exist today) have been around for thousands of years. After a few centuries, these trades develop collective wisdom to handle the known risks. Our business is so young that we can't even agree on a brace style.
And we are asked to make mistakes. What would happen if engineers were asked to build buildings like we're asked to write software? Limited budget, political restrictions on tools, short schedules, things that have not really been done before, and requirements changing on the fly. If you had to build a building in such an environment, it would fall.
We ask software developers to do this for two reasons. First, people don't die when most programs crash (there are special techniques to handle programs that do kill when they crash--just ask NASA). Second, if you built software with the same care people built buildings, you'd be ten years behind the times. In (most) software, we trade off quality for features, because that's what the market demands and because we can.
Easy things are easy, hard things are hard. The idea that other engineering disciplines don't make mistakes is not the case. If companies spent the money on software design that they spend on, say, building a building, or designing an aircraft, then the systems might be as safe (read tested). The thesis assumes there is a better alternative in existence. I contend that alternative is a fantasy, albeit a popular one. Sorry :). -- RichardHenderson.
I would argue that a) software development is not (yet) an engineering discipline in any useful sense of the world and b) for similar reasons ,they do not make (all of) the same classes of mistakes. They do, of course, make mistakes. It is not simply a matter of money.
If you think your software is free from bugs, then someone's been doing a very good job of hiding the facts from you. (And that someone may be you. ;-)
The following paper contains several data points on bug rates found during software inspection: http://panko.cba.hawaii.edu/HumanErr/ProgNorm.htm
Are we all on the same page? Are we talking about "low level" mistakes - commonly described as "bugs" - or "high level" mistakes - design flaws, excessively complex design, "getting the requirements wrong" and so on?
See GrandMasterEliminatesWrongMoves
Perhaps you are a GrandMasterProgrammer. It certainly is possible. However, GrandMastersAreRare?. I'm sure you're familiar with the research, summarized in CodeComplete, that shows that in a typical software organization, among people with the same amount of experience, there is a 10:1 ratio of the most productive programmer to the least productive. ProgrammersAreScarce?, and GoodProgrammersAreEvenScarcer?. There is a very simple methodology out there that says you hire the top 5% of programmers and team them up; problem solved. Obviously for most organizations, this is a fiction. XP and its practices are intentionally oriented towards the vast majority of programmers who are not GrandMasters?. This means that any team can lower its ratio to maybe 5:1 or 4:1 (this is a guess and not supported by evidence).
People are different. Some people are GrandMastersOfChess?. I learned in a CognitiveScience course that GrandMastersOfChess?, statistically speaking, do not have any higher IQ than their peers. It's a skill, either learned or innate, or both. It doesn't mean that they are somehow better people or more professional. The same goes with GrandMasterProgrammers. They have a skill, either learned, innate, or both, that sets them apart from other programmers. This doesn't mean that they are better people or more professional.
By the way, I like the comparison of chess to programming, but I'd like to express my opinion that ProgrammingIsHarderThanChess.
The focus of this page was mistakes, not productivity. Although you could hypothesize that some people are productive simply because they don't make as many mistakes (has anyone attempted to demonstrate this?). I don't think that anyone who has spent time with a broad range of programmers would argue with the idea that some are much, much more productive than others. In my experience, the `grandmaster' programmers make the same classes of errors with large systems. They may get there faster, and they may work around them faster, but they still make them.
I had no choice but to quote productivity because that is what the research measured. I believe, but I can't recall for certain, that there were similar findings for bug counts, which might be a closer parallel to mistakes.
Here's another example of the fallacy that making monumental mistakes means you're unprofessional (or whatever). Back in Plato's time, how many geniuses were there, proportionally speaking? I'd wager good money that it's very close to the same proportion of geniuses today. They didn't come up with the stuff we've come up with in the past few decades. Why? Because they were unprofessional or unskilled? Sort of, but not inherently. They didn't have the knowledge of generations that we have now. They didn't have the scientific understanding and the technology that goes with it. Surely this is not something you can fault them for.
But then there was Plato. He seemed head and shoulders above the rest. Why is that? Because he had a knack for philosophy. And, there was a lot of philosophy to explore at that time. He had a 'market niche' so to speak. But even Plato made some monumental mistakes. Correct me if I'm wrong (I may be attributing this to the wrong guy), but I believe it was Plato who came up with the four elements, earth, air, fire, and water. You couldn't do any serious physics today if you held onto that theory. Still, it proved a useful paradigm for millennia, and helped pave the way for the science of today. In a way, it brought everyone else up to Plato's level.
So it is with software today. There's a lot of hoojah-boojah going on in the methodology circles, and you've got this incredibly huge productivity gap between programmers of the same level of experience. Can you really blame them? SoftwareEngineering and ComputerScience have only been around for about half a century. It's not a matter of professionalism or skill level, it's a matter of history; i.e. there is none. So XP comes along (and I know it's a bit far-fetched, but bear with me), and it proves (or hopes to prove) to be a useful paradigm to bring everyone else up to a GrandMasterProgrammer's level. Isn't that a good thing?
GrandMastersOfChess??, statistically speaking, do not have any higher IQ than their peers.
The issue is skill, learning, practice, not IQ. The implication that better programmers are smarter was never ever made nor would it be supportable. The best brick layers are more skilled, not necessarily smarter. In software we don't even train our developers as well as brick layers are trained.
I've interviewed hundreds of "programmers" from all schools and experience levels. I'd be happy if 5% had a high level of skill. But they don't. They know very little about how to develop software systems. Most of them don't even know they are in a profession that has a history, that has a variety of available tools, and that you have to study and get better at. When I ask, "How do you make yourself better at your job?" in an interview, most look at me as if they have no idea what I'm talking about. Most developers do not even have basic skills. A methodology does not help these people; it does not magically turn a sow's ear into a silk purse. You can tell them to write unit tests and refactor, but this is not generative. It does not tell developers what to do. To know what to do requires skill and judgement. The Manhattan Project could never have succeeded, for example, with a group of unskilled people using a nifty methodology.
Please tell us HowToBecomeaSkilledProgrammer.
I don't think there are any people who do not make design and implementation errors on large complex systems.
That's not the assertion. The assertion is not that there are people who never make mistakes.
The essential question: Is the class of mistakes sufficiently pandemic to justify the assumptions and subsequent recommendations made by XP?
In my opinion, SkilledProfessionals? should not be making mistakes of the order where the assumptions behind XP become true. A SkilledProfessional? should be able to:
I am not convinced your SkilledProfessional? exists. I still believe that even the best programmers consistently make fundamental errors in the construction of large systems. Not because they are unskilled. Not because they are not using the current best practices for design and implementation. Because they, like everyone else, don't understand large scale software systems. They don't understand how they grow, how the parts interact, the long term effects of todays design decisions. They don't understand how to fix a system in maintenance hell, or how to stop one from getting there. You (or some group of people with similar posts) keep asserting these things, but I don't see any empirical evidence to support it. Perhaps the problem is simply that there are no (or a vanishingly small number) of skilled professionals in the software field. If that is true, it is probably more of a reflection on the field than the practitioners.
I am not a XP booster; I find some of the ideas interesting and may even try them sometime, but I am certainly not a convert. So understand that I am not objecting to your statements as an XP vs. the world response. Rather, I find your claims badly out-of-sync with reality as I know it, and it seems that I am not alone. My experience suggests that even under the best conditions, with cream-of-the-crop people working on it, a large-scale project will get into trouble.
I'll have to agree with the "the essential question" comments just above. Skilled professionals don't doom projects by doing BigDesignUpFront. Lots of BigDesignUpFront projects have been successful. The real claim of XP is not that all BigDesignUpFront projects fail. Rather it is that an XP project with the same task will have less risk, will deliver value to the customer sooner, will be less expensive overall, and will produce higher-quality code.
My experience has been that BigDesignUpFront projects always become unmaintainable over time--not because people make mistakes, but because most "skilled professionals" out there refuse to do refactoring of any kind. This is not an essential property of BigDesignUpFront, but it is a common practice. (Or lack of practice, as it were.) -- JeffGrigg
I still believe that even the best programmers consistently make fundamental errors in the construction of large systems.
So you are in trouble. Now what? Is the fix XP? If so, why? XP says not to be so afraid to change your system, which I think is excellent advice in smaller groups. Larger group dynamics are completely different, but the actual changing of software in XP is no different than for any other methodology. XP code is no easier to change than any other well-built code. If your XP code sucks, it will be hard to change as well. There's nothing in XP that creates well-built code. Testing just makes sure it works. It can be complete crap,, yet still pass all the tests. I see it all the time :-) And anyone can test. Well-built code (for a good overview one might look at RobertMartin) comes from skill, not the repeated application of tests and refactoring--these are not generative concepts. Even OnceAndOnlyOnce, a winning idea for a very long time, doesn't say how to do it, and the skills to do it are completely methodology independent.
I'll have to disagree with you on the "there's nothing in XP that creates well built code" comment: (Hmmm... I see we haven't been talking about this for a while, but...) When the CodeSmells, you're required to RefactorMercilessly until it doesn't. SimpleDesign and YouArentGonnaNeedIt keep you from doing complex messy things unless you can prove that the customer absolutely requires them. -- JeffGrigg
My experience has been that BigDesignUpFront projects always become unmaintainable over time.
That's kind of a strawman though. It's not a choice between big up-front design or XP. It's XP's fundamentalism that frames the debate in these either/or terms, but it is not so. There is a range in between that can and has delivered results. I would never recommend big up-front design, because you cannot see that far, but I can and have seen a lot farther than XP admits to.
You're right, it certainly isn't XP vs. BDUF. However, my experience is that all (non trivial) projects become unmaintainable over time. Better practices tend (but not universally) to increase the length of time and/or size of project before this happens, but they don't solve the problem. There are no current practices or methodologies that I am aware of that allow predictable, controllable construction of large systems. I wouldn't stake the chances of getting a 10+ Mloc project to fly correctly at better than chance today :)
It seems like the point of this page is not, "Why do people make so many mistakes." It's, "Why do programmers make so many mistakes?" (The answer to the first question, after all, is theological: that's just the way we are, ask a deity for details.)
The answer to the second question: because programmers are people too. (Although non-programmers sometimes wonder.) So maybe what we're really asking is, "Why don't other disciplines make so many mistakes? What are we doing wrong here!!???"
My answer: "What do you mean, they don't make mistakes?" Here's some primary-source tidbits I've heard recently:
-- JimLittle
On the other hand, maybe I've misunderstood. Maybe the question is, "What causes people to make mistakes?" In which case, I like VickiKerr's answer: complexity and unforeseen circumstances. I'll add 'miscommunication' to the list. By the way, I think this variant of the question has a lot more potential for yielding useful results. --JimLittle
They never make mistakes in other engineering disciplines. There was no Tacoma Narrows Bridge (http://www.bergen.org/AAST/Projects/Timeline/Transportation20/tacoma/main.htm http://www.nd.edu/~yzhou/tacoma.html ;-)
I would challenge the underlying assumption that individuals making mistakes are the root cause of software problems. The first step is to stop blaming the people and start looking for a solution. Most of the problems in software development are systemic in nature and cannot be tied to individual mistakes. -- WayneMack
No complex piece of software can ever be proven bug-free under all conditions, since the number of possible states is very large. Add in performance considerations and the system becomes impossible to model, and therefore impossible to determine without actually running it. Given that the smallest problem can destroy the logical integrity of a system, I think the programming problem is considerably harder than most engineering problems which simply require brute-force to model in arbitrary detail. Programming is about as non-linear as it gets. -- RichardHenderson.
Ah, well in that case, then why is it that software developers make so few mistakes?
I guess we're just dead clever :). Or test a lot.
Wayne, I like the part about not blaming (in the punishment sense), but I have always found that when I look for solutions I find causes in the bargain, and sometimes specific agents wound up in those causes, including me. I could cavil and say, "My behavior was only a cause in the context of a larger dynamic which I don't control," but that's evading the point: the simplest fix is for me to change a habit. Is that evil? -- WaldenMathews
It is easy to find who caused a mistake. It is much more difficult to understand why the mistake happened, and even more difficult to determine how to prevent the problem from recurring or if it should be prevented at all.
You focus on "habits," but the question goes deeper. How did you develop the bad habits? I'm guessing that, like many of us, the bad habits are the result of random trial and error. Wouldn't you expect mistake prone processes would result from such learning behavior? If we do not teach (or do not know how or what to teach) people ways to avoid mistakes, is it really acceptable to assign them the blame for the mistake?
The ultimate responsibility for fixing a problem falls to the individual, but the actual responsibility for preventing that type of problem rests much higher. -- WayneMack
I'd like to see you try teaching people how to avoid mistakes they haven't made yet and not come across like a blaming asshole. If I made a mistake, I want to be told what it is, and with a minimum of ceremony. Then I can fix it, figure out why I made it, and perhaps do something so I won't make it again. Does "responsibility" for this rest "much higher"? If so, I haven't a clue.
Were you taught to read, write, or do math or did you learn from your own mistakes? Do you want to have self taught surgeons? As the saying goes, "Experience is the worst teacher. She gives the test before the lesson." The responsibility for avoid mistakes rests higher than the individual programmer because we do not want each programmer to repeat every mistake for himself. There needs to be a higher level to collect and disseminate information on how to avoid mistakes. -- WayneMack
I was taught to read, write and do math and I learned all of these from my own mistakes. Your dichotomy between externally directed learning and feedback-based learning is too harsh and simplistic. When I need a surgeon, I want one who has operated enough to have made enough mistakes to be able to not make them on me. I specifically don't want a surgeon who has never made a mistake because his book learning was so thorough. Also, your assumption above about how "bad" habits developed is way off the mark. Trial-and-error is not a random process that yields random results--unless you do it that way, which means that your feedback mechanism is broken. As far as this issue being "deeper than habits", I'm not convinced. Why go deeper when you can do so much on the surface?
I seriously doubt you developed language and mathematics through a trial and error process. I'm sure you were taught the concepts of spelling, grammar, zero, fractions, and negative numbers. These things took thousands of years to develop. You will be far better off with an inexperienced surgeon of today compared to an experienced surgeon of 100 years ago. The reason we have so much to do on the surface is because we don't go deeper. Trial and error is a painfully slow process and there is no reason for each individual to have to repeat it. -- WayneMack
Wayne, your all-or-nothing reasoning is a bad fit with my experience. Even things I was "taught" required a lot of trial and error on my part before I learned them well enough to use them. It doesn't matter how many years of experience these disciplines took to develop, nor how much theoretical study you devote to their mastery. To do is to err is to be human. -- Walden
I don't buy the software sucks thing. How many people died in the last 50 years because of a software error? Now, how many people died in the last same 50 years because of an engineering error? When reliability is part of the requirements, our industry has an outstanding success record. Remember that no-one has ever died from software failure in a commercial airliner, yet. Now, when reliability is not part of the requirements, our industry has an outstanding success record in term of delivering value to the customer for a ridiculous cost. If we started writing software the same way architects build houses, not only it would take three years to implement notepad, but we would also be damned rich (or we would be unemployed, as nobody would buy our products).
-- PhilippeDetournay, software developer and proud of it
An extreme case can be found in the SpaceShuttle software.
The intro implies that people make more mistakes than expected. What is being compared? We are wired to be hunter-gatherers and evolved from apes. It's amazing we've gotten this far.
Managers push us to be better, faster, cheaper, etc.; and being the "fixers" we developers naturally are, this pressure sometimes makes us focus on what goes wrong instead of what goes right and makes us wonder if there is not a better way.
EditText of this page
(last edited September 26, 2008)
or FindPage with title or text search