In All My Years Ive Never

 In all my years of judging, I have never seen before
 Someone more deserving the full penalty of law
                             -- PinkFloyd, 'The Wall'

To sum up: Someone objected to "In all my years..." as a fallacy, which it is if and only if you are trying to prove something rigorously, but in a field without a strong theoretical basis "In all my N years..." is a useful data point.

It's not a fallacy in itself because it's strictly not an argument. It is useful as a contribution to a debate from experience, and sometimes useful to the speaker as reflection. However, in reasoned argument it may imply a number of other fallacies.

I sometimes catch myself saying something foolish: "In all my years I've never..." For instance, "...never run into that problem" or "...never needed to do something like what you're suggesting", with the implication that there isn't any such thing, because if there were, I'd know about it, and so since I don't, there isn't.

This is foolish because it is not just one logical fallacy, but usually a combination of them. Here are some of the fallacies that such a statement may be implying:

Probably the most important two are the assumptions that "all my years" means anything (if I have 8 years of experience with something, someone else here probably has 35), and the false generalization from my experience to all possibilities (any single one of us necessarily can only have experienced a subset of every programming problem/environment, never mind that we like to think we've seen it all).

I refreshed my memory on fallacies from http://www.nizkor.org/features/fallacies

The most recent time I said this was on the PascalCase page; I said I'd heard it called CamelCase but never PascalCase.

On SwitchStatement, some weeks before I wrote this page, I noticed someone (spontaneously, without discussion) change their own wording from "in twenty-two years of Smalltalk programming, I've never had to" to "they are exceedingly rare, because..." - an excellent example of explaining the point better rather than appealing to years of experience; there's always a way of explaining better.

Seeing that author correct himself got me thinking about my own use of that fallacy, and after some thought, motivated this page.

You know, the funny thing is, it seems very natural when I'm the one who says this, yet I pretty much find it irritating when other people say it. Which is unreasonable, so I'll try to avoid saying this.

I suggest y'all do the same; it comes up from time to time here, not just with me.

I see that it aggravates some people to suggest (ok, insist) that they are committing a logical fallacy; does that mean that I shouldn't bring up the subject? What about intellectual honesty? Which standards do you hold highest; being nice at all costs, or striving for discovery of truth? Is that preference, either way, a matter of our personality, our personal identity, our idiosyncratic goals in life, or is there some set of issues here larger than our selves?

With all due respect, there is some value in personal first-hand experience. Sometimes even the best theory does not correspond to practice (theoretically speaking).

Well, Tom, that's interesting. You seem to have taken it rather personally for me to accuse you of a logical fallacy. But I stand by that claim; see below.

Is a reference to my first-hand experience intended to end discussion? No. Is it intended to impose a particular outcome on a decision? No. Was my contribution on the MythOfMetadata intended to:

I don't know what the "Relativist fallacy" means, and I might not understand what "Special Pleading fallacy" means.

What I do know is that I know a little bit about how systems with metadata work, I know a little bit about how systems without metadata fail, I do have twenty years of experience working with systems of each type, and my experience was directly relevant to the text in question:

There is questionable value in having the metadata encoded with the same mechanism as the data itself. There are places where this works, and places where it doesn't.

My observation that in twenty years work experience, I've never seen an example where "having the metadata encoded with the same mechanism as the data itself" hasn't worked is, in my opinion, a perfectly reasonable response to a claim. It is NOT a fallacy, logical or otherwise. It's a statement of first-hand experience. Can you, for example, cite an example of a system where having the metadata encoded with the same mechanism as the data itself has not worked (whatever it is we mean by "worked")?

This is precisely the Burden of Proof fallacy.

http://www.nizkor.org/features/fallacies/burden-of-proof.html

Burden of Proof is a fallacy in which the burden of proof is placed on the wrong side. Another version occurs when a lack of evidence for side A is taken to be evidence for side B in cases in which the burden of proof actually rests on side B. A common name for this is an Appeal to Ignorance. This sort of reasoning typically has the following form:

Claim X is presented by side A and the burden of proof actually rests on side B.

Side B claims that X is false because there is no proof for X.

Tom, I am not required to prove your point to be incorrect merely in the process of pointing out that you are using a logical fallacy in "proving" your point yourself.

The point is that you said you've never seen a place where metadata BlahBlahBlah. My point is that, although yes, you have relevant experience, no, saying that you do is no proof of any sort. It is irrelevant to the argument. The whole point of logical fallacies is that they are seductive but ultimately irrelevant. Arguments should stand entirely on their own merits.

The claim, implied by the page title, is "Metadata is a myth". That is the claim to be supported. Text in question, quoted above, was an attempt to buttress that claim. My experience rebuts the claim "Metadata is a myth".

Well, no, the problem is that fallacies are fallacious regardless of whether they are made in support of a hypothesis, or against, for a NullHypothesis. Your experience rebuts nothing. Your arguments may rebut everything. Even though you create your arguments based on your own understanding of your experience. That's the difference.

The HotButton? you pushed is that I feel that this unsubstantiated claim has set back progress in our industry for decades, while the nay-sayers begrudgingly admit the metadata in question back into common practice (though always cloaking it with other words to avoid acknowleging both its presence and their earlier error).

I see. Well, I'm a lifelong metadata fan, myself, and have been using XML in particular at the drop of a hat practically since I first heard the word, so I tend to be at a loss to understand the intense hostility some people have for XML. I guess I haven't heard the grandiose claims that some people say have irritated them. XML isn't perfect, but the fact that it is a metalanguage, rather than a language, is very handy. So, perhaps we're in agreement on some central issues.

I removed the reference that seems to have offended you.

Thanks.

In my view, a logical argument -- no matter how phrased -- that leads to a conclusion that excludes experience is flawed. My experience is not a rebuttal, it is a data point. Whether or not it "rebuts" the claim of the page (in whatever rigorous definition of "rebut" you prefer) is beside the point. Here's an example: 

 Fisherman A: "There's some really big bass in this here pond"
 Fisherman B: "I been fishing this pond for twenty years, and ain't never caught a bass yet"
Now, I completely agree with you that Fisherman B's response neither proves nor disproves the claim of Fisherman A. At the same time, if I want to catch bass, and I overheard the conversation, I'd choose another pond.

I'm not sure how to be more clear. You are describing a common experience: we as humans in general do not operate by pure logic in pursuit of rigorous truths, we operate by the seat of our pants under time pressure using intuitive probabilities. There may or may not be fish in that pond, but why waste time, when there are other ponds?

Yet it is still the case that Fisherman B's experience, whether you call it a data point or anything else, is completely irrelevant to the question of whether there truly are fish in the pond.

If you want to try to switch this sort of thing from being a logical fallacy into a formal method that is non-fallacious, you'll probably need to introduce Bayesian analysis and RelevanceLogic?. "In 4,000 trials, 85% of the time that Fisherman B said there were fish in the pond, his claim could be verified independently; in 72% of those trials in which Fisherman B said there were no fish in the pond, BlahBlahBlah"

Probabilistically maximizing expected utility is quite different than making a logical argument, but even at that, both are formal approaches. We all use informal approaches, including outright logical fallacies, every day, that's part of the human condition, and not to be denigrated outright, because it is part of our intelligence, yet to be duplicated in AI software. In fact, it is precisely the fact that our human techniques go wrong quite frequently that allows us to escape the bounds of Goedel's theorem; if you insist on correctness every time, then Goedel's theorem applies: the results are incomplete. Allow incorrect results, and you can get completeness...and that's the approach evolution favored.

On the other hand, it doesn't help to deny that they are indeed logical fallacies, because the point is that we have learned that everyday seat of the pants ad hoc techniques aren't really fair fighting in a debate.

In short, yes, it was a logical fallacy, whether you like it being called that or not.

I'll choose somebody else to be my fishing guide, thanks.

Smart ass. Why write anything at all?

Oh, that is a constructive response. The name-calling really cinches your case. See if this helps clarify it for you: I disagree with you. I think you're wrong, as in "mistaken", as in "incorrect".

You misunderstand: I thought that you saying "I'll choose somebody else to be my fishing guide, thanks." was unconstructive and even insulting, to the extent that it implied that I have no common sense about deciding where to fish. So responding "smart ass" seemed to me to be highly descriptive, not an off-topic epithet; it seemed to describe your response: being offensively "witty" rather than responding directly to my attempts to explain at length.

Especially since you seem to have missed my acknowledgement of the pragmatism of your position above, when I said "You are describing a common experience: we as humans in general do not operate by pure logic in pursuit of rigorous truths, we operate by the seat of our pants under time pressure using intuitive probabilities. There may or may not be fish in that pond, but why waste time, when there are other ponds?" ...which is pure agreement with you about pragmatics; I then went on to distinguish that from the question of logical argument.

So for you to now complain that I am being unconstructive just seems like the pot calling the kettle black. Look. You made a logical fallacy. You are extremely resistant to that idea, but it is a black and white fact, nor have you made any apropos argument against it...your counter-argument about fishing was a non-sequitur, which I explained, by pointing out that everyday pragmatism is not the same question as whether something is a logical fallacy.

You're obviously an intelligent person (from your writing on the wiki), the question is whether you have it in you to see that it is possible for you to engage in a logical fallacy. So far, apparently not; this is true of an unfortunately large number of intelligent people, for exactly one reason.

I don't mind when someone talks about their experience; it helps put an opinion in context. But it starts to smell when such a person starts quoting numbers ("20 years of experience", "fifteen projects", etc.), as if it is a scoring system that should determine who is smarter. -- KrisJohnson

Quite so.

Hmmm. Hoping that this won't be too far out of bounds, at what point does a burden of proof exist in a situation like this:

(A) spends several years in an endeavor using method (M) and observes or achieves result (X). (A) recounts this experience to (B), and allows that he (A) believes that (M) can reliably produce (X). (B) finds that he disagrees with (M) for whatever reason, and says, "I don't see how that can be. Prove it."

Is it now (A)'s job to prove (M) -> (X) to (B)? Or is it (B)'s job to refute (M)?

And, yes, this is from real life. -- GarryHamilton

It depends on the context as to whether proof is in order or not, but the overall point is that experience is not a kind of proof. Assuming that it is, without determining causality, is the post hoc ergo propter hoc fallacy.

Pragmatically, if you say that your experience is that it reliably gives you that result, and if I have no experience one way or the other, then I should listen carefully to your suggestion.

But should I bet my life on your experience? No; I have yet to understand the causal mechanism that guarantees the reliability of your approach. Maybe it was just luck every time, and we're relying only on coincidence of causally unrelated factors.

Nor is this an outre suggestion. It is exclusively what happens when people successful in business publish HowTo books to explain their success and how to duplicate it. This subject has been studied. The authors are invariably wrong; they do not in fact understand the "how" of their own success, and therefore cannot reliably pass on their experience. They recommend a pattern of things that (sometimes!) they did in fact do, but these turn out to be coincidental rather than causal to their success.

Humans primarily use reasoning processes that are separate from formal logic, and that's for very good reasons. But we have also learned that formal logic is very helpful in certain circumstances. And that it is valuable to distinguish the two.

So I can usefully say "I don't really understand General Relativity, but if it was good enough for a smart guy like Einstein, then it's good enough for me". That really is useful sometimes. But I should understand that it is nonetheless a logical fallacy. I haven't proven anything by that appeal to authority. To prove something, I should go off and study some differential geometry and tensor calculus and tackle the subject head on.

Fisherman A: "I been fishing this lake for twenty years, and I get keepers every time I go out"
Fisherman B: "I been fishing this lake for twenty years, and ain't never caught a bass yet"
Fisherman C: "In 4,000 trials, 85% of the time that Fisherman A said there were fish in the lake, his claim could be verified independently; in 72% of those trials in which Fisherman B said there no fish in the lake, his claims could be verified independently. Using Bayesian analysis and Relevance Logic, I can prove that..."

I just rented a lakefront cottage for the summer, and I'm looking for a guide. Do I choose A, B or C? You guys can argue about logical fallacies all you want (I took four years of latin too), and it won't help you catch fish or choose guides. I'm not trying to be a "smart ass", nor am I trying to prove or disprove anything. I'm trying to catch fish. Or, as in MythOfMetadata, attempting to set right the kind of off-hand comments that often get thrown around and become "fact" by mere force of repetition.

Programmer A: If that system call returns a null pointer, your program will crash.
Programmer B: In 20 years, I've never seen that system call return a 

 Segmentation fault (core dumped)
The point is that, yes, this is a reasonable approach to picking a fishing hole/guide, but not a reasonable approach to a technical argument. Different domains. For one thing, in the first case, it's just about making your own personal choice. In the second, it's an attempt to persuade others of a truth, rather than of a desirable course of action. Something can be desirable to do for any number of reasons outside of pure logic, such as emotion or aesthetics. Truth, on the other hand...

And so the wheel turns round and round. We have no "theory" of, to be specific, metadata. There is no theoretical basis for virtually any of our "software engineering" that corresponds to the theoretical foundations of, say, electrical engineering (the discipline in which I received formal training). We are, therefore, a very long way from being able to depend on "truth" to resolve disagreements. Instead, we are forced to rely on the next best thing -- first-hand and anectodal experience. In a discussion like the one that set the context for this extended exchange, the closest we can come to "truth" is several practitioners describing and comparing their experiences.

A practitioner will occasionally repeat a piece of second-hand "common knowledge" or "conventional wisdom" as if it were "truth" -- for example, "There are places where [having the metadata encoded with the same mechanism as the data itself] works, and places where it doesn't." It is perfectly reasonable for another practitioner, who has never, in an extended period of practice, experienced a situation where it didn't work, to say so. It is also perfectly reasonable for that practitioner to ask the other to provide even one example to support his statement. Neither party is attempting a "proof", in anything approaching a formally demonstrable manner, because neither party (nor any others to my knowledge) have access to the theoretical foundation that would make such proof possible. A third, fourth, or nth party could, at any time, jump in and say "here is a situation where it doesn't work" -- and significantly advance the conversation. On the other hand, a contribution that says "pardon me for interrupting, but you've just committed a logical fallacy" is not helpful. It adds neither theory to buttress one side or the other, nor data that might be helpful in discovering such a theory.

We may be suffering, here, from a failure to communicate. You seem intent on explaining what a logical fallacy is. I thought I had already written that I believe I understand what it is. I have already agreed, many paragraphs and days ago, that I probably was. I don't have the sense that you appreciate that the original conversation was rather more like choosing a fishing guide and rather less like demonstrating "truth".

Sometimes logical reasoning simply won't get us from where we are to a "truth". Sometimes this is because we lack the necessary data. Sometimes it's because we lack the necessary theory. Sometimes it's because several outcomes share a common state and are therefore indistinguishable. My engineering training, supported by my mentors during the early years of my career, strongly encourages me to use whatever data I have available to me, make the best guess I can make, and move on. A hugely important part of that informal, chaotic, unscientific, illogical process is to pay attention to the experience of others. Sometimes that shared experience is the only "truth" we're going to get in our lifetimes.

Ok, well, compare these two sentences:

What precisely is the value of the former over the latter in any sort of discussion? Would you still be so fond of the former phrasing if you knew you were talking to exactly 5 people, each of whom had 25 years of Smalltalk experience? It seems to me that in that situation, you would automatically stop saying "in twenty years..." -- AnonymousDonor

The point of saying "In N years of doing X I've never observed Y" is that it gives your listeners some evidence that Y doesn't happen when doing X. There are two quite separate reasons for telling them what N is. The first is that (assuming your observation is good) the larger N is, the better-founded the inference that Y never (or very rarely) happens is. The second is that the larger N is, the more likely you are to be sufficiently expert in X to recognize Y when it comes along.

So, suppose I'm talking to a crowd of more experienced people. There are two different scenarios.

Scenario 1: They think Y does happen sometimes; they've seen it, or think they have. I'm hoping to persuade them. In this scenario, I'm onto a loser when talking to more experienced people unless I can show them reason to think I know more than my expertise suggests (e.g., I've made an intensive study of Y and found various circumstances in which it seems to happen but doesn't really). So I'm hoping my comments will be evaluated on the basis of something other than just the value of N. I should probably tell them what N is, but I shouldn't expect them to be impressed by it.

Scenario 2: They haven't (so far as I know) given any thought to whether Y happens. I'm hoping to inform them. In this scenario, comparing my N with theirs is irrelevant, at least until such time as they start thinking about whether they've ever seen Y. So, if my N is pretty big, then by all means I should tell them and expect them to pay more attention than if I didn't tell them.

If the people I'm talking to already know what my N is, then of course I shouldn't bother telling them unless for some reason I think they've forgotten. (In which case, depending on circumstances, it might be appropriate to say either "Look, I've been doing this for N years; I do have some clue what I'm talking about" or "Remember that I've only been doing this for N years; please tell me if your experience is different".) This applies regardless of whether they have (individually or aggregatedly) more experience than me, or less.

EditText of this page (last edited May 15, 2009) or FindPage with title or text search