Absence Of Evidence Is Not Evidence Of Absence
Absence Of Evidence Is Not Evidence Of Absence
The above is a favorite (and sometimes annoying ;-) catch phrase often repeated by many, many, many people (apparently including BradAppleton, for whatever that may be worth). Simply put, it means that if we don't know that something exists, it doesn't mean that it doesn't; It only means we don't know one way or the other, we just haven't been made aware of it yet so it's not part of our knowledge.
No, the paraphrase is utterly wrong -- evidence isn't knowledge or proof, not even remotely! We can have evidence of things that aren't true. Why is it so hard for people to grasp this obvious fact? And absence of evidence can pertain to any empirical claim, it doesn't have to be an existence claim. If you're going to set up a topic for discussion, just state it as is, don't seed it with your own harebrained misunderstandings.
If you look for "X" and don't find it, does that prove that there is no "X"? No.
But the more you look in places where X "ought to be" in ways and at times that X "should be likely to be there," the more confidence you can have that there is no "X".
Like, I flip a coin 5 times and get "heads" every time (no "tails"). Is it a "heads only" coin? (...having a "head" printed on both sides.) Well, say I flip it 500 times and get "heads" every time. Now I'm pretty convinced that it is "heads only."
Or you could just look at both sides of the coin. OTOH, if you find no evidence of a "tails" on either side, does that mean the tails isn't there? Hmm...
You could look at both sides, but that's a matter of obtaining evidence, which isn't what this page is about. However, if tails is not on either side then yes, tails isn't there ... duh.
Certainly - but you have some evidence here. You know that there is a "tails" on the other side of the coin, that does not come up when you flip it. And if you've done a good job of looking in likely places where you might find X, but still don't find it, there is no question that increases one's confidence in it's scarcity.
On the other hand, if I am the only person doing the coin flipping in the above example (and especially if I flip it with same hand every time in a similar fashion) then from the information provided, it's certainly quite possible that the problem is not with the coin, but with the way I'm flipping it time after time after time!
Unless I've been switching hands or had several others flip the coin as well with the same results, I'm dismissing the possibility that the problem may be me and not the coin. And in doing so, I've committed precisely the kind of error in judgment that is addressed by the saying AbsenceOfEvidenceIsNotEvidenceOfAbsence. I didn't consider the problem might be elsewhere, so I jumped to the conclusion it was the thing and not me.
Some people do the same with a great many things. "I've never successfully used (single or multiple) inheritance without always running into problems, therefore the problem must be that inheritance is bad (and not simply that I used it badly), despite the fact that others talk of using it successfully. They must just be fooling themselves." Or replace "inheritance" with "O-O" or "C++" or "pair programming" or "parallel development" or anything else that is the subject of many popular raging debates on both sides of the continuum.
The problem here, as with so much of the discussion, is with the word MUST. But evidence is not about proof or certainty or necessity, it is simply something that can contribute to belief. Failure to have ever used multiple inheritance successfully is EVIDENCE that it's bad, but it is in no way conclusive. This is the fundamental mistake that one finds throughout discourse about evidence -- the belief or assumption that there can only be evidence for claims that are true. But there is in fact a vast amount of evidence for claims that are in fact false.
Every quote or saying has a context that goes with it, and AbsenceOfEvidenceIsNotEvidenceOfAbsence is certainly no exception. It has to do with the flip-side of the all too frequently recurring case of assuming facts not in evidence. It's no less safe to assume those non-evident facts are true than it is to assume they are false. It has to do with the cases where people weren't aware of some possibility or piece of information and therefore jumped to the conclusion that it wasn't there. If we don't see or understand something, let's at least consider that maybe our vision or thinking needs some adjusting before rushing to declare it completely absent or unworkable.
It's all too easy to jump to that wrong conclusion when it's a case of blaming something other ourselves. It's so much easier to point the finger at "O-O" or C++ or inheritance or parallel development, etc., than it is to point the finger in the other direction (just remember when you point your finger the other way, all the other fingers on that hand are pointing back at you ;-)
The context here has to do with lack of awareness. When people dismiss that possibility simply because they weren't aware of it, or haven't made the attempt to explore it, or haven't looked beyond their own personal usage/viewpoint -- then they are all too frequently jumping to conclusions based on lack of information rather than the presence of it. It may be that the conclusion was correct after all, but that doesn't make it any less premature at that time, before it was adequately explored.
A lot of the comments here are just plain wrong and wrongheaded, feeble and flawed attempts to make a false quote not false. But the quote IS false, an absence of evidence IS evidence of absence, by the proper understanding of evidence as being an observation that lends support to a proposition. The only case in which absence of evidence is not evidence of absence is when no attempt whatsoever has been made to obtain evidence ... that's not merely absence of evidence, it's absence of investigation.
Psychological experiments by Ward Edwards and others have shown that humans are generally very conservative when cumulating evidence toward probabilistic results. For example, they generally estimate the probability that the coin is biased as much lower than Bayes' Theorem would indicate.
This is not evidence, of course, that when a person does draw a correct conclusion before we're ready, that they are in fact merely closer to veridical estimation than we are. However, Edwards' results do call into question our objections when there is some evidence, but we aren't personally ready to generalize.
On the other hand, I'm not aware of any research addressing people's ability to instantly [over-]generalize from zero or one cases. We're a weird species ... -- RonJeffries
An infinite number of lines go through one point, two points determine a line, three points landing on the same line is amazing. I can't imagine why this applies to people, but I notice people (myself included) immediately drawing a conclusion from a sample population of 1 - all different conclusions, of course. If they see two samples, then they announce the conclusion as "certain" or obvious. Three samples make it irrevocable. -- Alistair
For a skewed but related notion, see CoinTossingBof.
"1000 to 1 shots come up 9 times out of 10" - From TerryPratchett's DiscWorld novels.
It's "1,000,000 to 1 shots". And it has to be one million. If it's more likely than that, do something to make it worse (like blindfold yourself and stand on one leg when shooting the dragon - see "Guards, guards").
The only problem is, it's not true. In fact, million-to-one chances come up ten times out of ten. The trick is knowing which million-to-one chance is going to come up...
This page is full of StrawMan arguments, so I'm listing it in the FallaciousArguments.
Which arguments? How about the misstatement at the top, that AOEINEOA "means that if we don't know that something exists, it doesn't mean that it doesn't" -- it doesn't mean that at all. The latter is true, but AOEINEOA is false.
The "simply put" is simply mistaken; it doesn't mean that at all. The paraphrase is certainly true, and expresses the fallaciousness of argumentum ad ignorantiam -- absence of a proof is not proof of absence. But AOEINEOA says something different entirely; it is quite wrong, and indicates a misunderstanding of evidence and empirical science. Absence of evidence certainly is no proof of absence -- there aren't any empirical proofs in the mathematical sense -- but it is evidence of absence. That is, it does not falsify absence, and in fact gives reason to suspect absence. Just how good evidence of absence it is depends on how hard evidence of presence was sought. If we try really really hard to find evidence of life on Mars, using every technique we can think of, and yet we fail to do so, this is strong evidence that there is no life on Mars.
It is evidence of absence, true, but very, very weak evidence. To become a proof, one must exhaustively search all possible locations and establish absence in each. This is only rarely feasible, though it does happen. In those cases, AOEINEOA is a fallacy. Far more often, though, absence is established in some vanishing fraction of the possible locations of some phenomenon and that is held up as a proof. So the vast majority of times this saying is trotted out, the point holds. Compare "evidence of existence", where one good piece of evidence establishes the point beyond all doubt.
This misses 99% of the point I made, as it talks repeatedly about proof, which is not the subject at hand. A fine example is William Safire's claim that absence of evidence of WMD's in Iraq isn't evidence of absence -- he's just plain wrong, and the evidence isn't "very very weak", in fact it's very strong, because of the effort to find evidence and the eagerness of the searchers to find it. That I can't find any elephants in my living room is overwhelming evidence that there are none because they ought to be easy to find.
[Not quite the same. You can argue that you checked all possible locations of elephants in your living room, with no exceptions whatsoever, due to the size ratio. The most you can say about a country is that (for the sake of the argument) "all reasonable locations have been checked", or some such. Countries are too big to literally eliminate 100% of the possible locations, unlike your living room.]
I don't want to argue that I checked all possible locations because that's not my point. We're talking about evidence, not proof. Talk about "all", "100%", and "possible" is always talk about proof. Evidence is simply something that contributes to belief. Absence of positive evidence of elephants is a reason to believe there are no elephants, because of the obviousness of elephants. In the absence of reason to believe there are elephants in your living room, I have reason to believe there are no elephants there.
[And even in your living room, it's not the absence of reports of elephants that is evidence of something, it's the positive fact that you reported that you searched and found nothing that provides evidence. If on the other hand we just said "we haven't heard anything at all about elephants in that living room (but we don't know if anyone even checked)", that would be true "absence of evidence".]
This is just naysaying. Absence of reports of elephants is evidence of something, because of the meaning of the word evidence. I already made this point -- that this is a matter of empirical science; this is a fundamental issue in philosophy of science, and is discussed in any good text on the subject. And I already wrote "Just how good evidence of absence it is depends on how hard evidence of presence was sought", so you're repeating what I said with your talk of reports, but with less clarity. To add to that, it isn't just how hard evidence of presence was sought, but all sorts of other factors concerning the nature elephants, living rooms, or whatever. Absence of evidence of men with 25 penises is evidence of absence of men with 25 penises, even if no one has ever thought to look for one -- because if there were one, we would have been likely to hear of it. Evidence, unlike proof, is statistical in nature, and it is cognitive and subjective -- "a thing or things helpful in forming a conclusion or judgment", says the dictionary. Unfortunately your discussion, and much of the rest of the discussion of this subject, blithely ignores the meaning of the word. And there's no excuse for it, as I already wrote "[absence of evidence] does not falsify absence, and in fact gives reason to suspect absence". Reason to suspect absence is evidence of absence, by the meaning of the word evidence. People seem to want the notion of evidence to be stronger than it is. They don't like the fact that there can be evidence for claims that aren't true. But absence of evidence for P is evidence of (reason to believe) the absence of P, whatever P is, which is why we generally disbelieve things for which there is no evidence, rather than being purely agnostic about them.
The case for evidence of absence depends upon whether or not evidence of any kind exists. If none exists, then absence of evidence is neither evidence of absence or of existence.
A good example proving this statement is the usual discussion I get into around NonFunctionalRequirements in software specifications. Many people have no idea what an NFR is, and even when they know what it is, have no idea how to elicit them. However, NFRs always exist and will usually be discovered when it's most expensive and difficult to meet them. -- PhilStubbington
No example can prove this statement because a) examples can only prove existentially qualified statements and b) because this statement is false. The above assertion is yet another harebrained contribution that makes this page relatively useless. This is a problem with this wiki; unlike Wikipedia, there is no way to filter the content for quality or accuracy.
Absence of evidence certainly is evidence of absence. It's simply a matter of deciding whether it's very weak evidence or something stronger. Clearly it is not proof. -- DavidPlumpton
Absence of evidence is only "evidence" of absence if one has gone about making a reasonable attempt to gather such evidence (most likely beyond simply their own immediately recollectable experience). In that case, one has actually made the effort to look and found no evidence -- they have found evidence of the absence of evidence. Otherwise, there is no known absence of evidence, only ignorance of evidence or ignorance of the absence of evidence.
The statement at the top of this page is talking about the cases where someone simply "off the cuff" says something like "well, I've never seen it" or "I haven't seen it in my experience." That's a far cry different from making a concerted effort across numerous sources too look for it and then coming up empty handed. It's little more than an illegitimate excuse to keep a closed-mind closed.
No, it ISN'T talking about those cases, and your assertion is just a feeble attempt to rescue a false statement, or an inability to accept that it might be false. The concrete case that Rumsfeld was referring to was the presence of WMD's in Irag, for which absence of evidence most certainly WAS evidence of absence.
The real problem being addressed by the (perhaps oxymoronic, or just plain moronic :-) phrase at the top of this page is that IgnoranceOfEvidenceIsNotAbsenceOfEvidence. The phrase "AbsenceOfEvidenceIsNotEvidenceOfAbsence" may in fact be logically incorrect according to the rules of empirical science. However, its intended meaning (from both the original source and others on this Wiki that use it) is indeed what the top of this page says: Lack of knowledge/experience/awareness of something (which is a polite way of saying "ignorance") does not imply lack of existness/correctness. Ignorance of evidence is not the same as the "absence" of evidence, and therefore Ignorance of evidence is not evidence of absence.
Your assertions are wrong on all points. The claim stated is the claim stated, not some other claim. And your reference to implication is utterly wrongheaded in the same way that the paraphrase at the top of the page is wrongheaded -- EVIDENCE doesn't imply anything, it's too weak for that. It only LENDS CREDENCE.
Even if absence of evidence really is evidence of absence, ignorance of evidence is neither one of those things and shouldn't be mistaken to imply either one. Ignorance of evidence is evidence of ignorance (no, more and no less). -- BradAppleton
However, absence of evidence is certainly not evidence of existence. If one wants to prove something, he must provide some justification.
This is incoherent nonsense, riddled with errors, and has nothing to do with the issue at hand.
The straw man arguments point above is the best comment here. The Absence of Evidence argument is NOT argument FOR a phenomenon per se, it only refutes the argument the phenomenon cannot exist (unless you can circumscribe all possible occurrences, which is extremely rare). Occam's Razor is useful but not infallible. Buying a lottery ticket a hundred thousand times does not prove a lottery cannot be won, it at best proves the statistical unlikelihood. Annoying to some, but a critical point in fields in which evidence may be either hard to come by or even methodically suppressed.
This page is quote mining Carl Sagan. Carl Sagan didn't support that proposition, he criticized it. Here's those words in context:
You are confused. Sagan offers the phrase as a summary criticism of something he rejects. He isn't criticizing the proposition, he is asserting it AS a criticism.
Let H be a hypothesis that something is true, and E be an observation of evidence supporting H. The probability of H given E is P(H|E), and the probability of H given not-E is P(H|~E).
If P(H|E) > P(H) then conversely P(H|~E) < P(H). By Bayes' theorem, E increases our estimate of P(H) and ~E lowers our estimate of P(H).
Absence of evidence is indeed evidence of absence, always, though that evidence may be strong or very weak. How strong it is depends on how likely E is given H. If you don't search for evidence and don't observe any, that's obviously much weaker than if you had searched.
I can accept the Bayesian formulation above, but there is a stronger form which may be more correct and less dependent on the choice of lower bound of P(H):
Absence of evidence is not evidence at all. This results in no conclusion at all.
But that's simply false. In the concrete case where Rumsfeld used the phrase, the absence of evidence of WMD's in Iraq was strong evidence that there were none.
If P(H) is chosen low enough then for some bound epsilon they are the same. But some may not pick P(H) well, or they may pick it differently. Is P(god exists) high or low -- discuss ( but not here make a new page :) )
Just chiming in my two cents. Not trying to prove or disprove anyone. Simply trying to understand better. What is wrong with the following example: early humans did not have evidence/proof that Earth is a spheroid. Hence they lived on a non-spheroid planet? Say flat piece of rock? Modern humans did find evidence/proof that Earth is a spheroid. Hence we believe/know that we live on a round planet. Hence sometime in the past, say, 1000 years, Earth switched from flat piece of rock to a spheroid? What about some primitive tribes nowadays that have no evidence/proof to what planet are they on. Does it mean we share with them a flat and round planet at the same time? Help!!!!!
People WITH a brain. That one is for you. You secretly plant an Easter egg in John's house. Then you ask him "John can you find an Easter egg in your house?" John looks for Easter egg in his house and can't find it. John comes out and says "I don't have any evidence that the Easter egg is in the house." Question. What does that mean? If absence of evidence is evidence of absence then Easter egg is not in the house, as far as John is concerned? But you, and other witnesses, do know that Easter egg IS in the house. So what is it?
In any case, if you're going to complain about nonsense and crap, you probably should stop contributing to it by asking FalseDichotomy questions. Go learn about hidden Markov models and Kalman filters and Bayesian probability if you're interested in some cognitive and computational aspects of working with limited data in an open world.
{I protest the rudeness of this response.}
Please don't protest; just edit the text to convey the same idea with less abrasion. Most Wikizens don't mind that kind of editing of their contributions. When we do want our stuff left as is we'll sign it.
{What it is, is a confidence level. If John has searched 100sqft of his 1200sqft house then he has a low confidence level if he says that he doesn't think there is an easter egg in the house. If he has looked at 1190sqft then he has a high confidence level. CERN do this all the time in narrowing down where a particle might be in terms of energy levels. Indeed, the whole of science could be regarded as the process of trying to think of places we've not looked for easter eggs yet, coupled with a growing accumulation of knowledge as to where they are not.}
EditText of this page
(last edited December 26, 2014)
or FindPage with title or text search