Carl Sagans Baloney Detection Kit

Also http://www.xenu.net/archive/baloney_detection.html and http://www.uiowa.edu/~anthro/webcourse/lost/sagan.htm

What skeptical thinking boils down to is the means to construct, and to understand, a reasoned argument and to recognize a fallacious or fraudulent argument. The question is not whether we like the conclusion that emerges out of a train of reasoning, but whether the conclusion follows from the premise or starting point and whether that premise is true.

Use the following tools for testing and detecting fallacious or fraudulent arguments:

Tools

Detection list of some of the FallaciousArguments used to support or defend

Clever chap this CarlSagan was. Being able to recognize fallacious arguments and all.

Take, for instance, the fallacious argument in first paragraph above. He would have spotted that in a jiffy. I bet.

The question is not whether we like the conclusion that emerges out of a train of reasoning [...]

Often this is or at least should be the question. If I conclude from a train of reasoning that overcrowding in large cities could be solved by poisoning the water supply, I shouldn't feel content that my reasoning is sound. The fact that I don't like the conclusion is itself reason enough (catch that? reason enough) for me to continue along some other train of thought. -- ChrisSteinbach

You've confused "is" statements with "ought" statements. -- AnonymousCoward

While reading NaturalisticFallacy it dawned on me what Mr. Coward is trying to tell me here.

He means the detection kit applies only to descriptive and not prescriptive statements. Now, even the simplest argument requires both elements. How then, could I confuse them? -- ChrisSteinbach

It is correct to say that overcrowding can be solved by poisoning. But this does not automatically imply that in order to solve the problem of overcrowding, one must poison the water supply. The fallacy here: "ExcludedMiddle".

However, I do find that CarlSagansBaloneyDetectionKit is incomplete in a few ways. For one thing, it only tests whether an argument follows from a premise, but does not test whether a premise may well be false (and can be shown to be false). -- TkChia

You are right of course. What should also be said is that restrictions and rules applied to reasoning should be qualified by (possibly drastic) restrictions in their domain of application. It would be interesting to see if anyone can find a domain where the baloney detection kit can relentlessly be applied. -- ChrisSteinbach

Wouldn't it be easier to enumerate the domains where it can't be applied? e.g. religious presuppositions, art/aesthetics, what others? -- MarkTilley

The way some people frame religious presuppositions, one can actually derive contradictions from them. :-) I think the Baloney Detection Kit can be applied in any domain which works according to some sort of rules, and that includes about all of real life. However, to falsify certain premises, one may need to use ad-hoc techniques, such as empirical counter-examples. -- TkChia

It occurs to me that I may have read too much into this detection kit. If CarlSagan did not intend that it should be applied rigorously, MarkTilley and TkChia are right to think that it has a much wider application than I indicated previously.

This is not the most natural interpretation. The first section, marked tools, is clearly additive to reasoning. The list of items comprising the second section may be construed as restrictive to reasoning. However, running every idea and argument through this gauntlet would put a stop to progress. New ideas are often inconsistent in their beginnings. Observational selection and 'statistics of small numbers' are the norm during research. Meaningless questions become meaningful only after certain arguments have been accepted.

CarlSagan was a big supporter of science and expended much energy popularizing it. It would not have escaped his attention that the knowledge, methods, funds and equipment required to settle serious scientific arguments are not available to the majority. Would he advocate methods that would instantly categorize science as baloney? I was off track, to say the least, in thinking so. -- ChrisSteinbach

I think the onus is on the researcher to refine the exposition of his ideas, so that they move from the "baloney" stage to the "science" stage. Many people simply present their initial ideas as the One Final Truth, and this is the problem.

Regarding the difficulty of settling scientific arguments with limited resources, this is indeed a problem, but it can be ameliorated somewhat. If an idea has been verified many times over by skeptical parties, preferably using different techniques, then it is very possible that the idea is true. (Indeed, reportedly StephenHawking's ideas were subject to this treatment.) Of course, nothing beats personal observation... -- TkChia

I tend to agree with most of what you say.

the difficulty of settling scientific arguments with limited resources [...] can be ameliorated somewhat.

I agree on the precondition that 'skeptical parties' are not given a privileged or authoritative position. I assume you are referring to trusted members of the community or group to be convinced when you say this.

nothing beats personal observation.

Here I disagree. Or maybe I don't. I don't see observation as something simple or unambiguous. Before Galileo, the observation that an object dropped from a high tower falls vertically was considered repeatable confirmation that the earth doesn't move. Afterwards, the same the observation lost it power to persuade (see http://www.helsinki.fi/~kieseppa/change07.pdf). Other examples make the distinction between observation and theory even more complex (see Lakatos on brownian motion http://www.philosophy.ru/edu/ref/sci/lakatos.html).

None of this is decisive. As with many other gross dichotomies (fact/fiction, reality/appearance etc.), observation and theory can be given clearer meanings (e.g. by bringing observation closer to perception) for special purposes, convenience or even as a general preference. But who has the upper hand is not at all clear-cut. -- ChrisSteinbach

I think I get your idea. A good example I myself can think of is the wave-particle duality: traditionally people thought of electrons as "particles" and light as "waves", until experiments showed that electrons can have wave-like properties, and light can have particle-like properties. Agreed, observation isn't clear-cut, but I think it's still the best way so far to resolve disputes. -- TkChia

I was arguing from a different angle. I'm not sure I contend (as Popper, Lakatos and Kuhn do) that observation is theory-laden. Nevertheless, in the example above, brownian motion was only considered falsifying evidence after alternative atomic theories were introduced. Before that, it was not considered a refutation of the second law. For thirty years, brownian motion remained a curiosity and not an anomaly. It took extra theoretical input to show that there even was a dispute. The raw observational evidence had already been considered. -- ChrisSteinbach

Earlier on, I asserted that CarlSagansBaloneyDetectionKit doesn't test whether a premise is false. I found that this isn't really the case: premises falsifiable by counter-examples can be considered to fall under "Suppressed evidence".

I also asserted that the kit can be applied in any domain which works according to some sort of rules. I think the converse is also true: namely, it won't work well on degenerate 'belief systems' which seem to follow no rules, such as the notorious Scientology. And I think that's very scary. -- TkChia

Hmmmm. Degenerate 'belief systems'. Perhaps you have an example of a 'belief system' that's not degenerate? Maybe something you've studied? I have studied a dozen of what you would call 'belief systems' - including Scientology - and I find that baloney detection works pretty well in those contexts as well. If a system doesn't hang together, then it doesn't hang together. I've managed to employ Scientology principles to good advantage. I've also employed Christian principles to good advantage. Same with Buddhism. Wisdom is where you find it. I find that, on the whole, GravityWorks. It is completely unforgiving and doesn't care whether you can logically conclude it exists. Is it possible that you are arguing from the perspective of Argument from "authority" or maybe Argument from adverse consequences? I submit that baloney detection works just fine in these contexts. I'm not suggesting that you would particularly want to launch an argument with someone who fervently cherishes a set of beliefs over the baloney you've detected in their system, but that doesn't mean you can't detect the baloney. -- GarryHamilton

You're right, I guess I've not examined Scientology hard enough. -- TkChia

Scientology fails the DetectionKit? test at the "suppressed evidence or half-truths" stage. From court testimony, we know that the "religion" was started with a few guys sitting around a desk figuring out how to turn a fiction book into a way to milk rich people out of their money (suppressed evidence). A foundational assumption of the faith is that Man is good (half-truth). -- BrucePennington

Gary, I have done a great deal of research on 'belief systems' and found the Judeo/Christian faith to be the only un-degenerate (is that a word?) belief system, as all the others are either mutations or spin-offs of the original, or cleverly crafted lies (like Scientology). -- BrucePennington

One person's cleverly crafted half-truths are another person's inerrant, infallible scripture.

While watching the series "Cosmos" by Dr Carl Sagan, I was particularly astounded by something that he said near the end of the series. While the viewer watched fast motion views of the bustling city of New York, Sagan remarked:

If you were an observer from an alien world, you would have noticed that something very complicated has been happening here over the last few thousand years. It might take you a while to figure out the details, but you would recognize by its complexity, unmistakable evidence for intelligent life. On closer scrutiny you might even be able to recognize individual intelligent beings. The evolution of the city is due to their conscious activity, millions of human beings working more or less together to preserve the city, to reconstruct it, and to change it.

So, clearly the good doctor has the capacity to recognize the necessity of intelligent life to explain the complexity of New York city, but his religion, evolutionism, will not permit him to accept the same obvious truth about living things. -- BillZimmerly

Boy, that's some densely packed baloney right there, but let's cut to the gist: How does the emergence of cities from individual organisms support any view but evolution? To take the creationist standpoint, you'd have to posit that there was some unseen and unknown force external to the city that directed it into being, ignoring how we can provably demonstrate down to choices concerning the height of the lampposts and the width of the streets how the city came about as a function of its individual creators. Given that you're more interested in casting semi-clever aspersions on science to support the wholly unprovable and even unknowable hypothesis of creationism, I don't expect this argument to actually take hold, of course. Can't hurt to try.

An interesting observation, however what is obvious to one person is not necessarily obvious to another. An interesting question: Does this observation pass the tests above?

The book this material comes from ISBN 0345409469 is very much worth reading. I would not characterize Sagan as an "Evolutionist"; perhaps a "Rational Agnostic". If Sagan had religious beliefs, it was in the power of scientific thought.

Evolution is not a religion, though some treat it as one. Atheism is a religion, though some treat it as if it is not.

Evolution is an observable process, that neither requires, nor precludes, the presence of a higher intelligence.

There is no reason to believe that the process of creation did not follow an evolutionary path, nor is there a reason to believe that the process of evolution occurs completely without "divine intervention".

-- DirckBlaskey

Evolution is an observable process . . .

Do you mean to say we have observed evolution, or maybe just natural selection? Or if evolution occurred, we would (be able to) observe it? -- MarkTilley Yes - evolution has been observed many times. See talk.orgins FAQ for details.

Does this observation pass the tests above? CarlSagan happens to have phrased his words very carefully here: he says that the evolution of New York City is due to intelligent life, however this says nothing about whether the creation of New York City was due to intelligent life. Sneaky. -- TkChia

I believe that there is reason to believe that the process of creation did not follow an evolutionary path, Dirck, and here is my reasoning why...

1. Evolution requires the ability to reproduce as a PRE-CONDITION. So? Creationism requires Superman (or some such) as a pre-condition.

2. This logically excludes reproductive capability as being a PRODUCT of evolution. That is correct - the earliest reproductive capability was more akin to simple crystallization than to the capabilities of current living cells. So what?

3. Finally, in all OBSERVABLE cases of reproduction - cellular mitosis or whatnot - the complexity involved in the machinery of reproduction far exceeds anything that man has ever created. (And certainly anything that has been created by random chance!) That depends on what you allow to be called "reproduction".

How could this have "come about" in any way other than having been designed by an engineering mind that was capable enough for the task? - BillZimmerly

That's an ArgumentFromIncredulity - I don't know exactly how a modern cell reproduces, but there is nothing apparently supernatural in the process.

There's one way. Imagine a primordial universe, where electrons, protons and other elementary particles are distributed randomly, and move about by Brownian motion. Is it not conceivable that after billions of years, some of these particles will happen to group themselves into meaningful combinations, and form the beginnings of living organisms? And is it not conceivable that some of these meaningful combinations will also happen to be able to draw from the particles and energy surrounding them, and create copies of themselves, while other meaningful combinations die out? Notice that this theory explains both evolution and reproduction.

If such a process has not been successfully reproduced by mankind, it may be just because mankind has not been around for the billions of years required to produce significant results.

The Chinese have long had a similar theory: see http://www.edepot.com/taocosmology.html.

Of course, all these are just theories. I doubt anyone will be able to find conclusive evidence which points to a single possibility of how reproduction and evolution started. -- TkChia

There is no reason to believe that the process of creation did not follow an evolutionary path.

'Creation' may have occurred in setting up the initial conditions from which an evolutionary path followed. Also, natural selection requires 'random mutations' from which improvements may be drawn. It seems to me that is a hole large enough for a 'divine entity' to drive a truck through.

There is no evidence, either for or against, this interpretation.

'Absence of evidence is not evidence of absence.'

-- DirckBlaskey

1. Evolution requires the ability to reproduce as a PRE-CONDITION. - Not quite. Consider this: any entity that has the ability to reproduce itself, will rapidly overwhelm those that can't. Whether you call that evolution is a moot point (what exactly is evolving?), it's certainly survival of the fittest.

3. Finally, in all OBSERVABLE cases of reproduction - cellular mitosis or whatnot - the complexity involved in the machinery of reproduction far exceeds anything that man has every created. - Just because the only SURVIVING cases of reproduction are complex, does not mean that things didn't start off being pretty crude. It just means that that complex reproductive mechanisms are vastly more successful than simple ones.

-- StephenHutchinson

I'm glad to see that everyone's baloney detectors are at peak efficiency on this page... -- PhlIp

Sniff Sniff is that Irony I smell?

Maybe it's time for a CarlSaganVsEvolution page.

What's to debunk? Evolution is a model, just like the statement "atoms are a cloud of light negative electrons orbiting a tiny heavy positive nucleus" is a model. Use it where it's useful. -- PhlIp

There are good models, and bad models... -- TkChia

I think PhlIp makes a mistake appealing to instrumentalism/pragmatism here. The evolutionary model has been most 'useful' in its realistic interpretation. Yes, it does have other applications (e.g. GeneticProgramming). It may also have some predictive power I am unaware of. Still, an instrumentalist interpretation does not show evolution in its best light and it is hardly ever presented this way. I can understand someone taking this stance to avoid confrontation with creationists. But when you are not talking to a brick wall, evolution seems capable of standing on its own two legs.

I find myself partly agreeing with PhlIp. It really is just a model. It can (and probably will) be adjusted and corrected to remove anomalies. Or it may be replaced by another model. However, what form that model might take (in terms of predictive ability, empirical content, explanatory power, simplicity and elegance etc.), and who will find it most compelling (realists, instrumentalists or creationists) is impossible to judge in advance. -- ChrisSteinbach

If it's a model, don't treat evolution as religion. The latter is not a model; no conflict exists. -- PhlIp

Camelot! Camelot!! Camelot!!! It's only a model! Shhhhh!! It seems to me that there's very little admission that evolution is only a model and lot's of shushing to alternative models. -- MarkTilley

This is true, but what conclusion should we draw from this? That scientists are not in favour of theoretical pluralism? I don't think so. Seductive theories and arguments tend to be stubborn. Moving away from them takes a special effort without any guarantee that it will be worth it. This attitude may also be necessary for scientific progress. Consider this quote taken from DisciplineAndDogma:

I'll often bail out on an idea before it's had a fair chance, because I'm convinced prematurely that it won't work, or more commonly, because it didn't turn out as easy to implement as I'd hoped. I consider this a flaw in judgment, due to a lack of discipline on my part. The problem is, there's no negative reward for this behavior, except time wasted. On the other hand, if you hold onto an idea too long, you risk appearing as though you have trouble adapting to new things [... -- WaldenMathews]

You could, of course, argue that evolution has had 'a fair chance'. -- ChrisSteinbach

Excerpt from TheDemonHauntedWorld

Chapter 19 No Such Thing as a Dumb Question:

I meet many people offended by evolution, who passionately prefer to be the personal handicraft of God than to arise by blind physical and chemical forces over aeons from slime. They also tend to be less than assiduous in exposing themselves to the evidence. Evidence has little to do with it: what they wish to be true, they believe is true. Only 9 percent of Americans accept the central finding of modern biology that human beings (and all the other species) have slowly evolved by natural processes from a succession of more ancient beings, with no divine intervention needed along the way. (When asked merely if they accept evolution, 45 percent of Americans say yes. The figure is 70 percent in China.) When the movie Jurassic Park was shown in Israel, it was condemned by some Orthodox rabbis because it accepted evolution and because it taught that dinosaurs lived a hundred million years ago - when, as is plainly stated at every Rosh Hashonah and every Jewish wedding ceremony, the Universe is less than 6,000 years old. The clearest evidence of our evolution can be found in our genes. But evolution is still being fought, ironically by those whose own DNA proclaims it - in the schools, in the courts, in textbook publishing houses, and on the question of just how much pain we can inflict on other animals without crossing some ethical threshold.

During the Great Depression, teachers enjoyed job security, good salaries, respectability. Teaching was an admired profession, partly because learning was widely recognized as the road out of poverty. Little of that is true today. And so science (and other) teaching is too often incompetently or uninspiringly done, its practitioners, astonishingly, having little or no training in their subjects, impatient with the method and in a hurry to get to the findings of science - and sometimes themselves unable to distinguish science from pseudoscience. Those who do have the training often get higher-paying jobs elsewhere.

- CarlSagan

When you ask a question like "do you believe life arose over eons via gradual changes with no divine intervention by a deity needed?", you are indulging in a kind of experimental noise that the StatisticalProcessControl discipline calls "laminating". That's when you mix the product from two or more tools before measuring it. Don't do that. Ask two questions, "do you believe life arose over eons via gradual changes?" and then ask, "do you believe supernatural forces influence the shape of life?"

Sometimes skeptics - even the best - rig the questions so that the expected answers will be obtained. -- PhlIp

Maybe all arguments are rigged. I was tempted to use your own argument above as an example. So that it doesn't appear that I'm picking on you, I'll use one of mine. Somewhere above, I argued that evolution is just a (scientific) model. This characterization (and my argument) ignores any other possibility. For example, certain ideas move from the status of conjecture or scientific theory to mere observational reports (e.g. the idea that the earth is spherical). The only way to dislodge such ideas is to construct conspiracy theories, disregard them as illusions or to make minimal adjustments (e.g. the earth is not perfectly spherical).

(see ProofByRhetoric) -- ChrisSteinbach

Regarding TheDemonHauntedWorld, I can think of certain other books that appeal to Scientism while whining "how come everyone doesn't believe the same things as meee??" MK by AH comes to mind... ;-) -- PhlIp

The book had some flawed lines of reasoning in it. Although the claims he was debunking may be flawed, his counter-arguments were also flawed. For example, he did a fair amount of "second-guessing" what alien technology would look like.

For instance, alleged medical tools used on alleged alien abductees were claimed to be "too bulky" by Sagan to be realistic. He used a variation of MooresLaw to concluded the saucer-pilots would have much smaller medical tools. For one, we don't know how long MooresLaw will continue. Second, maybe the instruments pack in thousands of functions. Third, maybe they use technology where the parts need to be separated to some degree, such as an X-ray or sonic lens or the like. You cannot use flawed logic to counter other's flawed logic.

Another form of second-guessing is in the description of alleged aliens. Sagan insists that intelligent aliens would not likely look like humans in a general sense. However, maybe they are us in the distant future. Maybe there are many species of aliens, but only those who resemble us are interested in us, creating a kind of selection mechanism. We don't know enough to dismiss anything about the appearance of alleged "visitors".

Further, "incredible claims require incredible proof" was over-extended. Many UFO buffs believe that more research is warranted, not some fantastic claim. The threshold for "deserving more study" is much lower than the threshold for making a definitive conclusion about things observed (such as being "beings from other planets"). Sagan conveniently intermixed the two at whim to serve his debunking needs.

And, he seems to make the assumption that just because some (or even most) paranormal investigators are "kooks", that they all are kooks. There have been plenty of quack scientists also. The best investigators are those who are merely curious rather than trying to convince people. That works both ways.

Even if UFOs are just hallucinations, the fact that honest pilots and cops can have such vivid hallucinations (beyond mere "jumping lights" in many cases) is an interesting scientific question in itself. Letting pilots and cops hallucinate without knowing why is not smart science and perhaps bad for our safety. He is more focused on debunking than learning.

I used to read extensively about UFO's. I was determined to "get to the bottom of it" and figure out whether the phenomenon was hardware or WetWare. I never did come to a conclusion, for the debunking was usually disappointingly sloppy. But one thing I did learn by reading many witnesses accounts and the after-math is that if YOU see a flying saucer or UFO, shut the hell up. Reporting it is a recipe for headaches and ridicule. If you want to get it off your chest, report it to MUFON anonymously. --top

http://en.wikipedia.org/wiki/Chicago_O%27Hare_UFO_sighting_2006

As far as "alien technology would be too advanced to ever crash (in New Mexico)." Well AllRoadsLeadToBeeMinus is universal. They have in-boxes and budgets also.

Ghost of Carl Sagan Warns Against Belief in Supernatural

http://www.theonion.com/content/node/32640

This is amazing! Just 2 hours ago, I saw the guys from MythBusters? on the cover of Sceptical Inquirer magazine (IIRC). Then I clicked on that link, and the advertisement to go with that amusing piece is an ad for MythBusters?! What a synchronicity!! -- PhlIp

"Amusing" CelestialBilliards model moved to its own page. -- GarryHamilton

See also ScientificMethod, FrancisBacon, CarlSagan, ThinkingAndTheCriticalThinker, PseudoScience, ItFadSmell, GoodMetricsProduceNumbers

CategoryEvidence

EditText of this page (last edited November 9, 2014) or FindPage with title or text search