Empirical Evidence

What someone asks for when you give him good advice that he doesn't want to take.

The second step of the dance is when you give him EmpiricalEvidence and he denies that the experiment was valid.

The third step is when he tells you that his situation is different.

My advice is that when you are asked for empirical evidence you should just say "The evidence is out there if you look for it" and walk away. The exercise will do you good, and the conversation will not.

Of course I have no evidence for this.

If you can't back up your ideas with empirical evidence, or show why this is impractical, they probably aren't worth listening to anyway. Come back when they are fully baked.

But people could not find any empirical evidence that nested blocks are better than Goto's. Does that mean we should use Goto's? The best justification I have ever seen is that nested blocks and indentation gives the code visual clues as to the "shape" of the flow, while Goto's gave fewer or no such clues. But this is a kind of a ProofByPrinciple? rather than purely empirical.

Did you catch the second part of the first sentence? This is mostly a reaction to the fairly inane assertion implicit in the opening text: that if someone is asking for real, evidence that your idea (advice in this case) is a good one, they are somehow trying to blow you off. Of course it could be read tongue-in-cheek, but that hardly deserves the top spot on a page about empirical evidence.

If you came here thinking that a page labelled EmpiricalEvidence, might actually contain some, you will be disappointed. You may find it more satisfying to read ExtremeProgrammingResearch.

Where is the best OoEmpiricalEvidence documented? AdVerecundiam often does not go very far if a customer has been burnt by too many claims and dogma fads that have failed for them. Just because they are not open to opinions does not necessarily mean they are not open to something more tangible (source code examples, modification impact analysis, satisfaction surveys, etc.).

There is an AntiPattern to this DarkPattern as well:

Magic fantastic claim!

Is it true?

Well, here is a magic fantastic philosophical argument why it ought to work.

Well, while your argument seems valid, you have to prove your magic fantastic premises. Do you have any EmpiricalEvidence to suggest the magic fantasy you assume?

Well, here are some completely irrelevant, magic fantastic anecdotes!

That's nice, and while I understand that for you it seems to work, what you claim seems rather, well, different from my experiences. Maybe one of our experiences is abnormal. Do you have any general, verifiable data?

Oh, you just don't want to listen. Poo, poo on you.

Whatever. Not interested in SnakeOil.

Comment on previous dialogue

Just because the demand for EmpiricalEvidence is almost always ConversationalChaff doesn't mean it's wise to accuse the other person of being closed-minded. It is usually not a good idea to accuse someone of being closed-minded even if he is. It keeps the conversation in debate mode, only now the topic is whether your friend is being closed-minded or whether you're being unreasonable. A person who is arguing that you are being unreasonable to defend against the charge that he is being closed-minded is probably not listening to the idea you wanted to offer.

When the demand for EmpiricalEvidence is just a way of indicating lack of interest in what you have to say, you should accept that if you want to get this person to be open to your idea, you will need to first sell him on listening to you. That's different from selling him on the idea itself. The first step in getting an idea heard is getting the other person to lower his MemeFilters?.

Of course, you should be aware that most people do not have time or inclination to listen to new ideas, no matter how good. Therefore you might be wiser talking with someone else, or talking with this person about another topic.

In either case, it would help if you learnt how to argue properly. Take philosophy classes at your community college. That will help you understand why it's important to demonstrate premises that your interlocutor disagrees with and how, as the interlocutor, to properly reject claims. But as the proponent of the idea, when asked for EmpiricalEvidence, either you provide it or you drop the issue. Don't get all snooty like this page is until you have provided the required evidence yet your interlocutor still maintains a closed mind. Only then should you thoroughly crush said lunt's ego and dance on his tears!

Do you have any EmpiricalEvidence that arguing with people persuades or enlightens them?

If you'd like EmpiricalEvidence to the contrary, see Robert Cialdini's book Influence: Science and Practice. ISBN 0321011473 That book summarizes research in social psychology about what actually does persuade and what does not. As you've probably already guessed, facts and cogent argument against an idea usually move people to commit more strongly to the refuted idea (if they already favored it). Persuasion and enlightenment are complex matters, though; many factors work simultaneously, so of course there are also rare times when people will listen to detailed factual examinations.

It's a set-up

Nothing of generality can be proven by EmpiricalEvidence. This is one of the fundamental insights behind modern, Popper-style science. You can't empirically prove a theory (show that it applies to an infinity of cases), but you can empirically disprove one (find one case that it doesn't apply to).

The same insight explains what this page is about. Demanding that someone prove his statements by empirical evidence is just a set-up. It raises the bar higher than is theoretically possible to jump. On the other hand, asking that someone show you what leads him to think his idea is a pretty good guess is a way to share knowledge and perhaps cooperatively discover problems.

Evidence is not Boolean. EvidenceTotemPole

I quite understand the problems with anecdotal or empirically observed data. I have personally observed stuff. The only people that seem to "get it" are those with similar anecdotal experience.

Of the more minor bits, I have a spouse who knows things before they happen. At a distance. Deaths. Earthquakes. Financial disasters. Motive-based outcomes. Where the cards are in the friggin' deck (do not play cards with this woman). Does she know how she does it? No. Can she control it? No. Is there any neat, rational, scientific explanation how she knows these things? Not a chance. Can I "prove" she has ESP? Ha ha ha hee hee har har chuckle chuckle haw haw - hack - gasp - oh, don't do that!

Oh, yeah, and did I mention that this is one of the more minor observations?

Do you have any idea how many arguments I've "lost" because I can't prove I've observed stuff?

I could, you know, just pretend none of these things ever happened, that I must have imagined them, and fall back on "I'll believe it when I see it." Oh, wait, that doesn't work. I have seen it. But most of the people around me haven't seen it. So, what does that mean? "I'll believe it when someone else agrees they've seen it too?" or "I'll believe it when the scientific community does a peer review and accepts it as a working theory?"

I'm so glad I don't have to prove what I know to everyone. And, I might add, the rest of you should probably be equally thankful you don't have to prove what you know to everyone else.

Empirical may not be good enough for science, but without it individuals would wander aimlessly through a fog of doubt.

-- GarryHamilton

Such "hidden evidence" is not very useful for communication between individuals who may be skeptical of your observations, regardless of how much it may have changed your viewpoint. You can't just expect people to believe you at your word. It is not that they are accusing you of lying, it is often that they would just like to explore the scene more. For example, if you observed OOP making procedural code simpler, but did not keep the code, then a skeptic might not believe you because he may believe that he could have possibly fixed the procedural code by rearranging it without switching to OOP. But that cannot take place because that case is lost to the wind. Both sides are essentially stuck in nowhere-land with regard to your personal observation. It is not something to take personally, just the nature of the beast. If you expect to "sell" to your skeptics, then you will have to produce something more concrete. Otherwise, AgreeToDisagree and move on.

What is the alternative?

I'm re-starting this section, freely re-writing to make the opening question clearer (and possibly altering the meaning very badly in the process). Email me if you want the original text from 18-Jan-2006. -- BenKovitz

What should I do if someone says something I don't agree with?

Here are a few options:

I Googled this subject. The first hit came up with the following (on a biblical interpretation website):

Thanks for the edit! I think this section may now be able to converge on some positive discussion. I would suggest as a next step that contributers review the list of possible actions and eliminate the obviously ineffective approaches. Then we can possibly discuss the best of the rest.

See Also: OoEmpiricalEvidence


EditText of this page (last edited January 19, 2006) or FindPage with title or text search