Empirical Evidence

What someone asks for when you give him good advice that he doesn't want to take.

The second step of the dance is when you give him EmpiricalEvidence and he denies that the experiment was valid.

The third step is when he tells you that his situation is different.

My advice is that when you are asked for empirical evidence you should just say "The evidence is out there if you look for it" and walk away. The exercise will do you good, and the conversation will not.

Of course I have no evidence for this.

If you can't back up your ideas with empirical evidence, or show why this is impractical, they probably aren't worth listening to anyway. Come back when they are fully baked.

But people could not find any empirical evidence that nested blocks are better than Goto's. Does that mean we should use Goto's? The best justification I have ever seen is that nested blocks and indentation gives the code visual clues as to the "shape" of the flow, while Goto's gave fewer or no such clues. But this is a kind of a ProofByPrinciple? rather than purely empirical.

Did you catch the second part of the first sentence? This is mostly a reaction to the fairly inane assertion implicit in the opening text: that if someone is asking for real, evidence that your idea (advice in this case) is a good one, they are somehow trying to blow you off. Of course it could be read tongue-in-cheek, but that hardly deserves the top spot on a page about empirical evidence.

Or else, they're engaging in ShiftingTheBurdenOfProof. A common theme WRT ShiftingTheBurdenOfProof is "I won't withdraw my claim unless you prove it false". As many people enter debates with the hope of getting the other guy to admit defeat (either by doing so explicitly, or implicitly by withdrawing from the debate), this can be a surprisingly effective tactic at getting the other guy to tire of things. And, as alluded to above, if empirical evidence actually does get introduced, the burden of proof is shifted higher. The study is flawed, it doesn't apply, the proof is only statistical and not absolute (even if there exists a high correlation), etc. The moral of this story: Never confuse the pissing contests that occur in online forums such as WardsWiki with actual science.
But we often don't have actual science in this biz (DisciplineEnvy), so the PissingContests may be the consolation prize.
It may be true that ComputerScienceIsaSoftScience, but soft sciences are sciences nonetheless. We can devise experiments, take measurements, analyze and publish results. Unfortunately, the conclusions drawn are far less "solid" then axiomatic proof, but dismissing soft sciences as worthless is one of the ScientificSins.

If you came here thinking that a page labelled EmpiricalEvidence, might actually contain some, you will be disappointed. You may find it more satisfying to read ExtremeProgrammingResearch.

Where is the best OoEmpiricalEvidence documented? AdVerecundiam often does not go very far if a customer has been burnt by too many claims and dogma fads that have failed for them. Just because they are not open to opinions does not necessarily mean they are not open to something more tangible (source code examples, modification impact analysis, satisfaction surveys, etc.).

There is an AntiPattern to this DarkPattern as well:

Magic fantastic claim!

Is it true?

Well, here is a magic fantastic philosophical argument why it ought to work.

Well, while your argument seems valid, you have to prove your magic fantastic premises. Do you have any EmpiricalEvidence to suggest the magic fantasy you assume?

Well, here are some completely irrelevant, magic fantastic anecdotes!

That's nice, and while I understand that for you it seems to work, what you claim seems rather, well, different from my experiences. Maybe one of our experiences is abnormal. Do you have any general, verifiable data?

Oh, you just don't want to listen. Poo, poo on you.

Whatever. Not interested in SnakeOil.

Comment on previous dialogue

Just because the demand for EmpiricalEvidence is almost always ConversationalChaff doesn't mean it's wise to accuse the other person of being closed-minded. It is usually not a good idea to accuse someone of being closed-minded even if he is. It keeps the conversation in debate mode, only now the topic is whether your friend is being closed-minded or whether you're being unreasonable. A person who is arguing that you are being unreasonable to defend against the charge that he is being closed-minded is probably not listening to the idea you wanted to offer.

When the demand for EmpiricalEvidence is just a way of indicating lack of interest in what you have to say, you should accept that if you want to get this person to be open to your idea, you will need to first sell him on listening to you. That's different from selling him on the idea itself. The first step in getting an idea heard is getting the other person to lower his MemeFilters?.

Of course, you should be aware that most people do not have time or inclination to listen to new ideas, no matter how good. Therefore you might be wiser talking with someone else, or talking with this person about another topic.

In either case, it would help if you learnt how to argue properly. Take philosophy classes at your community college. That will help you understand why it's important to demonstrate premises that your interlocutor disagrees with and how, as the interlocutor, to properly reject claims. But as the proponent of the idea, when asked for EmpiricalEvidence, either you provide it or you drop the issue. Don't get all snooty like this page is until you have provided the required evidence yet your interlocutor still maintains a closed mind. Only then should you thoroughly crush said lunt's ego and dance on his tears!

Do you have any EmpiricalEvidence that arguing with people persuades or enlightens them?

If you'd like EmpiricalEvidence to the contrary, see Robert Cialdini's book Influence: Science and Practice. ISBN 0321011473 That book summarizes research in social psychology about what actually does persuade and what does not. As you've probably already guessed, facts and cogent argument against an idea usually move people to commit more strongly to the refuted idea (if they already favored it). Persuasion and enlightenment are complex matters, though; many factors work simultaneously, so of course there are also rare times when people will listen to detailed factual examinations.

It's a set-up

Nothing of generality can be proven by EmpiricalEvidence. This is one of the fundamental insights behind modern, Popper-style science. You can't empirically prove a theory (show that it applies to an infinity of cases), but you can empirically disprove one (find one case that it doesn't apply to).

The same insight explains what this page is about. Demanding that someone prove his statements by empirical evidence is just a set-up. It raises the bar higher than is theoretically possible to jump. On the other hand, asking that someone show you what leads him to think his idea is a pretty good guess is a way to share knowledge and perhaps cooperatively discover problems.

Evidence is not Boolean. EvidenceTotemPole

I quite understand the problems with anecdotal or empirically observed data. I have personally observed stuff. The only people that seem to "get it" are those with similar anecdotal experience.

Of the more minor bits, I have a spouse who knows things before they happen. At a distance. Deaths. Earthquakes. Financial disasters. Motive-based outcomes. Where the cards are in the friggin' deck (do not play cards with this woman). Does she know how she does it? No. Can she control it? No. Is there any neat, rational, scientific explanation how she knows these things? Not a chance. Can I "prove" she has ESP? Ha ha ha hee hee har har chuckle chuckle haw haw - hack - gasp - oh, don't do that!

Oh, yeah, and did I mention that this is one of the more minor observations?

Do you have any idea how many arguments I've "lost" because I can't prove I've observed stuff?

I could, you know, just pretend none of these things ever happened, that I must have imagined them, and fall back on "I'll believe it when I see it." Oh, wait, that doesn't work. I have seen it. But most of the people around me haven't seen it. So, what does that mean? "I'll believe it when someone else agrees they've seen it too?" or "I'll believe it when the scientific community does a peer review and accepts it as a working theory?"

I'm so glad I don't have to prove what I know to everyone. And, I might add, the rest of you should probably be equally thankful you don't have to prove what you know to everyone else.

Empirical may not be good enough for science, but without it individuals would wander aimlessly through a fog of doubt.

-- GarryHamilton

Such "hidden evidence" is not very useful for communication between individuals who may be skeptical of your observations, regardless of how much it may have changed your viewpoint. You can't just expect people to believe you at your word. It is not that they are accusing you of lying, it is often that they would just like to explore the scene more. For example, if you observed OOP making procedural code simpler, but did not keep the code, then a skeptic might not believe you because he may believe that he could have possibly fixed the procedural code by rearranging it without switching to OOP. But that cannot take place because that case is lost to the wind. Both sides are essentially stuck in nowhere-land with regard to your personal observation. It is not something to take personally, just the nature of the beast. If you expect to "sell" to your skeptics, then you will have to produce something more concrete. Otherwise, AgreeToDisagree and move on.

Indeed.

What is the alternative?

I'm re-starting this section, freely re-writing to make the opening question clearer (and possibly altering the meaning very badly in the process). Email me if you want the original text from 18-Jan-2006. -- BenKovitz

What should I do if someone says something I don't agree with?

Here are a few options:

Double-check the subject and predicate of the proposition that seems to be in dispute: verify that you know what the other person is referring to, by putting that in your own words and getting confirmation; and verify that you know what he's saying about it, by putting that in your own words and getting confirmation.
Explicitly state that you'd like to persuade the person that the proposition is wrong, and ask if they would be interested to hear your argument. If the answer is no, move on to some other activity. If the answer is yes, present your case, and when you're done, ask if they found themselves moved by what you said.
Explicitly state that the proposition sounds wrong to you, but you're interested in correcting your current view of things if you're mistaken. Then ask the person if they would be willing to address your objection. Only do this if you really are interested in learning something.
Ask what led the person to believe as he does.
If it's an abstract idea, ask for illustrations and examples. If it's a concrete claim, ask if he saw it personally or where he heard it.
Continue as if the person agrees with you.
Try to humiliate the person in his own eyes.
Send him to a concentration camp, and then accuse him of being the first to reach for GodwinsLaw.
Call him the stupidest, most moronic, vile, subhuman loaf of excrement that ever plopped onto the still-dewy, mid-morning grass.
Replace what the person said with something obviously stupid, and refute that.
Nothing.
Demand EmpiricalEvidence.
Eat an orange.

I Googled this subject. The first hit came up with the following (on a biblical interpretation website):

If someone says something I don't agree with, I'm going to try to get them silenced.

Thanks for the edit! I think this section may now be able to converge on some positive discussion. I would suggest as a next step that contributers review the list of possible actions and eliminate the obviously ineffective approaches. Then we can possibly discuss the best of the rest.