Scientific Method

Scientific Method is a formal method for understanding phenomena. It consists, briefly, of:

A description of the phenomena.
At least two hypotheses which "explain" the phenomena.
Extrapolation of the hypotheses until an experiment is found which will distinguish between them.
Performance of the experiment, and rejection of one or more hypotheses according to the result.

--- Contributors: anonymous, DaveHarris

Why must you reject one hypothesis? Can't two theories explain the same data? Also I wonder what worth a ScientificMethod has that only explains known phenomena. Shouldn't it also discover new things, postulate new phenomena?

You do not need to reject one hypothesis before an experiment can be found to reject it. 'Distinguish' is a bad descriptive here, since both hypothesis can, indeed, both explain the same data... including the ability fail at explaining the same data. A valid hypothesis must be capable of predicting observations on the phenomena that weren't utilized in developing the hypothesis (not 'new phenomena', which implies that science can change the universe), and must allow that identification of contradictory observations is possible, thus allowing rejection. Perhaps describing the necessary qualities of the 'hypothesis' should be part of describing the ScientificMethod. On the other hand, one should also define phenomena (i.e. in this context we speak of the empirical, the observable, be it directly or through use of tools) and experiment (which can range from controlled laboratory work to filtering through historical datasets that weren't used in developing the hypothesis).

As I understand it, there have been two major difficulties in producing either a descriptive or a generally applicable ScientificMethod. A descriptive ScientificMethod (one that describes scientific practice) has been elusive because no common theme, philosophy or method is discernible throughout the whole of what we call science. Every rule has been broken at least some time during the history of science and often for the better.

The objection to a generally applicable ScientificMethod is that science is more like a tradition than a religion with a fixed ideology. It has no single aim. PaulFeyerabend developed this idea (and others) in his book AgainstMethod. He argues that the only principle which does not inhibit progress (using whichever definition of progress you see fit) is AnythingGoes. -- ChrisSteinbach

The only scientific method is this:

you believe in something for whatever reason
you look for evidence until you're personally convinced that it is true
you publish your belief and whatever evidence supports it in a peer-reviewed journal
you argue with other scientists
you decide together what you're going to say to the illiterates (students and the public)

The real scientific method, what scientists actually practice, is this: shut the hell up when talking to the public. If you bother to look at evidence of how scientists actually behave then you'll see that Fleischman's cold fusion is reviled as quackery while Andrei Linde's cosmic foam is respected as a radical theory. Why? Because the former published directly to the press, while the latter published in peer-reviewed journals. There is no other reason.

I wrote more on this in OpenMindedScience? btw. -- RK

That's a method used by some practitioners, true. Looking for evidence you didn't use in developing the belief is a good start. Where it fails is in the avoidance of any search for counter-evidence. With the method you describe, you get religious leaders and confidence-men far more easily than you get scientists. Go ahead... apply the above five steps to 'charlatan'. It fits. Many scientists are, individually, charlatans. Fortunately for the ScientificMethod as a whole, there are a whole bunch of other people who want to disprove you simply to gain a reputation... and they'll work hard to do so, and if they succeed, they shouldn't be suppressed. The ScientificMethod operates far better at the level of a social construct than it does for individuals. -db

The original scientific method, also called Cartesian method was proposed by ReneDescartes in the seventeenth century. Its main tenet was that the entire world could be understood in terms of machines, and its main approach was divide and conquer.

The way to understand anything according to scientific method is:

Analyse: To divide any concept of thing, preferably into two parts;
To keep subdividing until reaching "clear and distinct" parts;
Synthese: To reassemble the parts bottom-up to create a whole, and
Denombrement: To do an overall audit to ensure the process was used properly.

I think formal experimental science came later as an outgrowth of the broader Cartesian methods of inquiry, so the above two definitions are relatively new perversions of the original.

In a little-cited part of Descarte's writings, he makes it explicit that there are problems to which scientific method is not well-suited.

Alexander talks much about Cartesian method in his books and why it fails.

-- JimCoplien

[Moved from FourNobleTruths]

Alternatively:

Initiation
Confirmation
Realization
Dedication

The Alternate may be loosely associated with the Scientific Method. As such it can connect truths as associated with thoughtful and actionable activity rather than with the disconnect from rationality that is made in FourNobleTruths.

http://www.soci.niu.edu/~phildept/Dye/method.html

I'm inclined to think that a lot of what we do when programming is a direct application of the ScientificMethod, although nobody ever outright states this. See DebuggingAndTheScientificMethod. -- CurtisBartley

I think there's something missing here. To my mind, the key to the scientific method is that after you've taken the four steps above, *you're not done*.

You have to analyze the data you've accumulated and come up with a new hypothesis (based on more current information) and start the process all over again. You may have to do several iterations before you believe that you're actually solving the problem. This is the model I use for programming.

-- AnthonyLander

Anthony: I agree with you that the formulations of the ScientificMethod above aren't quite right. Here's *my* formulation of the ScientificMethod which I used when I started the DebuggingAndTheScientificMethod page.

Identify the behavior you want to explain.
Form a hypothesis that might explain that behavior.
Conduct an experiment that tests the hypothesis.
If the experiment contradicts the hypothesis, revise it or replace it with a new hypothesis and go back to step 3.

Does this sound more like it?

-- CurtisBartley

Yes, it does. To phrase this in programming terms, (1) is a UseCase, (2) is a piece of test code (UnitTest), (3) is the implementation that runs the test (TestingFramework), and (4) is looking at what you learned about the problem while doing 1 to 3 so that you can revise (RedBar/GreenBar). WilfLaLonde? taught this to me many years ago at CarletonUniversity. He now calls it NeedsDrivenProgramming, and he spends a lot of time talking about it in his various courses. -- AnthonyLander

If we continue with this analogy, it would not only be the hypothesis that is open to revision. It would also be reasonable to amend the verification method (test code) and reinterpret experimental results as development continues. Are there some subtleties of the method outlined that would account for such changes?

The Scientific Method ultimately turns on the principle that Nature Knows. In other words, whatever the underlying reality, if you ask the proper question of nature it will give you either a confirmation or a disconfirmation. The hypothesis is simply a possible explanation on which we hang our thought processes. It becomes dangerous when we think that a confirming experiment confirms the underlying hypothesis. What a confirming result does is give evidence that the hypothesis is more correct than a hypothesis that does not predict the result. Thus all experiments are conditional in character. An experiment does not have to be framed in the context of a hypothesis. One can just do something to see what happens. Usually that is the real beginning. If what happens is consistent with known science, well fine. If it is not, then one has to 1) explain why not! - the first issue is to assume that the current theory is correct but that the experiment is wrong (about 90% or so of such observations can be explained by poor experimental technique), 2) if the experiment survives then you have to ask what is wrong with the theory that predicted a wrong result ... and there begins the process of theoretical transformation. The real process is no where near so cut and dried as the step by step structure suggests. Reality hardly every is quite as simple as we would like it to be.

When I was first learning to program, when subtle syntactic issues would come up, the response was generally "AskTheComputer" and we would form test cases to see what the compiler would do. This is rather like the idea of Ask Nature! Except in the case of the compiler we can change it. We don't have that much control of nature. -- RaySchneider

"2. Form a hypothesis that might explain that behavior." - really there are at least two hypotheses. One of them is the null-hypothesis, which states that all of the other hypothesis are wrong. Often you will do better if you have more.

For example, if you have some GUI ideas and you want to see if they work, don't knock one up and show it to some users, knock up two. Then the users have something to compare with. Give them a choice that goes beyond "take it or leave it." -- DaveHarris

The SkepticalInquirer? magazine is an excellent source of information on the scientific method. Their site at http://www.csicop.org/si/ is well worth browsing. -- JohnRochester

I found an interesting piece 'The End Of Science' on the CSICOP web site: http://www.csicop.org/si/9703/end.html some years ago. On re-reading, I find it a mixed bag of nonsense and muddle-headedness. Nevertheless, for me, as someone who knew very little about the philosophy of science, it was a good starting point. Be careful as you read however. The author of this article seems to be open to the rationalization of uncomfortable facts that CSICOP members generally disparage. -- ChrisSteinbach

The scientific method as is usually stated in science books is not how scientists actually work. It's only how they write the paper at the end.

Yes... but the paper is not the end of the process. The paper is the input to another process. After a lot of sieving out, the content of a paper may make it into the collection of ideas that form TextBookScience?.

I have a mnemonic for the scientific method that is very short, but encourages thought:

: Is this reasonable?

For me, it captures everything that is important. For example, how can you tell if what you have is reasonable unless you have thought about what results you might get, so you need at least one hypothesis and a prediction. Clearly, you also need to have thought about the data you already had, and so on...

Oh, and it also stops me (sometimes) from accepting obviously wrong outputs from programs that I write. -- ChrisBooth

The ScientificMethod was formulated as a response to the many struggles on what reasonable actually means. -- HaskoHeinecke

It all starts with a testable question.

Actually, it all starts with a scientifically falsifiable question (hypothesis). For the scientific method to be applicable, the hypothesis must be capable of being proved false through formal observations.

At least according to KarlPopper, who replaced the LogicalPositivism idea of "testable" with "falsifiable".

The ScientificMethod is a good AttentionDirector for any inquiry.

The following is not directly related but I thought you might find this interesting...

A new scientific theory must be able to:

a. Explain all previous experimental data b. Explain experimental data that has not yet been explained by previous (established) scientific theories c. Make predictions that can be proven or disproven by experiment

b is optional if the new theory is simpler than the old one. See OccamsRazor.

I don't think so. b is the first important step for accepting the validity of the new theory. Before Einstein published his Black Body Radiation and the Photoelectric effect papers, there was no explanation for these phenomena. This led to the evaluation of his ideas by the physicists of his time. Einstein's ideas (light is absorbed and emitted in discrete amounts of energy called photons) led to the emergence of quantum physics.

c is optional too. Predictions aren't made by theories, they're made by scientists. If a theory explains all previous data as well as any existing theory, it need suggest no further predictions to be considered equally valid, OccamsRazor notwithstanding. There is certainly no reason to accept an older theory in its place. Scientists may be able to make predictions on the basis of theories in order to add weight to one theory or another, but there's no reason to assume that that need be possible. Ask Godel.

No, c is not optional. If a theory cannot lead to a prediction that can be disproven, if it cannot possibly be disproven, then it is not a valid scientific theory. Rejection of c allows the SpaghettiMonster? style theories to apply: everything is the way it is because the SpaghettiMonster? makes it so. This explains all previous data. This makes no predictions. This will explain any possible future data. Oh, and your invocation of Godel here isn't particularly relevant. -db

According to Kuhn in TheStructureOfScientificRevolutions, new theories usually don't explain all previous scientific data. All theories have holes, things they don't explain. A new theory has different holes than the old, and usually explains things that the old doesn't, but might actually have more holes at first because its supporters haven't figured out how to fill the holes. He claims that Copernicus' proposal was neither simpler nor more accurate than Ptolemy's system, and that the theory of oxygen had more holes at first than the theory of phlogiston. He says that the main reason that people abandon a theory is that they are frustrated with its failures more than that are sure that the new one is better.

If you are happy with your old theory/programming methodology/religion, why switch? If you aren't happy, look for a new one.

Kuhn says that "normal science" is a lot like what people say here is the ScientificMethod; it is problem solving and involves setting up experiments to prove your point. But scientific revolutions do not follow this pattern. -- RalphJohnson

Unfortunately, SoftwareEngineering is more closely related to psychology than say math or physics IMO. Thus, the scientific method will probably not help us much in this field until the mysteries of psychology are better unraveled.

Wikipedia article: http://www.wikipedia.org/wiki/Scientific_method

There is no scientific method. Such is the conclusion of PaulFeyerabend which he goes on to demonstrate in his book AgainstMethod. There is no such thing as a method used by all scientists or all proper scientists or all successful scientists, whether in all history or even at one specific point in time. The notion is bunk, it's a myth, a lie. To believe in a scientific method despite its proven disproof is exceedingly unscientific, and the widespread belief in it by itself disproves its existence.

Well slow down there! Feyerabend didn't show that the scientific method is not used. What he tried to show is that there is no discernible set of rules that govern the progress of science. Scientists do apply the scientific method, but we are asked to consider that this does not imply that scientific method is responsible for scientific progress any more that a rain-dance is for subsequent rain fall. I don't think we can say that Feyerabend disproved the scientific method, or any similar philosophy. What we can say is that it appears, historically, that such rules have been broken to good effect, and yet on examination, there is nothing particularly irrational going on. The conclusion he would like us to take away is that rationality and progress are not bound to simplistic methods and philosophies.

While Feyerabend was undoubtedly a controversial figure, I believe that his philosophy was no more controversial than saying that science is more than just a magic mantra of "trial-and-error" methodology. Ironically, it is Feyerabend who is always portrayed as the mystic, although he was constantly fighting against popparians and positivists who thought they had found some mystical, simple formula to explain science. -- ChrisSteinbach

You realize that this is utter crap, yes? And that had you read AgainstMethod, you would realize that scientists don't apply any particular set of rules to guide their activities? Had you read Feyerabend, you would realize that he doesn't believe in rationality. His position isn't mystical, it's anti-ideological, but it certainly isn't mainstream as you claim. Feyerabend didn't advocate for a more complex ideology, he advocated for no ideology, period, none whatsoever. -- RK

I guess you are right about F's views regarding method. Indeed I have read AgainstMethod; several times in fact. Nonetheless, I appear to have gotten my knickers in a twist for a moment. What I should have said is that scientists often believe themselves to have applied the scientific method, which I admit is something quite different.

Concerning terms like 'rationally' and 'progress', it's not so much that Feyerabend doesn't believe in them. It's more that he believes rationalist conceptions of 'rationality' are not good for science. We shouldn't revere word like ?Reason? as platonic ideals. That doesn't mean the words then become meaningless. We should only be aware of how they can be used to bully and intimidate.

As for Feyerabend being mainstream, I agree, he's nothing of the sort. This is his fault alone for being so bloody confrontational and insulting (hint). I don't think it's particularly difficult to envisage a science where theories are argued for or against without recourse to a single set of ideals. I can imagine a world where scientists are suspicious of methodology talk. But, no, Feyerabend isn't remembered for anything worthwhile that he said. Instead he is routinely dismissed as the anti-science philosopher who believed that AnythingGoes. And how wrong is that? -- ChrisSteinbach

The Scientific Method seems to be a shorthand version of the 7 problem solving steps? 1. Define and Identify the Problem 2. Analyze the Problem 3. Identifying Possible Solutions 4. Selecting the Best Solutions 5. Evaluating Solutions 6. Develop an Action Plan 7. Implement the Solution

In the discussion of how scientific any of this is, I would like to point out that even in Step 1 of either way, the observer has already skewed their own results. In order to understand the problem the observer has to frame it within their own thought-references in order to arrive at at the definition of the problem. (I have often seen scientists posit ideas that are new (to them) simply because their thought-reference has no knowledge of the idea. When you do cross reference the idea across the whole of human knowledge, often there is already an explanation or 'result' available, but beyond that there is always a new piece of information that directly impacts the observers 'scientific method', resulting in a error laden result.) When implementing the Scientific Method or Problem Solving steps, one reaches an answer to only their definition of their idea. - September Amyx (Who hopes her definition of the answer makes sense)

Your steps are, as you've stated, for problem solving. They might be considered engineering, but they are not the scientific method. See the top of this page for an accurate definition. The steps that immediately follow that definition (signed "RK") are also not the scientific method. They are perhaps accurately reflective of a certain academic process, but they are no more reflective of the scientific method than your definition.

SCIENTIFIC METHOD REQUIRES VALIDATION

The steps mentioned at the top of the page lists the standard experimental procedures to be followed during the primary validation phase of the application of Scientific Method, namely the Primary Evidence Collection phase. They do not however constitute the complete requirements of Scientific Method.

The procedures serve as a primary validation of the quality of the scientific process by requiring the experimenter to add value to the specific scientific body of knowledge by adopting a deductive experimental approach. Namely, that the body of knowledge relating to a specific phenomena be established through a deductive and logically sound process. An experimental approach which uses an inductive process, namely, a process where a hypothesis proves true, adds value to the body of knowledge, but it can only be used to establish proof, and only after ALL possible non-eventualities have been eliminated. Proof, has an extremely heavy burden, an impossible burden.

Unfortunately, the financial and resource constraints faced by researchers, tend to urge them to adopt a non-scientific method approach, with the occasional padding of test results, in their haste to establish cause and effect. Such an approach will inevitably fail the secondary validation process.

Since it is possible for anyone to complete the primary stage of scientific method, it is the secondary validation process which is the most important. It is the process where the Principle of Full Disclosure is applicable. And even here, specific qualifiers are required. Full disclosure requires that the primary researcher identifies ALL materials which have been collected and distributes them to a group of independent peers, or more specifically to opposing peers or others with vested interests in opposition to the results of the study, who would have the necessary knowledge required to reproduce and independently verify the results obtained by reproducing the experiment results. Most importantly, it cannot be a group of peers who have a vested interest in support of the results of the primary researcher, since this would constitute a conflict of interest, and would assign their contributions as primary research. Whilst it may add value to the primary research results, it does not validate them. Science requires validation through dissent, not consent.

It is this second requirement which enables a clear identification of Scientific Method compliance. The unilateral consent by peers in support of a Specific Theory, can never satisfy the validation requirement of Scientific Method. A criticism of the evidence presented in support of Global Warming by Greenhouse Gases is that the experiment results do not fully comply with Scientific Method, since the primary research has never really been made readily available to the public or dissenting groups, and further, has been unilaterally supported by peers who had a vested financial interest in supporting the "Global Warming Theory", and who were questionably biased in their contribution.

The above statement is not a reflection on the merits of the link between Global Warming and increases in Greenhouse Gases, this may well be the case! But rather is serves as an example of an illegitimate application of the principles of the Scientific Method process. Albert Einstein once stated "No amount of experimentation can ever prove me right; a single experiment can prove me wrong". Scientific Method preserves the integrity of the scientific process by ensuring that opposing knowledge can never be silenced.

Is it possible that the difference between science and religion is simply that science is a deductively derived body of knowledge, whilst religion is inductively derived? -- HelderFigueira?

CategoryAbstraction (I think)