Introduction
My aim here is to talk about some popular misconceptions about what science is, what it does, and how to best interpret scientific findings. First, I will speak in general terms, and finally I will look at how these principles apply to CAM (i.e. complementary and alternative medicine), anti-vaccine, anti-GMO, and pretty much any science denialist position.
Before continuing, lets get the credibility part out of the way first. Who dafuq does a philosopher think he is talkin' 'bout wut science is an' sh*t? Well, without going too far into it, philosophical questions are by definition non-scientific questions. So, if there are questions we can ask about science itself that can't be resolved scientifically (e.g., "what is science?"), then a philosopher is a least one of the people you'll want to talk to.
"B.b.bbbb.b..bbut you aren't a scientist!" True. But do you need to know how to ride a bike to understand how a bike works? (That's a rhetorical philosophical question doubling as an argument by analogy for anyone keeping track). Enough yakity yak, lets break this shit down.
Popular Misconception 1: Science is just like a Religion
No. No, it isn't. Science is a method of seeking knowledge. Science is not a body of facts. The scientific method is (roughly) (a) postulate a hypothesis to account for a series of observed phenomena (b) perform controlled experiments that try to both confirm and falsify the hypothesis, (c) express, based on the available evidence, the degree of confidence in the hypothesis. If after many confirmations, as well as rigorous attempts to falsify the hypothesis, the hypothesis still stands, the hypothesis is elevated to theory.
If however, during the course of experiments and observations there is evidence to suggest a better explanation of the observed phenomena, then the old hypothesis is modified or rejected, and replaced by the new hypothesis. The important point here is that scientific conclusions are provisional. If a new batch of repeatable higher quality evidence falsifies the existing hypothesis, it is rejected and replaced/modified.
Consider some of the first giants of science: Boyle, Newton, Bore, Galileo, Darwin, and so on. I don't think anyone could seriously disagree with the assertion that these guys were doing science. But guess what? Their theories were all later shown to be wrong to varying degrees. So what happened?
Did scientists deny the new higher quality evidence because they wanted to "stick to the facts"? Maybe this happened with some during the transition period between theories--before all the new evidence was available--but eventually what happened? They changed their position based on the best available evidence.
Some people point to this as a weakness of science. "How we supposed to believe what science says is true if it keeps changing its mind?" they cry, tears streaming down their angelic faces. But sensitivity to new and better evidence is precisely what makes science such a powerful tool. If you don't adjust your beliefs to new and better evidence you are in the weak position.
This method of continual change and refutation of scientific hypotheses continues to this very day. It is known as the self-correcting nature of science and it is at the heart of what science is. If you follow any science website, podcast, or blog, revisions and rejections of hypotheses based on new evidence is a large part of what you'll hear about. Shit, after more than a century of wide acceptance, scientists rejected Newtonian physics! Let me say that again so it sinks in: they rejected Newtonian muther f*ckin' physics! Science is not committed to any body of facts (or theory, for that matter). Now, lets contrast this with religion.
Religion is defined by a more-or-less rigorous adherence to a set of certain truths and/or a world view. These truths are typically not amenable to contradicting evidence. There are no attempts to falsify the religion's dogmas. There is no sensitivity to contravening evidence: it is either explicitly denied, ignored, trivialized, or met with charges of blasphemy or conspiracy. At the heart of religion is a commitment to a set of truths. This could be no more different than what defines science. So, no. Science is nothing like a religion.
Evaluating Evidence (For the Intelligent Lay-Person)
How many times a day do you get some post in your news feed about something curing cancer or ancient Tibetan ear therapy that cures autism? Your first instinct is probably (hopefully) skepticism. But then your friend posts a *study* done by real scientists in labcoats that supports the conclusion: Now, you should believe!
Aside on Cognitive Biases
Before going any further we need to talk a little bit about cognitive biases. A cognitive bias is a mental shortcut or heuristic that is evolutionarily hardwired into your brain. You don't do it consciously and it's hard to stop yourself from doing it. It's automatic, like flinching when someone throws a ball at you. It takes training an conscious effort to overcome it.
Over the course of evolutionary history, these mental shortcuts led to more survival producing beliefs than non-survival producing beliefs. That's how we ended up with them. Unfortunately, the shortcuts don't always get it right. (Arguably) they work best in the evolutionary conditions under which they were formed. Maybe sometime in the future, we will evolve new heuristics for evaluating scientific studies posted in a facebook feed, but for now, we're stuck with the tools we've got.
The whole point of this spiel is to alert you to the most dominant cognitive bias: confirmation bias. Confirmation bias is when we only look at or only remember the evidence that supports our prior beliefs but ignore or trivialize evidence that contracts our prior belief. In pointing out the importance of being aware of confirmation bias it has been said, "man isn't the rational animal, he is the rationalizing animal." (multiple attributions) In other words, rather than looking at all the available evidence, evaluating it, then forming a conclusion, we instead start with a conclusion then pick the evidence and arguments that support it--regardless of quality of evidence. There is a wealth of literature supporting this.
Anecdotal Evidence Vs Sample Size
Suppose I'd never seen a bird before but I'd decided to become a bird scientist. The first thing I choose to study is what color birds are. So, I decide to make some observations. In my first experiment, I look out my window to my back yard. I notice a blue-grey bird. I immediately call you on the phone with my results. "Hey! You wanna know what color all birds are?" "Sure" you reply. "They are blue-grey" I reply confidently.
Now, given my evidence, how strong is my inference from 1 bird is blue-grey to all birds are blue-grey? Not very, right?
Trying to be kind, you explain to me that I really don't have a large enough sample to infer a conclusion about all birds. "Fine," I reply, in a little bit of a huff. I'll show you!
I look out my window for a few more hours and I see not 1 but 3 more birds and they are all blue-grey! Ah! My hypothesis in confirmed 3 times! Every bird I've ever seen is blue-grey. What more evidence could you need for the conclusion that all birds are blue-grey? My hypothesis even predicted it!
So there are a couple of things you might notice. (1) My sample is probably too small for my conclusion and (2) I've only encountered evidence that confirms my conclusion--a consequence (in this case) of my limited sample size (3) my prediction isn't very "novel".
Application to CAM, Anti-GMO, Anti-Vaccine Proponents
Often I will encounter studies in my news feed that are purported to be evidence for one of the above anti-science positions. Just recently, a friend of mine jubilantly sent a pro-CAM study as support for his treatment modality of choice. Ah ha! look! Scientists wearing lab-coats did a study...now that scientifically proves I'm right! What my friend failed to consider was that the study supporting his position had only a sample size of 12. Do you think a sample size of 12 is enough to infer a general claim about the efficacy of a treatment?
If you're a CAM supporter, consider this. If I posted a study with a sample size of 12 showing the safety of GMOs or vaccines or some new drug from the world's biggest pharmaceutical company would you be like, "yep, this is some high quality evidence. I totally recant my position." (Hint: probably not).
So, what went wrong? Why is it that for a conclusion that you support, a sample size of 12 "scientifically proves" the efficacy of the treatment while the same sample size doesn't prove anything if it's for a treatment you're ideologically opposed to? Hint: Cognitive bias.
As mentioned in many of my posts (and implied in this one), unless we are very careful, we will inevitably make cognitive errors--we're hard-wired for it. What's happened is that all we care about is whether something supports or disagrees with our pre-existing position. We're not interested in evaluating the quality of that evidence. We've all got mini George Dubya Bush's in us: We only care about if it's with us or against us. If it's with us, then it's ipso facto good evidence, if it's against us it's ipso facto bad or invisible evidence.
Bringing it Back to Science
Good science doesn't give a crap about who's right or who's wrong because science isn't committed to any position--it is merely a method of investigation. Contrast this with CAM proponents. There is an ideological commitment to a given modality regardless of quality of evidence.
If you are a proponent of CAM, ask yourself honestly, how much and what kind of evidence would you need to see before you believed that vaccines and conventional medication are safe and effective? If you are truly committed to the method of science, rather than the religion of your favored modality, the answer should be the same for both CAM modalities and conventional...If you do that, now you can science too!
Showing posts with label pseudoscience. Show all posts
Showing posts with label pseudoscience. Show all posts
Friday, October 18, 2013
Saturday, January 28, 2012
Thaaaats Not Science: Part 2
Notes and Thoughts on Imre Lakatos' 'Science and Pseudoscience'
My favourite quote:
"But the history of thought shows us that many people were committed to absurd beliefs. If the strength of beliefs were a hallmark of knowledge, we should have to rank some tales about demons, angels, devils, and of heaven and hell and knowledge. Scientist, on the other hand, are very skeptical of even their best theories."
Ok, lets get this out of the way right off the bat: I don't know if Imre Lakatos had an affinity for drinking milk.
The Demarkation Problem Revisited
Moving on to the demarkation problem and Lakatos' proposed solution. Here's the short version: A theory is scientific if it predicts novel facts and it's pseudo-scientific if its predictions are consistently refuted and it produces no new facts about the world. To see how he arrives at this position lets first look at his arguments against the competing demarkation theories including Popper's falsificationism and Kuhn's idea that science is marked by incorporating puzzle solving. Kuhn's demarkation line was drawn between enterprises that embrace unsuccessful predictions as research puzzles to be solved (sciences) and those that resort to hand waving to explain away unsuccessful predictions and do not incorporate the problems as part of their research (pseudoscience).
Vs. It's Science Because It's Derived from the Facts
The first argument is directed at the popular notion that what makes a theory scientific is that it is derived from the facts. But what exactly does it mean for a scientific theory to be supported by the facts? That Newton was doing science is pretty much indisputable, but did Newton derive his theory of motion from the facts? Newton claimed that he had deduced his theories from the facts of planetary motion which had been observed by Kepler. Kepler's observations were that the planets' orbits were elliptical but according to Newton's theory, planets would only follow such a path if they didn't disturb each other, which they do. Doh! So much for the facts. So, Newton devised a perturbation theory which holds that no planet moves in an ellipse.
The idea here is that facts are facts and theories are theories. Got it? A theory is an interpretation of facts, a story linking events together, if you will. There can be many different interpretations of the same set of facts, all of them conforming to the facts, no matter how wacky the the theory. There is also this idea that no theory takes into account every fact. It not uncommon to gloss over particular facts and/or outliers (events that don't conform with what we'd expect) which may be problematic for the theory.
Vs. Popper's Falsificationism (see: 'Thaaaat's Not Science: Part 1')
The prollem with falsificationism (for a full list see part 1) is that scientists don't throw away theories the moment a fact contracts it or doesn't conform to a prediction. Scientist will cling to theories for a long time before they are discarded. Disconfirming evidence is called an anomaly. Sometimes it's ignored for statistical reasons, sometimes the theory is modified to capture it. The point is, rarely is the whole theory thrown out for the sake of mere disconfirming evidence.
If scientists, like their pseudoscientist counterparts, don't reject theories based on non-conforming evidence, what makes one theory science and another pseudoscience?
Lakatos' Solution to the Demarkation Problem
Step one. We can have lots of fun. And by fun I mean, instead of considering individual hypotheses and theories to be the basic unit of scientific achievement, we should consider the unit to be "a research programme". Fun right? Science isn't simply a matter of trial and error, this would be trivial. A research programme is something that encompasses several interconnected theories and hypotheses.
Consider Newtonian physics. It is more that its four laws. These four laws form the core of the Newtonian program but are "protected from refutation by a vast 'protective belt' of auxiliary hypotheses". On top of all this, a programme has a problem solving method which includes mathematical and statistical techniques to interpret evidence and anomalies. If a planet doesn't move according to what the theory predicts, the scientist can check his hypotheses concerning atmospheric refraction, propagation of light in magnetic storms, and hundreds of other things that are all contained within the programme.
Newton's theory of physics, Einstein's relativity theory, quantum mechanics, evolutionary biology, homoeopathy, "The Secret", and alchemy are all research programmes that have a core and protective belt of elaborate problem-solving machinery. At every stage in their development there is disconfirming evidence, anomalies; there are problems. But how do we separate the scientific programmes from the pseudoscientific?
Step 2, I'll explain it to you. Scientific programmes are all able to predict novel facts, "facts which had been either undreamt of, or have indeed been contradicted by previous rival programmes." What are some examples of novel facts? Einsteinian relativity predicted that the path of light will be altered by objects with large mass. That's why light seems to bend when I flex my biceptors. I kid! I kid! Einstein's prediction was verified during a solar eclipse. If you're curious about the details, click on these pre-googled links:
http://en.wikipedia.org/wiki/Tests_of_general_relativity
http://www.youtube.com/watch?v=IVHQH4UPIFs
Another example is Halley's prediction, within the Newtonian programme, that the comet that bares his name (co-incidence or what?!) would return in 72 years. He died before it did, but his prediction didn't! Before Halley, er'body thought comets only travelled in straight lines but from within the Newtonian programme he was able to shew that some move in hyperbolas or parabolas, and others in ellipses.
So, what separates pseudo-scientific programmes from the scientific? The scientific programmes make predict previously unknown facts; they lead to further discovery; discovery that competing theories could not make. In a word, they are progressive.
Pseudo-science, on the other hand, is degenerative. Theories are retrofitted and modified only to accomodate existing facts. Has homoeopathy predicted any novel facts about the world? How about the programme of The Law of Stupidity, oops, Attraction? Nope. What people who work within these programmes do is reinterpret existing facts in such a way so that they will conform with their pre-existing model. Nothing novel about the world ever emerges.
"What really count are dramatic, unexpected, stunning predictions; a few of them are enough to tilt the balance; where theory lags behind facts, we are dealing with miserable degenerating research programmes."
My favourite quote:
"But the history of thought shows us that many people were committed to absurd beliefs. If the strength of beliefs were a hallmark of knowledge, we should have to rank some tales about demons, angels, devils, and of heaven and hell and knowledge. Scientist, on the other hand, are very skeptical of even their best theories."
Ok, lets get this out of the way right off the bat: I don't know if Imre Lakatos had an affinity for drinking milk.
The Demarkation Problem Revisited
Moving on to the demarkation problem and Lakatos' proposed solution. Here's the short version: A theory is scientific if it predicts novel facts and it's pseudo-scientific if its predictions are consistently refuted and it produces no new facts about the world. To see how he arrives at this position lets first look at his arguments against the competing demarkation theories including Popper's falsificationism and Kuhn's idea that science is marked by incorporating puzzle solving. Kuhn's demarkation line was drawn between enterprises that embrace unsuccessful predictions as research puzzles to be solved (sciences) and those that resort to hand waving to explain away unsuccessful predictions and do not incorporate the problems as part of their research (pseudoscience).
Vs. It's Science Because It's Derived from the Facts
The first argument is directed at the popular notion that what makes a theory scientific is that it is derived from the facts. But what exactly does it mean for a scientific theory to be supported by the facts? That Newton was doing science is pretty much indisputable, but did Newton derive his theory of motion from the facts? Newton claimed that he had deduced his theories from the facts of planetary motion which had been observed by Kepler. Kepler's observations were that the planets' orbits were elliptical but according to Newton's theory, planets would only follow such a path if they didn't disturb each other, which they do. Doh! So much for the facts. So, Newton devised a perturbation theory which holds that no planet moves in an ellipse.
The idea here is that facts are facts and theories are theories. Got it? A theory is an interpretation of facts, a story linking events together, if you will. There can be many different interpretations of the same set of facts, all of them conforming to the facts, no matter how wacky the the theory. There is also this idea that no theory takes into account every fact. It not uncommon to gloss over particular facts and/or outliers (events that don't conform with what we'd expect) which may be problematic for the theory.
Vs. Popper's Falsificationism (see: 'Thaaaat's Not Science: Part 1')
The prollem with falsificationism (for a full list see part 1) is that scientists don't throw away theories the moment a fact contracts it or doesn't conform to a prediction. Scientist will cling to theories for a long time before they are discarded. Disconfirming evidence is called an anomaly. Sometimes it's ignored for statistical reasons, sometimes the theory is modified to capture it. The point is, rarely is the whole theory thrown out for the sake of mere disconfirming evidence.
If scientists, like their pseudoscientist counterparts, don't reject theories based on non-conforming evidence, what makes one theory science and another pseudoscience?
Lakatos' Solution to the Demarkation Problem
Step one. We can have lots of fun. And by fun I mean, instead of considering individual hypotheses and theories to be the basic unit of scientific achievement, we should consider the unit to be "a research programme". Fun right? Science isn't simply a matter of trial and error, this would be trivial. A research programme is something that encompasses several interconnected theories and hypotheses.
Consider Newtonian physics. It is more that its four laws. These four laws form the core of the Newtonian program but are "protected from refutation by a vast 'protective belt' of auxiliary hypotheses". On top of all this, a programme has a problem solving method which includes mathematical and statistical techniques to interpret evidence and anomalies. If a planet doesn't move according to what the theory predicts, the scientist can check his hypotheses concerning atmospheric refraction, propagation of light in magnetic storms, and hundreds of other things that are all contained within the programme.
Newton's theory of physics, Einstein's relativity theory, quantum mechanics, evolutionary biology, homoeopathy, "The Secret", and alchemy are all research programmes that have a core and protective belt of elaborate problem-solving machinery. At every stage in their development there is disconfirming evidence, anomalies; there are problems. But how do we separate the scientific programmes from the pseudoscientific?
Step 2, I'll explain it to you. Scientific programmes are all able to predict novel facts, "facts which had been either undreamt of, or have indeed been contradicted by previous rival programmes." What are some examples of novel facts? Einsteinian relativity predicted that the path of light will be altered by objects with large mass. That's why light seems to bend when I flex my biceptors. I kid! I kid! Einstein's prediction was verified during a solar eclipse. If you're curious about the details, click on these pre-googled links:
http://en.wikipedia.org/wiki/Tests_of_general_relativity
http://www.youtube.com/watch?v=IVHQH4UPIFs
Another example is Halley's prediction, within the Newtonian programme, that the comet that bares his name (co-incidence or what?!) would return in 72 years. He died before it did, but his prediction didn't! Before Halley, er'body thought comets only travelled in straight lines but from within the Newtonian programme he was able to shew that some move in hyperbolas or parabolas, and others in ellipses.
So, what separates pseudo-scientific programmes from the scientific? The scientific programmes make predict previously unknown facts; they lead to further discovery; discovery that competing theories could not make. In a word, they are progressive.
Pseudo-science, on the other hand, is degenerative. Theories are retrofitted and modified only to accomodate existing facts. Has homoeopathy predicted any novel facts about the world? How about the programme of The Law of Stupidity, oops, Attraction? Nope. What people who work within these programmes do is reinterpret existing facts in such a way so that they will conform with their pre-existing model. Nothing novel about the world ever emerges.
"What really count are dramatic, unexpected, stunning predictions; a few of them are enough to tilt the balance; where theory lags behind facts, we are dealing with miserable degenerating research programmes."
Subscribe to:
Posts (Atom)