The Scientific Method Explained and Common Misunderstandings for the Religious and CAM Proponents

Introduction
My aim here is to talk about some popular misconceptions about what science is, what it does, and how to best interpret scientific findings.  First, I will speak in general terms, and finally I will look at how these principles apply to CAM (i.e. complementary and alternative medicine), anti-vaccine, anti-GMO, and pretty much any science denialist position. 

Before continuing, lets get the credibility part out of the way first.  Who dafuq does a philosopher think he is talkin' 'bout wut science is an' sh*t?  Well, without going too far into it, philosophical questions are by definition non-scientific questions.  So, if there are questions we can ask about science itself that can't be resolved scientifically (e.g., "what is science?"), then a philosopher is a least one of the people you'll want to talk to.  

"B.b.bbbb.b..bbut you aren't a scientist!" True.  But do you need to know how to ride a bike to understand how a bike works?  (That's a rhetorical philosophical question doubling as an argument by analogy for anyone keeping track).  Enough yakity yak, lets break this shit down.

Popular Misconception 1:  Science is just like a Religion
No.  No, it isn't.  Science is a method of seeking knowledge.  Science is not a body of facts.  The scientific method is (roughly) (a) postulate a hypothesis to account for a series of observed phenomena (b) perform controlled experiments that try to both confirm and falsify the hypothesis, (c) express, based on the available evidence, the degree of confidence in the hypothesis.  If after many confirmations, as well as rigorous attempts to falsify the hypothesis, the hypothesis still stands, the hypothesis is elevated to theory.  

If however, during the course of experiments and observations there is evidence to suggest a better explanation of the observed phenomena, then the old hypothesis is modified or rejected, and replaced by the new hypothesis.  The important point here is that scientific conclusions are provisional.  If a new batch of repeatable higher quality evidence falsifies the existing hypothesis, it is rejected and replaced/modified.

Consider some of the first giants of science: Boyle, Newton, Bore, Galileo, Darwin, and so on.  I don't think anyone could seriously disagree with the assertion that these guys were doing science.  But guess what? Their theories were all later shown to be wrong to varying degrees.  So what happened?  
Did scientists deny the new higher quality evidence because they wanted to "stick to the facts"?  Maybe this happened with some during the transition period between theories--before all the new evidence was available--but eventually what happened?  They changed their position based on the best available evidence.  

Some people point to this as a weakness of science.  "How we supposed to believe what science says is true if it keeps changing its mind?" they cry, tears streaming down their angelic faces.  But sensitivity to new and better evidence is precisely what makes science such a powerful tool.  If you don't adjust your beliefs to new and better evidence you are in the weak position.

This method of continual change and refutation of scientific hypotheses continues to this very day.  It is known as the self-correcting nature of science and it is at the heart of what science is.  If you follow any science website, podcast, or blog, revisions and rejections  of hypotheses based on new evidence is a large part of what you'll hear about.   Shit, after more than a century of wide acceptance, scientists rejected Newtonian physics!  Let me say that again so it sinks in:  they rejected Newtonian muther f*ckin' physics! Science is not committed to any body of facts (or theory, for that matter).  Now, lets contrast this with religion.

Religion is defined by a more-or-less rigorous adherence to a set of certain truths and/or a world view. These truths are typically not amenable to contradicting evidence.  There are no attempts to falsify the religion's dogmas.  There is no sensitivity to contravening evidence: it is either explicitly denied, ignored, trivialized, or met with charges of blasphemy or conspiracy.  At the heart of religion is a commitment to a set of truths.  This could be no more different than what defines science.  So, no.  Science is nothing like a religion.

Evaluating Evidence (For the Intelligent Lay-Person)
How many times a day do you get some post in your news feed about something curing cancer or ancient Tibetan ear therapy that cures autism?  Your first instinct is probably (hopefully) skepticism.  But then your friend posts a *study* done by real scientists in labcoats that supports the conclusion:  Now, you should believe! 

Aside on Cognitive Biases
Before going any further we need to talk a little bit about cognitive biases.  A cognitive bias is a mental shortcut or heuristic that is evolutionarily hardwired into your brain.  You don't do it consciously and it's hard to stop yourself from doing it.  It's automatic, like flinching when someone throws a ball at you. It takes training an conscious effort to overcome it.

Over the course of evolutionary history, these mental shortcuts led to more survival producing beliefs than non-survival producing beliefs.  That's how we ended up with them.  Unfortunately, the shortcuts don't always get it right.  (Arguably) they work best in the evolutionary conditions under which they were formed.  Maybe sometime in the future, we will evolve new heuristics for evaluating scientific studies posted in a facebook feed, but for now, we're stuck with the tools we've got.

The whole point of this spiel is to alert you to the most dominant cognitive bias: confirmation bias. Confirmation bias is when we only look at or only remember the evidence that supports our prior beliefs but ignore or trivialize evidence that contracts our prior belief.   In pointing out the importance of being aware of confirmation bias it has been said, "man isn't the rational animal, he is the rationalizing animal." (multiple attributions)  In other words, rather than looking at all the available evidence, evaluating it, then forming a conclusion, we instead start with a conclusion then pick the evidence and arguments that support it--regardless of quality of evidence.  There is a wealth of literature supporting this.

Anecdotal Evidence Vs Sample Size
Suppose I'd never seen a bird before but I'd decided to become a bird scientist.  The first thing I choose to study is what color birds are.   So, I decide to make some observations.  In my first experiment, I look out my window to my back yard.  I notice a blue-grey bird.  I immediately call you on the phone with my results.  "Hey! You wanna know what color all birds are?"  "Sure" you reply.  "They are blue-grey" I reply confidently.  

Now, given my evidence, how strong is my inference from 1 bird is blue-grey to all birds are blue-grey?  Not very, right?  

Trying to be kind, you explain to me that I really don't have a large enough sample to infer a conclusion about all birds.  "Fine," I reply, in a little bit of a huff.  I'll show you!

I look out my window for a few more hours and I see not 1 but 3 more birds and they are all blue-grey! Ah! My hypothesis in confirmed 3 times!  Every bird I've ever seen is blue-grey.  What more evidence could you need for the conclusion that all birds are blue-grey?  My hypothesis even predicted it! 

So there are a couple of things you might notice.  (1) My sample is probably too small for my conclusion and (2) I've only encountered evidence that confirms my conclusion--a consequence (in this case) of my limited sample size (3) my prediction isn't very "novel". 

Application to CAM, Anti-GMO, Anti-Vaccine Proponents
Often I will encounter studies in my news feed that are purported to be evidence for one of the above anti-science positions.  Just recently, a friend of mine jubilantly sent a pro-CAM study as support for his treatment modality of choice.  Ah ha! look! Scientists wearing lab-coats did a study...now that scientifically proves I'm right!  What my friend failed to consider was that the study supporting his position had only a sample size of 12.  Do you think a sample size of 12 is enough to infer a general claim about the efficacy of a treatment?  

If you're a CAM supporter, consider this.  If I posted a study with a sample size of 12 showing the safety of GMOs or vaccines or some new drug from the world's biggest pharmaceutical company would you be like, "yep, this is some high quality evidence.  I totally recant my position."  (Hint: probably not).

So, what went wrong?  Why is it that for a conclusion that you support, a sample size of 12 "scientifically proves" the efficacy of the treatment while the same sample size doesn't prove anything if it's for a treatment you're ideologically opposed to?  Hint:  Cognitive bias.  

As mentioned in many of my posts (and implied in this one), unless we are very careful, we will inevitably make cognitive errors--we're hard-wired for it.  What's happened is that all we care about is whether something supports or disagrees with our pre-existing position.  We're not interested in evaluating the quality of that evidence.  We've all got mini George Dubya Bush's in us:  We only care about if it's with us or against us.  If it's with us, then it's ipso facto good evidence, if it's against us it's ipso facto bad or invisible evidence.

Bringing it Back to Science
Good science doesn't give a crap about who's right or who's wrong because science isn't committed to any position--it is merely a method of investigation.  Contrast this with CAM proponents.  There is an ideological commitment to a given modality regardless of quality of evidence.  

If you are a proponent of CAM, ask yourself honestly, how much and what kind of evidence would you need to see before you believed that vaccines and conventional medication are safe and effective? If you are truly committed to the method of science, rather than the religion of your favored modality, the answer should be the same for both CAM modalities and conventional...If you do that, now you can science too! 

The Vaccine Debate Part 2: With Whom is the Debate and Should We Listen?

The Vaccine Debates Part 2: With Whom Is the Debate?

Intro

     As I've been looking more into the vaccination “debate” (I'm using quotes because amongst scientists and physicians there is no debate) I've discovered a wealth of literature. Because I am far from an expert in biochemistry I will not discuss the individual studies that compile the mountains of positive evidence for vaccines but will instead look at general plausibility of anti-vaccination claims.

Who are the Anit-Vaxers and Should we Listen?

     It might be instructive to take a quick look at who the anti-vaccination people are. Just like any group, there are many subgroups but there are some common traits. Some are suckers for the naturalist fallacy, some are anti-government activists, and some have an affinity for conspiracy theories. Most members of this group have a mistrust of what they call the “medical industrial complex”. I will grant that in the context of the profit based US medical system, there is warrant for healthy skepticism when considering the claims of large for profit drug companies and insurance companies. Where I feel the justification for this skepticism breaks down is when we consider the attitudes toward particular health policies of systems that are not profit driven. Consider that every single country that has some form of nationalized/socialized medicine has a national vaccination program. Coincidence? Not only that, but the WHO (World Health Organization) also recommends vaccines and has mountains of research demonstrating their safety (vs. the risk of not vaccinating).

http://www.who.int/immunization/policy/immunization_tables/en/index.html

     Lets pause for a second and think. I understand that conspiracy theories are exciting but is it really plausible that every scientist that works for every national healthcare plan and every WHO scientist has been bought out by big pharma to act as their shill? Really? I know what you're thinking...how else can you explain the fact that all these scientists churn out year after year of long term studies demonstrating vaccine efficacy? Um....may I humbly suggest that it might be because the research and its conclusions are valid? Is it really possible that big pharma is paying all these scientists to fudge their research? Lets suppose it's true. Where's the smoking gun? By now at least one disgruntled scientist or spouse of a scientist would have spilled the beans. Ah! I forgot. The media's in on in it too and are covering up the story. Right.

     So, lets say you're still not convinced. You poke your finger in my chest and your logical fallacy detector screeches “Argument from authority! Aruga! Aruga!” You are right to point out that an argument from authority is not a good argument but sometime appeal to authority is valid. The argument from authority as a logical fallacy usually only applies when you only appeal to an individual or a small group of “experts” like, say...appealing to an Andrew Wakefield study.

     When there is broad consensus amongst a field of experts, this is not an argument from authority; it is prevailing wisdom. A quick example: if your car has a problem and you take it to a hundred mechanics and 99 of them say there's a problem with the regulator but 1 says the problem is with the alternator, which are you going to get fixed? Add to that example that the other 99 mechanics say the lone mechanic is a hack. Who would you listen to? Please don't say that “well what if the 99 mechanics are being paid by the regulator industrial complex and the 1 mechanic is fighting the good fight”. I might have to hurt you.

     Lets apply the example to something less controversial (than vaccines, not regulators/alternators) like climate change or evolution. 100% of biologists support evolution and all but one or 2 climate science experts support the idea of anthropogenic climate change. Should we listen to theologists when it comes to evolution? Should we give equal weight to the views of the 1 or 2 dissenting climate scientists? Hopefully not.

     To be sure there is disagreement among the experts about interpretation of the data and the details, just as there are matters that are still unsettled. That's why research is ongoing. But none of these experts in their respective fields would deny the main tenants of the overarching theories: That evolution happened/is happening, that there is anthropogenic climate change and that the benefits of vaccines far outweigh the risks of not being vaccinated.

     Ok, so I know what you're thinking. “Science” has been wrong before, and those in the minority view were eventually vindicated. True. Sometimes it happens. But it takes extraordinary evidence to overturn mountains of confirmatory evidence for an overarching theory. And so far that evidence hasn't materialized. As the skeptics say “extraordinary claims demands extraordinary evidence.”

Next post I'll (finally) write about the basic science surrounding the vaccination question)