I recently discovered one of the most entertaining web reads I’ve come across for ages … Bad Science which is a site dedicated to pointing out where the (mostly media) uses “Bad Science” or falls victim to “Bad Science”. The author (Ben Goldacre) is a medical doctor so most of the criticisms are in relation to medicine rather than science in general. But the debunking of rubbish media reports on (mostly) medical issues is worthwhile and done in an entertaining way.
It is interesting that many of the more foolish reports in the media have to do with bad statistics rather than bad science itself. That is I suppose not too surprising, as statistics seems to be widely misunderstood.
I have the advantage that many years ago I spent some time studying statistics, and many media reports have the effect of making my inner statistician jump up and down in fury shouting “Bullshit” over and over again. Fortunately he doesn’t shout too loudly or I’d run the risk of being shut away in a room with nice soft walls.
Statistics don’t lie, but they don’t always say what we think they do
I’m going to make use of an example relating to cannabis and an article publish by that paragon of excellent and accurate reporting, the Daily Mail. The article itself is here … scary isn’t it?
A report that has statistics that says that people who smoke cannabis have a 41% higher risk of schizophrenia, indicates that cannabis smokers are more likely to have schizophrenia than the general population. That doesn’t mean that cannabis causes mental health issues; that is an untested hypothesis. A quick uneducated guess at a number of possible reasons why includes :-
- Cannabis use increases the risk of mental health problems (yes it is possible).
- People with mental health problems are more likely to use cannabis than others.
- Cannabis use makes existing mental health problems worse.
- There is no link between cannabis use and mental health problems; the correlation is accidental.
- The study that found a correlation between cannabis use and mental health problems is flawed and there is in fact no such correlation.
One of the biggest mistakes anyone can make with statistics is to take a link between two variables (a correlation) and assume that one variable causes another (cannabis use causes mental health issues). This is known as “Correlation does not imply causation”; stealing a Wikipedia example, there is a correlation between going to bed with shoes on and waking up with a headache. Sleeping with shoes on does not cause headaches, but drinking copious quantities of alcohol makes it more likely that you will sleep with your shoes on, and far more likely you will wake with a headache.