'Lies, Damn Lies and Statistics'
CAERS SUBSTACK ARTICLE #24
‘LIES, DAMN LIES AND STATISTICS’
CAERS SUBSTACK ARTICLE #24
The title of this article is a famous aphorism attributed to British Prime Minister Benjamin Disraeli, although Mark Twain made it popular. It highlights two very important ideas.
The first, is that in the last five hundred years or so since the start of the scientific revolution, we have relied heavily on the use of data to understand the universe. This is not a bad thing but it can lead us into becoming mesmerized by data and hence the sarcasm of the caption above.
The second is that it illustrates that data must be analyzed and interpreted in order for it to be helpful and not lead us astray.
In other words, although science does rely on the collection of data and statistics, there is another very crucial step involved in doing science well: thinking.
For example, if two vaccines of equal effectiveness were offered to you, vaccine ’A’ that produces side effects in 100% of those who take it, and vaccine ‘B’ that produces side effects in only 0.1% of those who take it, which would you choose?
It is tempting to almost immediately answer, ‘Well, vaccine ‘B’ with 0.1% of course!’ without even thinking. It’s a statistic, a piece of data, and it is easy to fall for it. But if I told you that the side effects for vaccine ‘A’ were limited to cold symptoms for three days and vaccine ‘B’ involved death of 0.1% of recipients, you might change your mind. Had I prefaced my question with the qualifier that their side effects were identical, then clearly ‘B’ would be the better choice. But I didn’t say that, and without that important little bit of information you could make a very bad decision. It illustrates the necessity of thinking and asking questions about raw data before coming to conclusions.
It's easy to get fooled by statistics. Another example: as far as we know, no human has ever died on the moon but billions have died on earth, therefore the moon must be safer for us than Earth. However, we must remember that as far as we know, there have only been twelve people who have spent time on the moon, and they were vigorously protected by a massive team that spent millions of dollars to assure their safety.
So often during the pandemic I have heard people quote statistics and say things like ‘the data doesn’t lie’ or ‘the data speaks for itself’. But that simply isn’t true. Good data is critical, but it is never the whole story. Before any good scientist collects data, they must make an observation first: apples always fall from a tree down to the earth not up to the sky. Then they ask themselves: why is that? Then they make a conjecture about why that is so, and then they develop a process for testing that conjecture. That process does not involve collecting data haphazardly, but rather doing so in a very purposeful fashion. If they don’t, then it will be too difficult to distinguish the signal from the noise. They must ahead of time know what data to collect that will either confirm their theory or falsify it. In other words, they have to do a lot of thinking both before and after data collection in order to come to accurate conclusions. Blindly believing statistics can be very dangerous, and hence Benjamin’s cynicism above.
Have you ever been misled by data and statistics? Have you ever felt that people have fooled you, innocently or otherwise, because they have used statistics to prove something to you that they wanted you to believe? Has that happened to you during the pandemic?
We don’t need to wait for mountains of data from ballistics experts and pathologists to know that the game of Russian roulette is dangerous because we are able to think for ourselves first. Thinking before collecting and analyzing data is absolutely necessary if we are going to do science well and learn more about how the universe works.
It would be very helpful for those presenting data to guide us as to how to think well about data and statistics rather than just presenting them to ‘prove’ their point. It may be true that statistics don’t lie, but that doesn’t mean that we cannot be fooled by them.
Think about that if you ever plan to vacation on the moon!
J. Barry Engelhardt MD (retired) MHSc (bioethics)
CAERS Health Intake Facilitator
Thanks for reading CAERS Newsletter! Subscribe for free to receive new posts and support my work.