Have you ever gotten the sense from watching the news that scientists seem to change their minds constantly about the health benefits of food substances like coffee, or that virtually everything is linked to cancer (until it's suddenly not)? Keeping track of current scientific findings can feel frustrating — or, worse yet, pointless — because of how quickly the received wisdom on these subjects seems to shift.
But in reality, the problem isn't that the scientific consensus changes so quickly on these subject. It's that the media, and sometimes scientists themselves, discuss new findings in a misleading and irresponsible fashion.
This 20-minute segment from Last Week Tonight, the political comedy and discussion show hosted by former Daily Show correspondent and writer John Oliver, does an outstanding, hilarious, and frequently profane job of describing this problem and illustrating how it happens and why it's so dangerous. (It also features a rare live-action cameo by H. Jon Benjamin, the voice actor behind the protagonists of animated sitcoms Archer and Bob's Burgers.) You can watch the segment in full below; we've also summarized some of its main points in Oliver's own words underneath the embedded video.
[PICTURE]
The big points:
On contradictory new studies regarding coffee's health effects: "Coffee today is like God in the Old Testament. It will either save you or kill you depending on how much you believe in its magic powers. And after a certain point, all that ridiculous information can make you wonder, 'Is science bullshit?' To which the answer is clearly 'no,' but there is a lot of bullshit currently masquerading as science."
On where bad science comes from: "Not all scientific studies are equal. Some may appear in less-than-legitimate scientific journals. Others may be subtly biased because of scientists feeling pressured to come up with eye-catching, positive results....There are all sorts of ways you can consciously or unconsciously tweak your studies. You can alter how long it lasts, or make your sample size too small to be reliable, or engage in something that scientists call p-hacking...which basically means collecting lots of variables, and playing with your data until you find something that counts as statistically significant, but is probably meaningless."
On the lack of sufficient follow-up studies: "There is no reward for being the second person to discover something in science. There is no Nobel prize for fact-checking."
On the significance of individual studies: "Scientists know not to attach too much significance to individual studies until they're placed in the much larger context of all the work taking place in that field. But too often, a small study with nuanced, tentative findings gets blown out of proportion when presented to us, the lay public...Some of this is on us, the viewing public. We like fun, poppy science that we can share like gossip, and TV producers know it."
On red flags (or the lack thereof) in science reporting: "Just because a study was industry-funded, or its sample size was small, or it was done on mice, doesn't mean it was automatically flawed. But it is something the media reporting on it should probably tell you about."
On the dangerous consequences of bad science reporting: "If we start thinking that science is a la carte, and if you don't like one study, don't worry, another will be along soon, that is what leads people to think that man-made climate change isn't real, or that vaccines cause autism, both of which the scientific consensus is pretty clear on...Science is, by its nature, imperfect. But it is hugely important. And it deserves better than to be twisted out of proportion into morning show gossip."
We've discussed the dangers of sloppy science reporting and the challenges of interpreting individual studies on this blog before. If you'd like to read more about this subject, check out this post about the useful Science blog, this post about our Common Misconceptions quiz, this post about the placebo effect, or this post about irrational health scares.