"Erroneous analyses of interactions in neuroscience: a problem of significance," a paper in Nature Neuroscience by Sander Nieuwenhuis and co, points out an important and fatal statistical error common to many peer-reviewed neurology papers (as well as papers in related disciplines). Of the papers surveyed, the error occurred in more than half the papers where it could occur. Ben Goldacre explains the error:
Let’s say you’re working on some nerve cells, measuring the frequency with which they fire. When you drop a chemical on them, they seem to fire more slowly. You’ve got some normal mice, and some mutant mice. You want to see if their cells are differently affected by the chemical. So you measure the firing rate before and after applying the chemical, first in the mutant mice, then in the normal mice.
When you drop the chemical on the mutant mice nerve cells, their firing rate drops, by 30%, say. With the number of mice you have (in your imaginary experiment) this difference is statistically significant, which means it is unlikely to be due to chance. That’s a useful finding which you can maybe publish. When you drop the chemical on the normal mice nerve cells, there is a bit of a drop in firing rate, but not as much – let’s say the drop is 15% – and this smaller drop doesn’t reach statistical significance.
But here is the catch. You can say that there is a statistically significant effect for your chemical reducing the firing rate in the mutant cells. And you can say there is no such statistically significant effect in the normal cells. But you cannot say that mutant cells and mormal cells respond to the chemical differently. To say that, you would have to do a third statistical test, specifically comparing the “difference in differences”, the difference between the chemical-induced change in firing rate for the normal cells against the chemical-induced change in the mutant cells.
More than 800 American energy and Earth science researchers have signed a letter to Donald Trump outlining six steps they’re urging him to take to address human-caused climate change to protect “America’s economy, national security, and public health and safety.” The letter is accompanied by a public change.org petition to “Tell Trump To #ActOnClimate.” Here […]
James Delingpole is an invective-hurling anti-climate science columnist who has candidly admitted that he doesn’t bother to read scientific papers, calling himself a “an interpreter of interpretations.”
Kratom (previously) is a widely used herb that has been very effective in treating opioid withdrawal and other chronic, hard-to-treat conditions — it also became very controversial this year because the DEA decided, without evidence, to class it as a dangerous drug, and then changed its mind (unprecedented!) after a mass-scale petition that included interventions […]
The Boing Boing Store’s Gift Guide is full of ideas for pretty much anyone in your life like hipster ice cub trays, Xbox controllers, Halo Boards, and even diamond necklaces. As always, all products in the Boing Boing Store come at great discounts, too. Shop by price bucket starting at under $20. Under $20:Bloxx Jumbo Ice Trays […]
Unlike traditional lighters, the SaberLight features an electronic plasma beam that’s both rechargeable and butane-free. This sleek lighter is even approved by TSA, so you’ll never be stuck buying lighters you’ll just have to throw away partially used. For some people, like me, this is a pretty big game-changer. The SaberLight’s beam is actually both hotter and cleaner […]
Holiday shopping is in full swing, and the Striiv Touch is one of the best gift ideas I’ve landed on. Its simple design works for females and males, and its wide range of features makes it suitable for even the non-fitness enthusiasts in your life.Unlike traditional fitness trackers, the Striiv Touch also acts as a smartwatch. It […]