"Erroneous analyses of interactions in neuroscience: a problem of significance," a paper in Nature Neuroscience by Sander Nieuwenhuis and co, points out an important and fatal statistical error common to many peer-reviewed neurology papers (as well as papers in related disciplines). Of the papers surveyed, the error occurred in more than half the papers where it could occur. Ben Goldacre explains the error:
Let’s say you’re working on some nerve cells, measuring the frequency with which they fire. When you drop a chemical on them, they seem to fire more slowly. You’ve got some normal mice, and some mutant mice. You want to see if their cells are differently affected by the chemical. So you measure the firing rate before and after applying the chemical, first in the mutant mice, then in the normal mice.
When you drop the chemical on the mutant mice nerve cells, their firing rate drops, by 30%, say. With the number of mice you have (in your imaginary experiment) this difference is statistically significant, which means it is unlikely to be due to chance. That’s a useful finding which you can maybe publish. When you drop the chemical on the normal mice nerve cells, there is a bit of a drop in firing rate, but not as much – let’s say the drop is 15% – and this smaller drop doesn’t reach statistical significance.
But here is the catch. You can say that there is a statistically significant effect for your chemical reducing the firing rate in the mutant cells. And you can say there is no such statistically significant effect in the normal cells. But you cannot say that mutant cells and mormal cells respond to the chemical differently. To say that, you would have to do a third statistical test, specifically comparing the “difference in differences”, the difference between the chemical-induced change in firing rate for the normal cells against the chemical-induced change in the mutant cells.
Beatboxer Tom Thum had ENT doctor and laryngeal surgeon Dr Matthew Broadhurst shine an endoscopic camera down his throat while beatboxing: “I wanted to find out how my larynx functioned when beatboxing compared to how it functions normally with speech, and whether or not there were any abnormalities in my laryngeal anatomy. I also had […]
Gastric bypass surgery is remarkably effective at promoting weight-loss (it cuts the long-term risk of early death from morbid obesity by 40%), and it’s long been presumed that the major action by which it worked was that, by bypassing the parts of the gut where most food absorbtion takes place, it limited the calories that […]
Timothy writes, “Diego Gómez is a Colombian conservation biologist. When he was a college student, he shared a single research paper online so that others could read and learn from it, just as he did. Diego was criminally prosecuted for copyright infringement, and faced up to 8 years in prison.”
While some people still maintain that everything in Apple’s walled garden “just works” and is immune to the rampant malware of the Windows world, the reality is different. The Mac’s growing market share has made it a much more viable target for malicious actors, and its built-in tools aren’t always enough to fix things. Drive […]
Boasting an IPX6 waterproof rating, the Trakk Bullet Ultra Compact Waterproof Bluetooth Speaker resists dust and heavy rainfall. It’s currently available in the Boing Boing Store.The Trakk Bullet offers the same wireless convenience as other portable speakers, but few are built as tough as this one. Its utilitarian construction is designed to be a totally low-maintenance […]
The Ticwatch 2 Active Smartwatch is a simpler take on an active wearable that raised over $2m dollars on Kickstarter and is currently offered in the Boing Boing Store.Somewhere in between the single-day battery life and platform-specificity of the Apple Watch and Android Wear devices, there exists the Ticwatch. Instead of trying to shoehorn another […]