"Erroneous analyses of interactions in neuroscience: a problem of significance," a paper in Nature Neuroscience by Sander Nieuwenhuis and co, points out an important and fatal statistical error common to many peer-reviewed neurology papers (as well as papers in related disciplines). Of the papers surveyed, the error occurred in more than half the papers where it could occur. Ben Goldacre explains the error:
Let’s say you’re working on some nerve cells, measuring the frequency with which they fire. When you drop a chemical on them, they seem to fire more slowly. You’ve got some normal mice, and some mutant mice. You want to see if their cells are differently affected by the chemical. So you measure the firing rate before and after applying the chemical, first in the mutant mice, then in the normal mice.
When you drop the chemical on the mutant mice nerve cells, their firing rate drops, by 30%, say. With the number of mice you have (in your imaginary experiment) this difference is statistically significant, which means it is unlikely to be due to chance. That’s a useful finding which you can maybe publish. When you drop the chemical on the normal mice nerve cells, there is a bit of a drop in firing rate, but not as much – let’s say the drop is 15% – and this smaller drop doesn’t reach statistical significance.
But here is the catch. You can say that there is a statistically significant effect for your chemical reducing the firing rate in the mutant cells. And you can say there is no such statistically significant effect in the normal cells. But you cannot say that mutant cells and mormal cells respond to the chemical differently. To say that, you would have to do a third statistical test, specifically comparing the “difference in differences”, the difference between the chemical-induced change in firing rate for the normal cells against the chemical-induced change in the mutant cells.
“A sneezing monkey, a walking fish and a jewel-like snake are just some of a biological treasure trove of over 200 new species discovered in the Eastern Himalayas in recent years,” reports the World Wildlife Foundation today.
Three scientists won the world’s top science prize today, for their “mechanistic studies of DNA repair.” Their work mapped how cells repair deoxyribonucleic acid (DNA) to prevent damaging errors from appearing in genetic information. Tomas Lindahl, Paul L. Modrich and Aziz Sancar today received the Nobel Prize in Chemistry for “having mapped and explained how […]
Maciej Ceglowski (previously) spoke to a O’Reilly’s Strata Big Data conference this month about the toxicity of data — the fact that data collected is likely to leak, and that data-leaks resemble nuclear leaks in that even the “dilute” data (metadata or lightly contaminated boiler suits and tools) are still deadly when enough of them […]
Watching Netflix, Hulu or other streaming services can unfortunately be difficult while traveling outside the US. Rather than bypass these restrictions with the help of a complex and slow VPN, choose a faster and simpler solution with Getflix. Instead of rerouting all your Internet traffic through a different server, this handy service only routes the […]
Shake, stir, and muddle your way to delicious homemade cocktails with this must-have bar set. Expect only the finest quality tools from MakersKit — enabling you to unleash your inner mixologist.Top 12 Favorite Things of 2014, Sunset MagazineQuart-size vintage-style Mason jar shakerRetro double jigger for accurate measurementsStrainer & spouts for a mixologist-style smooth pourHardwood muddler […]
The Lytro Illum dares to be different, boasting even more robust features than its first generation predecessor and a sleek design reminiscent of professional DSLRs. What’s so cool about it? Most cameras capture the position of light rays, producing a statoc 2D image.