"Erroneous analyses of interactions in neuroscience: a problem of significance," a paper in Nature Neuroscience by Sander Nieuwenhuis and co, points out an important and fatal statistical error common to many peer-reviewed neurology papers (as well as papers in related disciplines). Of the papers surveyed, the error occurred in more than half the papers where it could occur. Ben Goldacre explains the error:
Let’s say you’re working on some nerve cells, measuring the frequency with which they fire. When you drop a chemical on them, they seem to fire more slowly. You’ve got some normal mice, and some mutant mice. You want to see if their cells are differently affected by the chemical. So you measure the firing rate before and after applying the chemical, first in the mutant mice, then in the normal mice.
When you drop the chemical on the mutant mice nerve cells, their firing rate drops, by 30%, say. With the number of mice you have (in your imaginary experiment) this difference is statistically significant, which means it is unlikely to be due to chance. That’s a useful finding which you can maybe publish. When you drop the chemical on the normal mice nerve cells, there is a bit of a drop in firing rate, but not as much – let’s say the drop is 15% – and this smaller drop doesn’t reach statistical significance.
But here is the catch. You can say that there is a statistically significant effect for your chemical reducing the firing rate in the mutant cells. And you can say there is no such statistically significant effect in the normal cells. But you cannot say that mutant cells and mormal cells respond to the chemical differently. To say that, you would have to do a third statistical test, specifically comparing the “difference in differences”, the difference between the chemical-induced change in firing rate for the normal cells against the chemical-induced change in the mutant cells.
New research on trends in adult human height over the last century confirm that, no surprise, humans are getting taller overall due to better nutrition and disease control. However according to the health science group Non-Communicable Diseases Risk Factor Collaboration (NCD-RisC), the gain in adult height varies dramatically by country. From their paper, published in […]
Between April and July, Iran’s salty Lake Urmia changed from a bright green color to a blood red. NASA’s Aqua satellite captured the image above and reported on the science behind the strange transformation. According to NASA, the periodic color change is caused by micro algae producing carotenoids that help with photosynthesis and act as […]
In the late 1960s and 1970s, the mind-expanding modus operandi of the counterculture spread into the realm of science, and shit got wonderfully weird. Neurophysiologist John Lilly tried to talk with dolphins. Physicist Peter Phillips launched a parapsychology lab at Washington University. Princeton physicist Gerard O’Neill became an evangelist for space colonies. Groovy Science: Knowledge, […]
It’s one thing to enjoy dinner at home and a nice glass of Cabernet Sauvignon with your best friend, Netflix, but it’s another thing entirely to make that meal from scratch and get that wine delivered right to your doorstep.But what if we told you there’s a way to make this possible? To keep your social life, […]
Having to pack and drag your stuff through security can put quite the damper on your vacation plans. Thankfully, we’ve got your back with one way to make traveling more painless: the Jumper Overnighter Travel Bag.This compact bag is so lightweight that you can effortlessly carry it, and fit it into any overhead compartment. But just […]
Learning is a 24/7/365 proposition, and it never ends. And if you’re truly serious about leveling up your skill sets and career prospects, get a subscription to Stone River Academy’s massive course collection. This offer normally is worth over $1,400, but is now available for just $89 in the Boing Boing Store.A respected name in information technology […]