"Erroneous analyses of interactions in neuroscience: a problem of significance," a paper in Nature Neuroscience by Sander Nieuwenhuis and co, points out an important and fatal statistical error common to many peer-reviewed neurology papers (as well as papers in related disciplines). Of the papers surveyed, the error occurred in more than half the papers where it could occur. Ben Goldacre explains the error:
Let’s say you’re working on some nerve cells, measuring the frequency with which they fire. When you drop a chemical on them, they seem to fire more slowly. You’ve got some normal mice, and some mutant mice. You want to see if their cells are differently affected by the chemical. So you measure the firing rate before and after applying the chemical, first in the mutant mice, then in the normal mice.
When you drop the chemical on the mutant mice nerve cells, their firing rate drops, by 30%, say. With the number of mice you have (in your imaginary experiment) this difference is statistically significant, which means it is unlikely to be due to chance. That’s a useful finding which you can maybe publish. When you drop the chemical on the normal mice nerve cells, there is a bit of a drop in firing rate, but not as much – let’s say the drop is 15% – and this smaller drop doesn’t reach statistical significance.
But here is the catch. You can say that there is a statistically significant effect for your chemical reducing the firing rate in the mutant cells. And you can say there is no such statistically significant effect in the normal cells. But you cannot say that mutant cells and mormal cells respond to the chemical differently. To say that, you would have to do a third statistical test, specifically comparing the “difference in differences”, the difference between the chemical-induced change in firing rate for the normal cells against the chemical-induced change in the mutant cells.
The EVATAR is a working model of a reproductive system made from a mouse ovary and bits of a human uterus, cervix, vagina, fallopian tubes, and liver. Developed by Northwestern University researcher Teresa Woodruff and her colleagues, the EVATAR is intended to help better test the effects of medicines and toxins on women. And now […]
Texas State University’s Body Farm (AKA Forensic Anthropology Center at Texas State University or FACTS) is a 45-year-old facility where the corpses of medical body donors are left to decompose so that researchers can observe the rate at which human remains are consumed by the elements, scavengers and microbes, allowing them to accurately date the […]
A better understanding how a sperm swims its way toward an egg could help inform new treatments for male infertility. Researchers from the University of York have now come up with a mathematical formula to model how large numbers of moving sperm interact with fluid they’re swimming through. From the University: By analysing the head […]
Maybe it’s entirely because of podcast ads, but drag-and-drop tools like Squarespace have gotten immensely popular in recent years. While it’s definitely a great tool for any non-coders who want to get a small website up and running quickly, managing content with a primarily visual interface can become a pain once you have more than […]
When you can’t wait for the world’s longest meeting to end, the mindless leg bouncing makes your boredom obvious and just annoys everybody else. Everyone knows the TPS reports need the damn cover sheet, but some sadistic colleague keeps forgetting, probably on purpose just to eat into your lunch hour. Enough is enough!While serving a […]
What could be more fun than a slingshot that shoots tiny airplanes? A slingshot that shoots tiny glowing airplanes of course! These toy planes are outfitted with ultra-bright LEDs, so you can fly all night without losing them in the trees.Whether you are a regular-sized child, or an overgrown adult one, these light-up flyers offer […]