"Erroneous analyses of interactions in neuroscience: a problem of significance," a paper in Nature Neuroscience by Sander Nieuwenhuis and co, points out an important and fatal statistical error common to many peer-reviewed neurology papers (as well as papers in related disciplines). Of the papers surveyed, the error occurred in more than half the papers where it could occur. Ben Goldacre explains the error:
Let’s say you’re working on some nerve cells, measuring the frequency with which they fire. When you drop a chemical on them, they seem to fire more slowly. You’ve got some normal mice, and some mutant mice. You want to see if their cells are differently affected by the chemical. So you measure the firing rate before and after applying the chemical, first in the mutant mice, then in the normal mice.
When you drop the chemical on the mutant mice nerve cells, their firing rate drops, by 30%, say. With the number of mice you have (in your imaginary experiment) this difference is statistically significant, which means it is unlikely to be due to chance. That’s a useful finding which you can maybe publish. When you drop the chemical on the normal mice nerve cells, there is a bit of a drop in firing rate, but not as much – let’s say the drop is 15% – and this smaller drop doesn’t reach statistical significance.
But here is the catch. You can say that there is a statistically significant effect for your chemical reducing the firing rate in the mutant cells. And you can say there is no such statistically significant effect in the normal cells. But you cannot say that mutant cells and mormal cells respond to the chemical differently. To say that, you would have to do a third statistical test, specifically comparing the “difference in differences”, the difference between the chemical-induced change in firing rate for the normal cells against the chemical-induced change in the mutant cells.
Mathematician Stephen Wolfram and his company do a lot of consulting for Hollywood. But he doesn’t often do it on an urgent basis because a movie is about to shoot and they neglected to “tech the tech.” When I first started looking at the script for [Arrival], I quickly realized that to make coherent suggestions […]
Can you “hear” motion or light flashes? If so, according to new research from City University London, you may be experiencing a not-so-rare form of synaesthesia. Synesthesia is the fascinating neurological phenomenon whereby stimulation of one sense involuntarily triggers another sensory pathway. For example, a synesthete might taste sounds or hear colors. (In this study, […]
Scientists declared the ruby seadragon a new species in 2015, but that was based on dead specimens in a museum. Now though, Scripps Institution of Oceanography biologist Greg Rouse who led the team that originally discovered the species, managed to find two of the wonderful fish swimming around the Recherche Archipelago, off the south coast […]
You know as well as I that writing complex, long-long form text requires significant organization. You’re probably also well aware that Word just isn’t up to the task. That’s why I’m a huge fan of Scrivener, the software suite used by best-selling authors and technical writers alike.Scrivener is much more than another digital typewriter. With a […]
Looking to upgrade your weekend? Here are three randomly awesome products on my mind this week.#3 FRESHeBUDS Pro Magnetic Bluetooth EarbudsAs more and more phones and gadgets switch to Bluetooth-only compatibility, you’ll need to get Bluetooth headphones like the rest of us. I’ve been super impressed with these affordable magnetic headphones. Pull the magnetic earbuds apart to auto-connect […]
Traditional folding wallets are designed for paper bills—but these days, carrying cash is rarely a necessity. More often than not, I don’t carry cash at all. This Bogui Clik Wallet is the best answer I’ve found for avoiding the hassle of those tight-fitting credit card pockets.This attractive, minimalist wallet features a protective lip, so my cards don’t […]