When I give a talk, it's often the case that the question period after is dominated by dudes, a substantial fraction of whom bloviate ("More of a comment than a question," "It's a two-part question") rather than ask questions. Read the rest
How did a bug like krack fester in WPA2, the 13-year-old wifi standard whose flaws have rendered hundreds of millions of devices insecure, some of them permanently so? Read the rest
Robbo writes, "A number of so-called scientific journals have accepted a Star Wars-themed spoof paper. The manuscript is an absurd mess of factual errors, plagiarism and movie quotes. We know this because Neuroskeptic wrote it and posted about it on the Discover Magazine site. The paper was about Midi-chlorians and attributed to Dr Lucas McGeorge and Dr Annette Kin. Neuroskeptic takes us through the process used to create the bogus paper and the varied repsonses from the half-assed 'peer review' journals who accepted the work." Read the rest
Brian Wansink is one of the most-cited nutrition researchers in the world; 30,000 US schools use his advice to design their lunch programs, drawing on studies he's done that show that kids eat more carrots when they're called "X-ray vision carrots" and that putting out fruit bowls improves eating habits, and that smaller plates reduce portion sizes. Read the rest
Psychologist Gert Storms doesn't want to review scientific papers if their authors refuse to share with him the underlying data. The American Psychological Association (APA), which publishes the journal he edits, has asked him to resign.
Nature.com's Gautam Naik reports that the effort to force him out is a test of The Peer Reviewer's Opennness Initiative, a move crafted to "increase transparency in a field beset by reports of fraud and dubious research."
Read the restStorms, a psychologist at the Catholic University of Leuven in Belgium and a consulting editor for the APA’s Journal of Experimental Psychology: Learning, Memory, and Cognition, accepted an invitation last year to review a study for the journal, and pointed out his new open-data policy. The journal's editor, Robert Greene, wrote back to say that Storms’s stance set “a terrible precedent” because it was unfair to the author of the paper and opposed the APA’s policies and the guidelines followed by other reviewers. “Given that your policy conflicts with that of the journal, I think that it's best that you step down from the editorial board,” he wrote.
Storms refused, writing that he would continue to do what he thought was necessary to “prevent sloppy science”. And he forwarded his correspondence to other editors at the journal. Two of them, Robert Hartsuiker and Marc Brysbaert, both psychologists at Ghent University in Belgium, wrote to Greene saying that they, too, would quit if Storms was forced to resign. "The policy of asking people to leave rather than inviting a discussion and getting critical voices — I found that quite inappropriate," said Hartsuiker.
Researchers have taken a second look at the NSA SKYNET leaks, as well as the GCHQ data-mining problem book first published on Boing Boing, and concluded that the spy agencies have made elementary errors in their machine-learning techniques, which are used to identify candidates for remote assassination by drone. Read the rest
UPDATE: After you read this story, make sure you check out the follow up piece. Editors at Embargo Watch have found evidence that The Sustainable Food Trust manipulated the media to prevent public criticism of this paper.
Yesterday, in an aside to a post criticizing an astroturf political campaign in California, Mark mentioned a new study that supposedly found GM corn causes tumors in rats. As Mark said in an update to that post, this study is severely flawed, but I wanted to follow-up on that with some discussion about why it's flawed.
After all, the study was peer-reviewed, right? Doesn't that mean we can trust it?
Here's the thing. Peer review is not perfect. It's not a panacea. It's simply the basic level of due diligence. By submitting work for peer review, a scientist has allowed people outside her own team to critique her work. And the journal might require some changes to the paper based on the critique — anything from edits for clarity to requesting that the scientist perform another experiment in a different way. If a paper hasn't gone through peer review, you should be more skeptical of it. Avoiding peer review means that the researcher decided to show the public her results before allowing those results to be critiqued by independent experts.
But, at the same time, just because something has gone through peer review doesn't mean it's been certified to be accurate. It just means that roughly three other experts have looked at the paper before publication. Read the rest
The two papers documenting evidence that CERN has found a particle matching the description of the Higgs Boson have cleared peer review and are now published in the journal Physics Letters B. Ironically, that journal is the offspring of Physics Letters, the journalwhich rejected Peter Higgs' 1964 paper that first hypothesized the existence of the Higgs Boson. Higgs' paper was eventually published by a different journal, Physical Review Letters. You can read it online. (Many thanks to Rachel Courtland for the history, and to Jennifer Ouellette for the Higgs paper link.) Read the rest
Remember how scientists discovered alien-esque life forms in California and the Internet was all, "Oh, sheeeet!" But then other scientists started critiquing the research and there was a giant debate about whether one scientist could call out another scientist for bad data on a blog, rather than in a peer-reviewed journal, except that the peer reviewed critiques basically said the same thing and the "discovery" turned out to be totally incorrect? I'm making light of arsenic life here just a bit, but this story of de-discovery continues to be interesting and important. Today, on NPR's Science Friday, science journalist extraordinaire Carl Zimmer will explain why, and will talk about what happens when scientists are wrong. Read the rest
Anything that inspires a good angry rant in real life can be turned into a Downfall video.
Getting a peer reviewed research paper through the aforementioned review process can be a stressful, rant-inducing experience. Remember, in order to be published, the paper is read by three (usually anonymous) reviewers who work in the same field of science. They judge things like whether the experiments described in the paper were done well enough, whether the work is original, and whether the take-away conclusions the scientist is presenting match up with the results of the experiments.
Last year, I wrote up a longer piece explaining peer review in more depth. Give it a read, and then see if you're surprised that there are multiple versions of peer review Hitler.
Above, Hitler is having problems with the third reviewer on his peer review board. Below the cut, Hitler's grant proposal is rejected by the National Institutes of Health. Read the rest
Peer-review does many things, but it isn't built to weed out fraud. In the wake of large scandals like the expose of Andrew Wakefield's fraudulent autism study, the British government is starting to consider regulating science for fraud the same way it regulates restaurants for public health. Brian Deer, the journalist who helped expose Wakefield, supports the idea. What do you think? (Via Ivan Oransky)
