Andrew C. Chang and Phillip Li undertook a study of "67 papers published in 13 well-regarded economics journals" for the US Federal Reserve and attempted to replicate their conclusions, using the code and data-sets provided by the authors: in two-thirds of cases, they were unable to replicate the findings without help with the authors; with the help of authors, they were still only able to replicate 49% of the papers.
In this article, we attempt to replicate 67 papers from 13 well-regarded economics journals
using author-provided data and code replication files. Improving on existing work evaluating
the state of replication in economics, our sampling frame is broader across different journals
and covers a large number of original research articles. We replicate 22 of 67 papers (33%)
by using only the authors’ data and code files, and an additional 7 papers (for a total of
29 papers, 43%) with assistance from the authors. The most common cause of our inability
to replicate findings is that authors do not provide files to the journal replication archives,
which constitutes approximately half of our failed replication attempts (21 of 38 papers,
55%). Because we are able to replicate less than half of the papers in our sample, we
conclude that economics research is generally not replicable.
Is Economics Research Replicable? Sixty Published Papers from
Thirteen Journals Say ”Usually Not” [Andrew C. Chang and Phillip Li/Federal Reserve]
My most recent essay film, Visual Disturbances, premiered in the open access journal [in]Transition yesterday. This open access journal features peer reviewed academic video essays and showcases a wide variety of film and media analysis. Visual Disturbances uses some cutting-edge eye tracking visualizations to explore how film audiences both perceive and mis-perceive movies.
A peer-reviewed study conducted by a trio of Princeton and NYU political scientists and published in Science Advances systematically examined the proliferation of fake news in the 2016 election cycle and found that, contrary to earlier reports, disinformation did not get shared very widely, and that most of it was right-wing, and that the people […]
It's been ten years since the people of Juneau, Alaska succumbed to conspiracy theories and voted to ruin their kids' teeth by removing fluoride from the drinking water, and it shows.
Looking for a career in music behind the boards, either as a music producer or DJ? It’s a good bet that you’re going to be working with Ableton Live. Each new iteration of this powerful workstation gives the user more tools to create, and it’s just as well suited for the task of meticulous track […]
The graveyard of failed startups is littered with concepts that just got lost in translation. At its core, that’s what great front-end design is about: Making an app or website usable, translating its best ideas smoothly to the user. It’s a skill so broad there might be no one book or course that covers it […]
Robotics: It’s a field that used to exist only as science fiction. Now it’s science fact, and it’s not just a playground for MIT prodigies. Thanks to the ROS (Robot Operating System) framework, anyone willing to learn robotics can practice robotics. And the easiest way to learn ROS? The Complete Robotics eBook Bundle. Combined, the […]