Illustration by Edel Rodriguez for PBS NewsHour
From PBS NewsHour's Jenny Marder, an interesting piece this week about the weird cyberworld of professional study subjects–and specifically, the role that Amazon's Mechanical Turk plays in academic research.
Jenny says,
Academic researchers are increasingly using this forum to collect data and recruit study subjects, and the studies, which pay well relative to other HITs, are such a draw for workers that they complete them again and again and again. We talk to workers who have completed as many as 40,000 academic studies.
Over time, these so-called "super Turkers" become extremely familiar with the models used in research. They know many of the questions, they know how to play the games, and they know when they're being manipulated. Which means, in many cases, the experiments no longer work on them.
Our question at the heart of the story is how reliable are the data these research subjects generate. And have the humans behind the machine essentially become machines?
On the other hand, these super Turkers may be more representative of the population than university undergraduates, the traditional go-to study subjects. And it turns out doing thousands of studies has some interesting effects on the worker that we hadn't anticipated.
We think this is a fascinating subject, and it just became more interesting the deeper we dug.
"The Internet's hidden science factory" [pbs.org]
Sarah Marshall makes about 50 percent of her income from Mechanical Turk, mostly doing research studies. She works at home while watching her son. Photo by Mike Fritz for PBS NewsHour.
