Lots of social scientists use Prolific.co to conduct massive surveys, by paying randos — Amazon-Turk-style — to give their responses to prompts.
But this summer, they started to notice something was weird — they were suddenly getting a wildly disproportionate amount of responses from young women. (This researcher found 91% were women.) What was going on?
Welp, on July 23 a teenager named Sarah Frank had posted a TikTok video telling her viewers how they could make some extra dough by doing Prolific.co surveys — and it went viral, racking up 4.1 million views, mostly with young women.
Boom: They all headed over to Prolific.co, suddenly and decisively tipping the demographics of who was answering questions.
As the Prolific.co CTO Phelim Bradley told The Verge …
According to Bradley, about 4,600 studies were disrupted by Frank's TikTok, around a third of the total that were active on the platform during the surge. Of those, he said, the vast majority should be salvageable.
Prolific has refunded researchers whose studies were significantly impacted by the surge in women survey takers and introduced a new suite of demographic screening tools. The company announced these steps a month after Frank posted her video. The company has now re-organized, putting a team in charge of demographic balancing in order to more quickly recognize and respond to this sort of problem in the future.
"Honestly, we were somewhat caught by surprise, and we didn't predict how large the impact was going to be," Bradley said.
Frank's influence has some upsides, as some researchers note: The surge of TikTok young women is beginning to fade, but some will stick around in a way that might help diversify and invigorate the population of folks who tend to answer surveys online.
(Public-domain photo via Pxhere)
