404 Media investigated the phenomenon of Facebook AI slop—grossly unpleasant images from "Shrimp Jesus" to begging amputee children—and uncovered a grossly unpleasant fact: Facebook is paying people to make it.
"Facebook itself is paying creators in India, Vietnam, and the Philippines for bizarre AI spam that they are learning to make from YouTube influencers and guides sold on Telegram," writes Jason Koebler.
"The post you are seeing now is of a poor man that is being used to generate revenue," he says in Hindi, pointing with his pen to an image of a skeletal elderly man hunched over being eaten by hundreds of bugs. "The Indian audience is very emotional. After seeing photos like this, they Like, Comment and share them. So you too should create a page like this, upload photos and make money through Performance bonus."
He scrolls through the page, titled "Anita Kumari," which has 112,000 followers and almost exclusively posts images of emaciated, AI-generated people, natural disasters, and starving children. He pauses on another image of a man being eaten by bugs. "They are getting so many likes," he says. "They got 700 likes within 2-4 hours. They must have earned $100 from just this one photo. Facebook now pays you $100 for 1,000 likes … you must be wondering where you can get these images from. Don't worry. I'll show you how to create images with the help of AI."
Don't make me tap the sign: "Whatever you now find weird, ugly, uncomfortable and nasty about a new medium will surely become its signature." — Brian Eno
Churning it out is full-time work, not well-paid but enough to live on in the developing world, which guarantees an explosive secondary market of get-rich-quick grifters and tutorialists. Most of the traffic is from the U.S., all the same—presumably the people crudely derided as "boomers" who either don't know what they're looking at or are so immersed in the online shit vortex that they will never know anything again.
One former Meta employee with direct knowledge of its content moderation and ad approval systems, who spoke on the condition of anonymity because they signed an NDA at Meta, told me that Facebook is often aware of these loopholes but layoffs have left its content moderation teams so spread thin that they cannot actually keep up with how quickly people are exploiting them.
Koebler deduces that poor or partially-translated English prompts. mechanically-translated from other languages, can result in the extreme imagery that A.I. image generators now try to avoid producing.
Spammers use a mix of copy-paste AI image prompts in Microsoft's AI Image Creator. They get those prompts from influencers or buy them from other, more established spammers who are guessing what might go viral or what American audiences might engage with. Some of the bizarreness of the images is caused by prompts that are written in Hindi, Urdu, and Vietnamese, which are underrepresented in large language model training data. Other bizarre outcomes arise because people are using Google Translate and speech-to-text to say a prompt aloud in Hindi, translate it to English, and are using that often poorly-translated prompt to generate the images.
At the end of the algorthmic centipede? The advertisers tricked into paying for it all.
