Botwars vs ad-tech: the origin story of universal surveillance on the Internet

Maciej Cegłowski's posted another of his barn-burning speeches about the Internet's problems, their origins and their solutions (previously), a talk from the Fremtidens Internet conference in Copenhagen called "What Happens Next Will Amaze You."

Cegłowski traces the history of "clickfraud," the practice of using software bots to load, click on, and interact with ads in order to generate fraudulent income for unscrupulous "publishers" who run the bots. In order to defeat the advertisers' countermeasures, the bots grew incredibly sophisticated — watching videos, filling in forms, and otherwise impersonating legitimate traffic — and so did the countermeasures.

This created a full-blown ecosystem of ad-tech that aggressively monitored users and aggressively shoved ads into their face (as Cegłowski notes, it was considered fraud when a bot tricked an advertiser into thinking an ad had been seen, but it's totally legit for an advertiser to trick you into seeing an ad), and fraudbots that figured out how to get past the countermeasures.

Meanwhile, all this surveillance was changing the way ads were delivered. If you have a ton of data on readers and their activities across the Web, why not use that to target and re-target ads that follow them.

These databases have not yet been publicly breached, but they will rupture, because all databases rupture. The tracking data held by ad-tech companies is vaster and potentially more damaging than the data breached by Ashleymadison, the Office of Personnel Management, and other high-profile dumps.

Some people try to draw a distinction between government surveillance and corporate surveillance, but it's a distinction without a difference. The NSA and other spy agencies use advertising tracking cookies as a cheap and easy way of tracking individual users across multiple devices, locations and accounts.

That means that regulators who are supposed to be overseeing ad-tech companies are working for governments who depend on mass-scale ad-tech surveillance to make government spying cost-effective.

Cegłowski finishes his talk with a set of six proposals aimed at regulators, tech companies, publishers and users. I don't necessarily agree with all of them, but they're provocative and plausible, and let us know that all hope is not lost.

Ad fraud works because the market for ads is so highly automated. Like algorithmic trading, decisions happen in fractions of a second, and matchmaking between publishers and advertisers is outside human control. It's a confusing world of demand side platforms, supply-side platforms, retargeting, pre-targeting, behavioral modeling, real-time bidding, ad exchanges, ad agency trading desks and a thousand other bits of jargon.

Because the payment systems are also automated, it's easy to cash out of the game. And that's how the robots thrive.

It boils down to this: fake websites serving real ads to fake traffic for real money.

And it's costing advertisers a fortune.

Just how much money robot traffic absorbs is hard to track. The robots actually notice when they're being monitored and scale down their activity accordingly.

Depending on estimates, ad fraud consumes from 10-50% of your ad budget. In some documented cases, over 90% of the ad traffic being monitored was non-human.

So those profits to advertisers from mass surveillance—the fifteen to thirty percent boost in sales I mentioned—are an illusion. The gains are lost, like tears in the rain, to automated ad fraud.

Advertisers end up right back where they started,still not knowing which half of their advertising budget is being wasted. Except in the process they've destroyed our privacy.

The winners in this game are the ones running the casino: big advertising networks, surveillance companies, and the whole brand-new industry known as "adtech".

The losers are small publishers and small advertisers. Universal click fraud drives down the value of all advertising, making it harder for niche publishers to make ends meet. And it ensures that any advertiser who doesn't invest heavily in countermeasures and tracking will get eaten alive.

But the biggest losers are you and me.

What Happens Next Will Amaze You [Maciej Cegłowski/Idle Words]