Over on Hackernoon, data scientist and "language nerd," Jeff Kao, has posted the results of a data analysis he did on Net Neutrality comments submitted to the FCC between April-October 2017. Using natural language processing techniques, he was able to look for suspicious patterns in the language used. What he found was alarming.
The first and largest cluster of pro-repeal documents was especially notable. Unlike the other clusters I found (which contained a lot of repetitive language) each of the comments here was unique; however, the tone, language, and meaning across each comment was largely uniform. The language was also a bit stilted. Curious to dig deeper, I used regular expressions to match up the words in the clustered comments:
It turns out that there are 1.3 million of these. Each sentence in the faked comments looks like it was generated by a computer program. A mail merge swapped in a synonym for each term to generate unique-sounding comments. It was like mad-libs, except for astroturf.
When laying just five of these side-by-side with highlighting, as above, it’s clear that there’s something fishy going on. But when the comments are scattered among 22+ million, often with vastly different wordings between comment pairs, I can see how it’s hard to catch. Semantic clustering techniques, and not typical string-matching techniques, did a great job at nabbing these.
Finally, it was particularly chilling to see these spam comments all in one place, as they are exactly the type of policy arguments and language you expect to see in industry comments on the proposed repeal, or, these days, in the FCC Commissioner’s own statements lauding the repeal.
Oh, and guess what? Of the 800,000 comments that Kao determined likely to be "organic," 99+% of them were pro-Neutrality.
More on Net Neutrality fuckery:
According to Wells Fargo, a "computer glitch" caused the improper denial of 870 loan modification requests, which led to 545 foreclosures in which Wells Fargo customers lost their homes; the bank is now offering those former homeowners -- some of whom saw the breakup of their marriages as the result of the stress of foreclosure […]
New York City's "marshal" service is a throwback to the Dutch colonial days; the 35 marshals are appointed by the mayor, draw no salary, and earn their livings by skimming a percentage off of the debts they collect, operating with impunity and reaching around the world.
China-watchers observed the rise-and-rise of Chinese premier Xi Jinping with caution and sometimes alarm, but also held out some hope that despite his authoritarian tendencies and thin skin, Xi was genuinely committed to rooting out the rampant corruption that has plagued the country since its rapid industrialization under Deng Xiaoping: the creation of an untouchable […]
When it comes to tech, smaller is better, and these items fit the bill both in terms of size and price. We’ve rounded up our favorite stocking-ready gadgets, most of which are already on sale – and you can take an additional 15% off any of them with the special code MERRY15. iPM 3-in-1 Fast […]
So you’ve got a good eye for pictures? We’ve got a good eye for deals. And this holiday, there are some solid deals out there for photographers. Check out some of our favorite recent discounts on gear, software, and e-learning for photogs of any experience. Gadgets RevolCam: The Multi-Lens Photo Revolution for Smartphones This […]
Take a scroll through any app marketplace and you’ll see that the doors are wide open for any game these days – and any game developer. Like any creation, virtual or analog, it all starts with an idea. And if you’ve got one of those, the Complete Unity Game Developer Bundle can walk you the […]