Phrenology (the fake science of predicting personality from the shape of your cranial bones) is like Freddy Kruger, an unkillable demon who rises from the grave every time some desperate huckster decides they need to make a few extra bucks. Read the rest

It looks like everyday folks -- recruited on Mechanical Turk, no less -- are better at predicting recidivism than the elite, secretive private-sector algorithms used by courts.

The software, COMPAS, has passed judgment on 1 million offenders since it was introduced in 1998. It uses 137 bits of data about offenders to predict their chance of recidivism, but its accuracy is dodgy: A Pro Publica investigation of its output suggests it isn't much better than coin toss, and also that it's racist, being nearly twice as likely to suggest blacks will reoffend than whites with comparable criminal records.

It would be nice to examine the algorithm to see how it's weighing the variables, but no dice: Northpointe, the firm that runs COMPAS, says it's a proprietary secret.

But now there's yet more evidence that COMPAS's algorithmic precision isn't great. Two Dartmouth college researchers did a shoot-out. They took 1,000 defendants and examined COMPAS' predictions for recidivism. Then they paid 400 people on Mechanical Turk to look at the defendants' files and make their own prediction. The researchers gave the turks only seven pieces of data about each defendant, none of which was race. The researchers also had the real-life data on whether the defendants really had, in fact, reoffended -- so they could see how well COMPAS stacked up against random folks on Amazon Turk.

The result?

The randos did better than the software. The turks' predictions were right 67% of the time; the software, 65%. As Wired reports:

"There was essentially no difference between people responding to an online survey for a buck and this commercial software being used in the courts," says Farid, who teaches computer science at Dartmouth.

Facewatch is a private-public system that shopkeepers and the police use to keep track of "persons of interest," a list that includes anyone a shopkeeper doesn't like and registers with the system. Read the rest

Kevin Norton and James Stumbo were arrested this weekend near the Pokémon World Championship after showing up with a 12-gauge shotgun and an AR-15 they boasted about on social media. Read the rest

In The Atlantic, Alexander Furnas debunks the DHS's proposal for a "precrime" screening system that will attempt to predict which passengers are likely to commit crimes, and single those people out for additional screening. FAST (Future Attribute Screening Technology) "will remotely monitor physiological and behavioral cues, like elevated heart rate, eye movement, body temperature, facial patterns, and body language, and analyze these cues algorithmically for statistical aberrance in an attempt to identify people with nefarious intentions." They'll build the biometric "bad intentions" profile by asking experimental subjects to carry out bad deeds and monitoring their vital signs. It's a mess, scientifically, and it will falsely accuse millions of innocent people of planning terrorist attacks.

First, predictive software of this kind is undermined by a simple statistical problem known as the false-positive paradox. Any system designed to spot terrorists before they commit an act of terrorism is, necessarily, looking for a needle in a haystack. As the adage would suggest, it turns out that this is an incredibly difficult thing to do. Here is why: let's assume for a moment that 1 in 1,000,000 people is a terrorist about to commit a crime. Terrorists are actually probably much much more rare, or we would have a whole lot more acts of terrorism, given the daily throughput of the global transportation system. Now lets imagine the FAST algorithm correctly classifies 99.99 percent of observations -- an incredibly high rate of accuracy for any big data-based predictive model. Even with this unbelievable level of accuracy, the system would still falsely accuse 99 people of being terrorists for every one terrorist it finds.