Why the DHS's pre-crime biometric profiling is doomed to fail, and will doom passengers with its failures

In The Atlantic, Alexander Furnas debunks the DHS's proposal for a "precrime" screening system that will attempt to predict which passengers are likely to commit crimes, and single those people out for additional screening. FAST (Future Attribute Screening Technology) "will remotely monitor physiological and behavioral cues, like elevated heart rate, eye movement, body temperature, facial patterns, and body language, and analyze these cues algorithmically for statistical aberrance in an attempt to identify people with nefarious intentions." They'll build the biometric "bad intentions" profile by asking experimental subjects to carry out bad deeds and monitoring their vital signs. It's a mess, scientifically, and it will falsely accuse millions of innocent people of planning terrorist attacks.

First, predictive software of this kind is undermined by a simple statistical problem known as the false-positive paradox. Any system designed to spot terrorists before they commit an act of terrorism is, necessarily, looking for a needle in a haystack. As the adage would suggest, it turns out that this is an incredibly difficult thing to do. Here is why: let's assume for a moment that 1 in 1,000,000 people is a terrorist about to commit a crime. Terrorists are actually probably much much more rare, or we would have a whole lot more acts of terrorism, given the daily throughput of the global transportation system. Now lets imagine the FAST algorithm correctly classifies 99.99 percent of observations -- an incredibly high rate of accuracy for any big data-based predictive model. Even with this unbelievable level of accuracy, the system would still falsely accuse 99 people of being terrorists for every one terrorist it finds. Given that none of these people would have actually committed a terrorist act yet distinguishing the innocent false positives from the guilty might be a non-trivial, and invasive task.

Of course FAST has nowhere near a 99.99 percent accuracy rate. I imagine much of the work being done here is classified, but a writeup in Nature reported that the first round of field tests had a 70 percent accuracy rate. From the available material it is difficult to determine exactly what this number means. There are a couple of ways to interpret this, since both the write-up and the DHS documentation (all pdfs) are unclear. This might mean that the current iteration of FAST correctly classifies 70 percent of people it observes -- which would produce false positives at an abysmal rate, given the rarity of terrorists in the population. The other way of interpreting this reported result is that FAST will call a terrorist a terrorist 70 percent of the time. This second option tells us nothing about the rate of false positives, but it would likely be quite high. In either case, it is likely that the false-positive paradox would be in full force for FAST, ensuring that any real terrorists identified are lost in a sea of falsely accused innocents.

Homeland Security's 'Pre-Crime' Screening Will Never Work

(Image: Brockhaus and Efron Encyclopedic Dictionary, a Creative Commons Attribution (2.0) image from double-m2's photostream)

• [Deidzoeb]

  We interpret nervous behavior as signs that you are up to no good, so there’s nothing to worry about. Don’t be nervous, or else.

  • [Marja Erwin]

    A nightmare for survivors. Can you imagine being grabbed, and being strip-searched, if not rape-searched, for being triggered?

• [suburbanhick]

  Philip K. Dick was right. Again.

• [vsherbie]

   I can’t believe they even gave it the same name.  I know better than to expect government types to read, but summer blockbuster movies with Tom Cruise are pretty high profile.
  Maybe somebody at DHS is reading Dick for good ideas.

• [Paul Butler]

  stephen j gould would like to comment. 

  • [Ultan]

     Except that he’s dead.

• [benher]

  Ho ho! Perhaps they can drill a hole in my frontal lobe and simply catch the pre-crime liquid in a 3 oz. vial as it spills forth!

• [janusnode]

  More damage done by the failure to make learning Bayes’ Theorem mandatory at an early age…

  • [Ultan]

     It’s one of those things that takes a surprisingly high intelligence to get. I’ve seen a  laboratory director/ professor/chief of staff try to get it across to his fellow doctors, and most of them just couldn’t get it.

• [pKp]

  Way to fail at basic stat literacy, DHS. I’ve never had a math course past high school, and even I can understand why this is a horrible idea.

• [jwkrk]

  Maybe they should just jump immediately to phrenology and be done with it.

  • [suburbanhick]

    “Of course you’d say that! You’ve got the brainpan of a stagecoach tilter!”

• [sickkid1972]

  There is, of course, one very simple step they can take to reduce the number of false positives… just redefine the term “terrorist” to include anybody who sets foot in an airport,….. or train station,…… or sets foot outside their own home,…… or inside their own home.

  Problem solved.

• [RedShirt77]

  Why do people try to realize the nightmare scenarios that end in tragedy in books and movies?   I know the Apollo missions worked out better than “From Earth to the Moon”  But precrime, armed AI, and teaching the ways of the force to space slaves are all most definitely bad ideas.

• [namnezia]

  As someone with Tourette’s (and thus having involuntary tics and twitches) I can’t even begin to imagine what I’ll do to their “aberrant facial expression sensor”. Sigh.

• [blissfulight]

  When has common sense, much less scientific analysis ever been an impediment to the TSA?  

• [timquinn]

  The phrase, “What could possibly go wrong?” was invented for this scenario exactly.

• [Rock Hardwood]

  If yer standing in line at an airport waiting to get through screening and you know there’s some bio-sensor taking remote readings of your scrotum tightness, ya… I can see this not working very well.

  And it assumes this condition is a result of you preparing to act on your years of training at ‘Summer Camp’ in Pakistan…   not that you are late making a connection, or you just had a fight with the wife, or you are generally not feeling well. 

  Hundreds of reasons why the alarms would go off, and none of them having to do with you wanting to crash a plane.

  One more reason to hate flying.

  • [Dee]

    That may be part of the overall effect, if not outright strategy (which I’m beginning to suspect):  Get us accustomed to invasive humiliation and assuming positions of surrender and open, compliant and willing submission by making it implicit thru the function of the system that outliers and anyone outside a prescribed norm don’t belong because they might be dangerous.

• [William Perrin]

  the Victorians photographed a lot of criminals convinced that the new technology could reveal the criminal type.  an excellent collection of criminal photographs from 1871 to 1873 is online at the Tyne and Wear museum on Flickr commons 
  http://www.flickr.com/photos/twm_news/sets/72157625464218629/ 

• [Listener43]

  Let me guess, Cory. You have some “interesting” lumps on your head that you wish to hide. Am I getting warm?

• [Miriam Rubinoff]

  I can only imagine the alarms set off by the autistic or otherwise neurologically different, not to mention those who are just nervous about flying. Someone on his way to cheat on his spouse might set off the “going to do something wrong” meter, while a graduate of terrorist training camp who believes he is doing nothing wrong breezes right through. No way this can go wrong at all.

• [oasisob1]

  This is what I call a Dick move.

• [Dee]

  “…any real terrorists identified are lost in a sea of falsely accused innocents.”

  Yes.  Isn’t that what we have now?

  “Sir, you CAN ***NNOOTT*** bring that much toothpaste onto  an airplane!!!  You MUST discard it or go back out and put it in a checked bag!”  …Proceed with full carry-on bag search and commence the groping…  [I opted to not fly that day rather than endure the grope.]

  What a joke.  Security theater.  What a shame.

  This mess with false positives will certainly serve to compound that disaster.

  Looks like TSA can’t catch real terrorists and things are looking dire for their continued budget largesse … so, maybe they can impress by catching people who think like terrorists or about them or are just neurologically atypical.

  All this also smacks of an attempt to overcome the objections to TSA and its ilk ’round the globe that amount to pointing out the “TSA[Emporer]-has-no-clothes” and their post hoc magical thinking:  “See, since 9-11 & TSA, we haven’t had any [successful] terrorist attacks.”

  When this error and its attendant strained high-pitched rhetoric and smoke and mirrors is called out, it’s often in the form of questions like, “So, where are the terrorists TSA has caught attempting to get on aircraft?  Where?  Where are those TSA has caught or stopped?  Where?”

  We know, based on the “great catches” TSA publishes amid great hoopla and fanfare (fake hollow hand grenade or plastic knife shaped toy anyone?), that if TSA had anything even near a real catch of  any terrorist trying to get on a plane it would be all over every news outlet for weeks or months.

  Also with TSA’s haste to get into the news and limelight when the shoe bomber, the liquid scare and the underwear bomber “failed to thwart our security and safety systems” – as if they had anything to do with either the failures or stopping terrorist actions.

  So, just more theater, smoke and mirrors:  misdirection, illusion and ridicule.  All to get compliant masses reassured that Big Brother(s), TSA, DHS, USA is(are) watching over us.

  • [Dee]

    …Oh, and then there are all the contractors, lobbyists and a gigantic metastasizing graft attracted ecosystem that perpetuates DHS & TSA.  Is it really all about the money?

• [knappa]

  Plus, any real terrorist could take a valium/paxil cocktail.

  • [Jay Pasquantonio]

    “These are just a few of the images we’ve recorded. And you can see, it wasn’t what we thought. There’s been no war here and no terraforming event. The environment is stable. It’s the Pax. The G-23 Paxilon Hydrochlorate that we added to the air processors. It was supposed to calm the population, weed out aggression. Well, it works. The people here stopped fighting. And then they stopped everything else. They stopped going to work, they stopped breeding, talking, eating. There’s 30 million people here, and they all just let themselves die…About a tenth of a percent of the population had the opposite reaction to the Pax. Their aggressor response increased beyond madness. They have become…Well, they’ve killed most of us. And not just killed… they’ve done things…”

• [ocker3]

  Did the 9/11 terrorists look sweaty, out of place? Can’t basic acting training hide the tics they’re looking for?

  • [Jay Pasquantonio]

    Agreed. Assuming that someone who is in the mental state where they find it acceptable to plan & carry out mass murder will have the same psychological and physiological response in an airport as “normal” subjects asked to do “bad deeds” under experimental conditions is so inherently flawed, it boggles the mind.

• [anansi133]

  This is only flawed if you choose to believe that they’re actually trying to prevent terrorism with this stuff. If they were looking to cast a wider net, to catch people cheating on their spouses, to find folks who’ve cheated on their taxes or spit on the sidewalk, or have overdue library books… then sure, why not put everyone to the test?

  And if being examined like this doesn’t annoy you, or if you have enough money to buy your way out of the line, then you’re exactly the kind of person they’d like to see moving around the country freely.

  This was never about preventing terrorism, it’s only ever been about training brownshirts.