CIA threat-tracking technology is fascinating, creepy


60 Responses to “CIA threat-tracking technology is fascinating, creepy”

  1. Matt Herron says:

    You overuse the phrase “creeped out” where something like “threatens our rights and civil liberties” comes to my mind, but i think the point is made well, nonetheless.

  2. Cory Doctorow says:

    Here’s what I don’t get: how do they possibly validate this software, when they have such a statistically insignificant sample?

    Even assuming that there is one terrorist threat to Walt Disney World every month (a number that is probably two orders of magnitude higher than in reality), that’s one needle out of a haystack that contains more visitors than the entire UK sees in the same period. So they’re using the characteristics of retrospective positives (the ones they caught, anyway), and making prospective guesses.

    But the number of positives to study in the set are infinitesimal, and there’s no reason to believe that the indicators that were present in retrospect will be prospective indicators, too.

    So how do they prove that their software is doing *anything*? How do they know that the WDW is terrorist-free because it’s *terrorist-free* and not because of their modern-day extispicy?

    • Timothy Krause says:

      You’re just upset over the “harmless eccentrics . . . fascinated by amusement park infrastructure” line, amirite? :P

      Ooo, and thanks for extispicy! That’s some inauspicious haruspices if I ever did see any!

    • awjt says:

      My guess is they have true “knowns” about terrorists and criminals.  And they also have a list of “contrived knowns” where they GUESS that the guy who posts on a political internet forum and buys a one-off shipment of ammonium nitrate fertilizer goes into this particular watch bucket.  But the farmer who posts on FARMing internet forums and buys the same amount of fertilizer year after year… doesn’t go into that bucket.  There are different probabilities for each.

      They compile lists of many of these things.  And even if they are miniscule, they DO have probabilities attached to them, and A can be differentiated from B.

      Then, the lovely part of this whole thing, is to combine these disconnected prior probabilities using Bayes’ Theorem.

      If you trust your priors, you do not need a large sample size.  You just need to be very careful estimating your priors, and making sure that you have all the priors of interest.

      That’s what worries me.  What about the prior probabilities for which there ISN’T a handy bucket at the CIA?  The stuff they DIDN’T think of? I must admit that even though I’m usually regarded as fairly bright, I had never thought of crashing a full jetliner into a building… until it happened.

      Now I am thinking about our unprotected food and water supply…

    • Chesterfield says:

      There are also lots of connections between Thiel, Palantir, and HB Gary. I wouldn’t want anything to do with these people.

    • DeargDoom says:

      I work on a system somewhat similar to Palantir as it is used by PayPal but where the numbers of positives are presumed to be much smaller than in PayPal’s data set.

      To my knowledge we have a 100% false positive rate to date. So apart from investigating a large amount of false positives we also presumably missed all activity we were trying to detect.

    • travtastic says:

      Simple enough. If you can’t calibrate it, how do you prove it doesn’t work? It’s not a bug!

    • alconnolly says:

      They don’t validate anything. They make things up with hypothetical nonsense and then wow some government agency with a splashy presentation. Then someone write them a massive blank check! Kaching!!! I doubt it is very effective at anything except destroying civil liberties by throwing up huge numbers of false positives for every real one. How is the scenario did someone getting a speeding ticket flag the system? Only one possible answer, every single encounter with the law or a major institution flags you. It either cannot detect real threats or detects tens of thousands of fake threats for every real one.

    • mike says:

      The problem with your analysis is that you’re presuming they really are doing this to catch terrorists…

  3. Timothy Krause says:

    The rationalization by Thiel at the end of the article is particularly disturbing-disgusting. To wit, we can’t have another 9–11 because it “opened the door” to all kinds of civil-rights abuses and First Amendment–rollbacks: so we need to leave the door permanently open, and widen this opening all the time, so the government has proactive recourse to all kinds of surveillance technologies and practices—all these little abuses are necessary so the really big abuses that happened after 9–11 won’t happen again. Disgusting!

    Even worse was the lovemaking to “Dr. Karp.” I’m rilly rilly glad he’s got cool hair: wish he had a socially just vision of how to use technology along with that cool hair. “Libertarian Technofascist Has a Cold” would be a great title for this hackwork.

  4. Guest says:

    a program like Palantir has the potential to spot plots in the making with less hassle to the general public.

    See, I’ve always thought of unreasonable search to be more a civil rights violation than a hassle. But that’s just me, with my lofty ideals…. and view of John Adams house.

  5. That_Anonymous_Coward says:

    “The technology is based on a system developed by PayPal”

    And this explains why DDOSing a website got more of a response than destroying the economy, or police brutalizing protestors.

    Isn’t this just an offshoot of Arron Barrs faulty ideas of connecting the unconnectable?

  6. tomrigid says:

    The fellas what make this tech know exactly how creepy it is. They named it after the second creepiest thing in LOTR. They should just call the next one “Sauron, Unblinking Eye of Mordor” and be done with it.

  7. Why do something difficult, like paying salaries and training for skilled intelligence and police personnel when you can just buy some software because of a 10 minute sales presentation instead.

    Selling technology to the government to fight terrorism is the easiest buck since PT Barnum.  

  8. Roy Sablosky says:

    “Fikri” — the terrorist they caught — “isn’t real,” they admit. They’ve never caught such a terrorist. They have no evidence that even one such terrorist exists. It’s not that they have little data, Cory. They have none. The whole scenario depends entirely on lies.

  9. Arcanafex says:

    @tomrigid: Thank you! I was amazed no one had pointed out yet the Tolkien reference! However, I hope this means that the DHS is considering calling whatever permanent future operational offices they may have build Orthanc (or something similar). Seems appropriate considering they now intend to use the palantir.

  10. pjk says:

    They had me going up until “Fikri isn’t real.” Sweet, so your software works perfectly in a vacuum and in the absence of gravity and friction. What could possibly go wrong?? Also, as a person living outside the US/Europe, that PayPal reference is not a selling point, as I find PayPal EXTREMELY difficult to deal with. It ALWAYS cries fraud when I try to use my credit card through it. Basically, this system means us non-US-living people are going to have to deal with more of this kind of false positive, no-judicial-review bullshit:

  11. lavardera says:

    So the transit authority is saving the photos of every car that passes through every toll, everywhere? And every bank is saving every photo of every ATM transaction everywhere? And this software has access to it all on an ongoing basis?

  12. Guest says:

    >As the CIA analyst starts poking around on Fikri’s file inside of Palantir, a story emerges.

    I dislike allusions to fiction in my law enforcement.

  13. bob d says:

    What I don’t understand is what they’d do with their hypothetical “terrorist” (or, more accurately, person who is suspicious when their various actions are viewed selectively) in their example?  The police are swooping down on him to do what?  If all you have is a collection of acts that add up to something suspicious (but not, as far as you know, illegal), what can you do as a response?  Have him followed 24 hours a day (until he actually does something illegal)?  Pick him up and interrogate him, hoping he’ll cop to being a would-be terrorist?  Imprison him indefinitely without charges (since there’s no actual evidence)?
    Never mind that the software requires a surveillance state of breath-taking width to even begin to operate.  How did they know, in their example, that this Syrian guy was taking pictures but not riding the rides?  His activity on surveillance tapes?  That’s not the sort of surveillance that could be automated (at least not for the foreseeable future), and if you rely on paranoid citizens informing to the CIA, every dark-skinned person who visited Disneyland and took pictures would be turned in by some racist yahoo.  I fail to see how such a system wouldn’t be immediately overwhelmed by a flood of false positives.  Paypal already has problems with innocent behavior being flagged, and they’re only looking at a very limited range of activities; try to apply that to everything we do in the real world and the system would explode.

  14. brianary says:

    Let’s change “creepy” to “unreliably life-destroying”, maybe.

    Khalid El-Masri and Byron Sonne weren’t simply creeped out, after all.

  15. patrick dodds says:

    Meanwhile, everyone else waits for the penny to drop and for terrorism-profiteers to accept that, guess what, the world isn’t crawling with terrorists just itching to blow themselves and everyone else to hell. The penny will take a long time, though, while there is still a populace to scare, papers to sell and governments wanting to stand on the necks of the voters.

  16. Teirhan says:

    My girlfriend interviewed for a QA position at Palantir’s finance division.  Palantir is a place where, in her words, “the whole QA team was full of young, white male nerds who like to play beer pong.  On Fridays.  In the office.”

    Sounds like a fun place to work! Provided you’re a YWM anyways.

  17. the_dannobot says:

    I think it’s super weird that they chose to name their surveillance software after fictional objects that brought doom to anyone who used them.

  18. pitchspork says:

    So with all this massive creepy tech and bazigabytes of data required to feed it, we only catch Fikri because: he gets a speeding ticket.  I also like how they use “even the dullest analyst” in a sales pitch to the bosses of those alleged supersmart analysts. Nice one.

  19. Manny says:

    Could they have picked a more creep-inducing name than Palantir?

  20. bardfinn says:

    The Palantir were magical communication devices. They were used for good and are a kind of Elvish family heirloom.

    It is when Sauron acquired one that it became dangerous to use one — as it would open the user to the corrupting influence of the embodiment of evil.

    Since Sauron’s been defeated, the Palantir are completely safe to use now.

    • Antinous / Moderator says:

      The Palantir were magical communication devices. They were used for good and are a kind of Elvish family heirloom.

      They were made by Fëanor, and we all know where that inevitably leads.

    • Laroquod says:

      “It is when Sauron acquired one that it became dangerous to use one — as it would open the user to the corrupting influence of the embodiment of evil.”

      I guess the lesson is, the Palantir (both fictional and real) is only as good or as bad as who’s looking in it.

  21. Nadreck says:

    Well, I hope that they got permission from the Tolkien estate to use that name! Their copyright lawyers use the real thing to sniff out violations!

    I was part of a project to build some municipal police IT systems once.  We, of course, didn’t have access to the real databases so we had merry laffs entering management and each other into our test databases as pimps and serial killers.  Good clean fun: until we somehow managed to ship a copy of the test database with the product.

  22. Rohn Koester says:

    There is a troubling contradiction behind every national security story like this: that in the age before Big Data surveillance technology, we were essentially open to any malicious plot any villain wanted to carry out . . . and yet, Disney World wasn’t blown up, airplanes weren’t driven into the ground, and acts of domestic terrorism almost inevitably involved weirdos pulling guns on political leaders at close range.

    The post-9/11 assumption seems to be that desperate political enemies of the US weren’t clever enough to figure out any of this in our national security pre-history, but now they are, and we have to catch up before they destroy the republic. The clearer story is either that (a) post-WWII / pre-9/11 national security was much more sophisticated than we’re led to believe, and (b) the notion of foreign invaders eager to destroy the US is a dystopian fantasy cooked up by Cold War bureaucrats eager to loot the public treasury of trillions of dollars.

  23. scolbath says:

    I think the biggest miracle that Palantir brought off is paying a max salary of $127k to engineers who presumably have security clearances.  That’s crazy for the valley area, and unheard of in the DC area.

    And boy, they are playing the ‘crazy genius CEO’ role to the hilt.  Never learned to drive because he was too busy?  It takes what, two days to learn to drive?  I hope he washes his hands after he leaves the bathroom.

    • dnebdal says:

      Two days? Really? Remind me to never go near a road whereever you live.

      • scolbath says:

        Driver’s ed in my state was *6 hours* of practicum over six days accompanied by a matching amount of ‘classroom’ work [NB:  this was in 1983 - no idea if it is still true].  You could take the road test at any time thereafter.  So, yea, you could do it in 48h and probably pass.  And probably be as good as most people on the road…!

        • dnebdal says:

          Huh, that’s not much. I’m Norwegian, and we require this:
          * Basic theory course.: 17 hours. (A bit less if you’re over 25.)
          * Training lessons with a qualified instructor: At least one (to qualify for the next stage), though more are suggested.
          * Safety course (handling, closed circuit): 4 hours, plus one more hour with an instructor (to qualify for the next stage)
          * Safety course (on the road): Driving in the dark, safe passing, complicated environments, summary: 13 hours, a bit more is recommended.
          * Theory test (modestly hard; 2 weeks waiting time if you fail)
          * Driving test (quite hard, takes about an hour, some rescheduling delay if you fail.)

          Practically speaking this takes some months, and a fair bit of money.

  24. Marc45 says:

    We got a call from an FBI guy at my company a while ago.  It was clear that he was alerted by their database software.  There wasn’t anything to be found so this begs the question of false positives and how many resources they use up.

    I may be wearing my rose colored glasses but wouldn’t it be more productive to try and figure out why terrorism exists and fix the cause?

    • Daniel says:

      I may be wearing my rose colored glasses but wouldn’t it be more productive to try and figure out why terrorism exists and fix the cause?

      That would certainly make sense assuming that the purpose of the “War on Terror” is to stop terrorism.  It would be counterproductive if the purpose of the “War on Terror” was to erode civil liberties, scare citizens into compliance, serve as a pretext for military adventurism, justify ubiquitous surveillance, create opportunities for government graft and corruption, funnel money into military and intelligence contractors, or any combination of those goals.

  25. Daniel says:

    Tangentially relevant:

    He’s holding up an open, black umbrella on a warm, sunny day in Dallas.  A few moments after he’s passed by the president’s limousine the president is shot dead.  Gotta be part of a sinister conspiracy, right?

    Well, no…

  26. AirPillo says:

    They certainly score points for knowing how to pick an appropriate name.

  27. xenphilos says:

    I’m all for better intelligence tools given they respect your civil rights and are evidence-based. This fails on both counts.

  28. Michael Roberts says:

    Fail to frolic, go to jail!

  29. RichaEcke2 says:

    We had Mohammed Atta here in Idaho and Bill Clinton burn noticed me for failure to join his communist party, he also said he was gay so I guess that means I supposed to be a Monica for him too. So I knew exactly what Mohammed Atta was like like, a smartass who would do something intense. So I just let these idiot cops and idiot Idaho soldiers push me around and keep silent on Mohammed, I was friends with Mohammed as an agent and simply could have asked him to stand down, he would have done it, but now we have commies in congress and 4 commie presidents in a row, so what the hell good is your stupid software? Take a crap on an agent and watch it hit the fan!

    • DeargDoom says:

      It’s a shame Boing Boing doesn’t have some kind of comment of the day feature. Your genius deserves greater exposure.

  30. DirkSJ says:

    I don’t do anything illegal and I don’t plan to do anything illegal so I don’t really care what the government knows about my life.  If they want to look through my bank account, sort it and crunch it, and discover how amazingly tasty I find Sonic…so be it.  I will forever be branded a Sonic-liker in their database.

    If civil rights got to the point where I feared the government imprisoning me even though I do nothing wrong I would move to another country.  Similarly if it was ever appearing that moving to another country would stop being allowed I would move to another country before it stopped being allowed.  If neither of those two things occur the sky has not fallen and I really care less what they know.

    Actually let me amend that: I want them to know as much as they can, everything really.  It reduces the likelihood they will incorrectly flag me.

  31. gwailo_joe says:

    All I know is anyone who dares to gaze into the Minas Tirith stone has little to look forward to except a withered and burning pair of old man hands: I’ll pass.

    (and of course this ‘security’ concern (besides an excellent choice in name theft) seems little more than snake oil selling profiteers sucking off the Great American Defense Budget Phallus…because the people who sign the checks don’t need to understand the effectiveness of a given system: simply the act of spending taxpayer money means problems are being addressed!)

    Unless…that shit actually works.  And if a few dumbass terrorists are caught, well then…good show beer pong white guys!  

    Yet eventually, as always; evil will find a way to commit some devious act; and a scared populace will cry ‘how could you not have Known?!’  And more freedoms will be lost and billions spent trying to close a barn door that the horses have already exited…

  32. Nadreck says:

    And it’s not as if all of the False Positives are going to be accidental.  See “The Prisoner” episode “Hammer Unto Anvil” for an idea of the fun that can be had with a Surveillance Society.

  33. ComradeQuestions says:

    Fikri isn’t real…

    Talk about a buried lede.

    Sounds like this software is really good at identifying the plots of Tom Clancy novels.

  34. Camp Freddie says:

    It’s complete balls.

    For this to be attempted, you’d need a database of all ticket sales to all major tourist attractions, all international phone calls, all flight tickets, all bank transfers and all traffic violations made by anyone who has every visited or been born in America. Oh, and all the CCTV footage from every tourist attraction, which someone would have to compare against the passport/driver’s license photos of everyone who has a passport/license.  This isn’t possible outside of fantasy land.

    Even if it existed, it wouldn’t catch any terrorist who was sensible enough to not use his own phone to call uncle Osama, who paid with cash, took a friend to disneyland and/or remembered to drive 55 on his was to his flying lesson.

    Wasn’t Palantir a company created by that bullshitter Aaron Barr who tried to ‘take down’ annonymous? (Google says yes!) And epically failed in doing so? (Ars Technica says yes!)

  35. I commented on this article on my blog. It’s a bit too long for here, but I hope you find it interesting.  The Bloomberg article overlooks a lot of potential for abuse with this system (not to mention the irony of the name Palantir). Seems like another sad step towards the surveillance state.

  36. PathosBill says:

    Palantir is an anagram of anal trip. Just sayin…

    (Apologies to Mr. Tolkien, et al)

  37. Baldhead says:

    One thing nobody seems to have mentioned is that even in their fake example, “Fikri” appears to have used, if not his real name, at least the same name for all f these things he’s done. Real terrorists/ criminals don’t do that. So not only does the software apparently require that the terrorist make amateur mistake #1, but also that there be only one person with that name. After all, what if John Michael Smith A is a farmer who needs to buy a lot of fertalizer, John Michael Smith B likes guns a lot and John Michael Smith C is taking flying lessons? and all three live within 150 miles of each other?

  38. Slipgrid says:

    “it’s interesting because it’s one of the few examples of counter-terrorism work that is actually proactive”

    There are many examples of proactive counter-terrorism.  The mass arrest after 9/11 were proactive attempts to prevent more attacks.  Torture is proactive counter-terrorism.  Massive domestic spying is proactive.

  39. Sebkha says:

    …what do people think the CIA does all day, if not data-mining?  Walmart probably knows even more about you.

Leave a Reply