HOWTO search the Web like the NSA

Wired's Kim Zetter rounds up some of the highlights from Untangling the Web: A Guide to Internet Research [PDF], an NSA guide to finding unintentionally published confidential material on the Web produced by the NSA and released in response to a Muckrock Freedom of Information Act request. As Zetter notes, the tactics discussed as described as legal, but are the kind of thing that weev is doing 3.5 years in a Federal pen for:

Want to find spreadsheets full of passwords in Russia? Type “filetype:xls site:ru login.” Even on websites written in non-English languages the terms “login,” “userid,” and “password” are generally written in English, the authors helpfully point out.

Misconfigured web servers “that list the contents of directories not intended to be on the web often offer a rich load of information to Google hackers,” the authors write, then offer a command to exploit these vulnerabilities — intitle: “index of” site:kr password.

“Nothing I am going to describe to you is illegal, nor does it in any way involve accessing unauthorized data,” the authors assert in their book. Instead it “involves using publicly available search engines to access publicly available information that almost certainly was not intended for public distribution.” You know, sort of like the “hacking” for which Andrew “weev” Aurenheimer was recently sentenced to 3.5 years in prison for obtaining publicly accessible information from AT&T’s website.

Use These Secret NSA Google Search Tips to Become Your Own Spy Agency


  1. Minor technical difference between search engine ‘hacking’ and what Weev did. Pages linked to by search engines are indexable.. meaning at some time, somewhere, there was a link to the page, even if it was just a directory listing. 

    What Weev did was manually change URL parameters to get access to places he wasn’t supposed to. Weev didn’t get his info from a search engine — it wouldn’t have been possible to. The two things are only similar inasmuch as they’re about getting webservers to give up the goods without cracking any passwords.

    Also.. I can’t find a definitive answer on the web about whether or not Weev informed AT&T of the problem and gave them a chance to fix it before going public. Different sources say different things. Anyone got an answer to this?

    1.  Another minor difference is that Weev doesn’t work for the NSA. I think that had a lot more to do with it.

      1. Interesting how the accounts of people who allegedly reported Ariel Castro to the police were universally believed while Weev’s claims are (almost) universally disbelieved.

        Most people trust whatever claims reconfirm their previously held beliefs/suspicions.

        1. Weev used to be very open about the fact that he did it all for the lulz, that he just liked fucking with people. Then he gets in trouble and suddenly he’s an upstanding citizen looking out for everyone’s rights and security. One of those things is a lie. Which one seems more likely, under the circumstances?

          1. Yep! I am nothing if not consistent: I don’t like assholes. I wouldn’t’ve thought that was a controversial position, but on this great big internet of ours, it seems to be.

          2. Hm, being an asshole in public does not seem to justify years in prison, at least to me. It certainly did not help him getting less jailtime, but the question if the law is broken and AT&T as well as the Fed should have dropped the case in the first place is not solved by him being an obnoxious person. Or is it?

        2.  WTF does Ariel Castro have to do with this?  Kinda seems like you have more of an axe to grind than PhasmaFelis (who freely admits to having an axe to grind).

  2. It once used to be possible to search Google for filetype:php~
    This would give a list of backup files created when the linux/unix editing tool VI/VIM was used, and could easily give away server passwords and database logins for various dynamic websites because the webserver was often not configured to hide or process files using the PHP engine, so they would show up as plaintext.

  3. This is hardly hacking; this is Google 102.  As I recall, Google used to actually publish a doc explaining how to use these search modifiers to find open indices and such.  It doesn’t take much imagination to use them to go digging for data that’s commonly left unsecured.  Honestly, I suspect you’d find mostly outdated or erroneous cruft left in the open by noob admins and individuals.

    By NSA standards, this is kids stuff.  I’m sure they run their own custom crawlers these days.  You can find a lot of interesting stuff if you just ignore robots.txt and poke at common Apache mis-configurations/bugs.  That or they just use one of the immensely intrusive commercial data mining tools/services that exist.

  4. > By NSA standards, this is kids stuff. 

    No kidding. The book says to use Google Groups to search newsgroups, unlike some people I could mention who know how to type ‘grep’.

    > I’m sure they run their own custom crawlers these days.

    I hope so, otherwise we live in a terrifying world where if terrorists take down Google, the NSA will have to rely on Wikipedia for its research, and that’ll just lead to them looking for terrorists named “BABA BOOEY BABA BOOEY” or possibly “COLBERT RULES, LEMONPARTY”.

  5. Wtf is this? 1337 speak in the title w00! Any guesses on what the background image is supposed to be? “tangled stuff” I guess….

  6. Can we seriously stop throwing Weev out as a martyr like he’s the modern Kevin Mitnick or something? It is right and good that this life-destroying, rape-threatening motherfucker is in prison. It’s a damn shame that they had to use an unjust law to do it because a better one didn’t exist, but if you want to repeal that law, saying “without it, Weev would be a free man!” is not going to win you any support.

    1. See my comment above. See also this article in the guardian, wich has admittedly a very mild view on this dreadful Auenheimer person. It also has a very mild view on Kim Schmitz. But the overall picture is what counts. All those who are now facing ‘the force of the law’ should remind us that the law is broken. It’s not the internet which is broken and has to be ‘fixed’.

      I refuse to take *any* of those guys as a martyr. This is including Assange. And I am *very* unsure that Kevin Mitnik counts as a martyr. I’m also not sure about Boris. But whatever your mindset about these people as a person: the law is broken. Internationally. We have to fix it.

      Putting assholes in jail because of broken laws does not solve anything.

      /edit: fixed broken link, my bad.

Comments are closed.