MI5 warning: we're gathering more than we can analyse, and will miss terrorist attacks

In 2010, the UK spy agency MI5 drafted memos informing top UK officials that its dragnet surveillance programme was gathering more information than it could make sense of, and warning that its indiscriminate approach to surveillance could put Britons at risk when signals about dangerous terror attacks were swamped by the noise of meaningless blips from the general population.

The memos are part of the Snowden docs, and it was published today by The Intercept, along with analysis by Ryan Gallagher, who notes that security service whistleblowers have warned that lives were being put at risk by indiscriminate surveillance, which is a liability for intelligence analysis, but an asset for civil service empire-building, given the budgets, procurements, and staffing associated with such projects.

The memos follow on from a 2009 study of Preston, a warrant-based telephone call wire-tapping programme, which found that 97% of the 5,000,000 communications intercepted under Preston in a six-month period were never reviewed.

The publication of these documents comes a day after Parliament debated the Snoopers Charter, a sweeping surveillance bill that will radically increase the amount of indiscriminate bulk surveillance undertaken by the UK security services.

The security services' own assessment of their info-glut have never been part of the debate. It's true that these assessments were secret until now, but even without them, it is obvious on its face that if you're looking for a needle in a haystack, you do yourself no favours by making the haystack as large as possible.

A GCHQ document dated from late 2010 indicated that MILKWHITE was storing data about people's usage of smartphone chat apps like WhatsApp and Viber, instant messenger services such as Jabber, and social networking websites, including Facebook, MySpace, and LinkedIn. Access to the data was provided to law enforcement through an "internet data unit" hosted by the Serious Organized Crime Agency and it was accessible to tax investigators through what one GCHQ document described as established "business as usual" channels.

By March 2011, GCHQ noted that there was "increasing customer demand" for the service offered by MILKWHITE and the agency planned to grow its capacity, seeking £20.8 million ($30.6 million) to update the program's "advanced analytics" capabilities and to maintain its "bulk" storage of metadata records. "Bulk" is a term GCHQ uses to refer to large troves of data that are not focused on individual targets; rather, they include millions and in some cases billions of records about ordinary people's communications and internet activity.

Carlo, the policy analyst with Liberty, said the revelations about MILKWHITE suggested members of Parliament had been misled about how so-called bulk data is handled. "While MPs have been told that bulk powers have been used only by the intelligence community, it now appears it has been 'business as usual' for the tax man to access mass internet data for years," she said. "This vindicates the warnings of security experts and the call by opposition parties for an urgent, independent review of bulk powers. The compromise review recently announced is a poor substitute and without the time and technical expertise, will struggle to address this issue of national importance."

Preston Study [National Technical Assistance Centre/GCHQ]

Digint Imbalance [MI5/The Intercept]

The Digint Programme [MI5/The Intercept]

[Ryan Gallagher]

(Image: Geralt/Pixabay, public domain)