Symantec caught issuing rogue certificates

Your browser trusts SSL certificates from hundreds of "Certificate Authorities," each of which is supposed to exercise the utmost caution before issuing them -- a rogue cert would allow a criminal or a government to act as a man-in-the-middle between you and your bank, email provider, or employer, undetectably intercepting communications that you believed to be secure.

But not all CAs are trustworthy. Diginotar, a Dutch CA, had to shut its doors after it was revealed that it had issued rogue Google certs that were implicated in Iranian government spying on political dissidents.

In response to incidents like this one, Google created the Certificate Transparency initiative by which browsers and users from across the world cooperate to create a near-realtime index of all the certificates seen in the wild, making it much more likely that rogue certs will be detected and the CAs who issued them will be struck out of the browsers' root of trust.

Today, Symantec was caught issuing rogue Google certificates. The company is one of the biggest names in security, and one of the world's most prominent certificate authorities. Issuing a rogue cert for one of the Internet's biggest companies -- a company that handles an unimaginable amount of sensitive data -- is big news.

Worse: the cert issued by Symantec was an "extended validation" certificate -- meaning that it was signed in a way that guaranteed that Symantec had done extra homework to validate that this was a real, official certificate for Google.

The company says the cert was issued as part of systems testing and that it fired the people responsible.

We learned on Wednesday that a small number of test certificates were inappropriately issued internally this week for three domains during product testing. All of these test certificates and keys were always within our control and were immediately revoked when we discovered the issue. There was no direct impact to any of the domains and never any danger to the Internet. Further, we are in the process of proactively notifying the domain owners and our major partners.

In light of these events, we must reassert our commitment to stand behind our values and our position as a trusted industry leader. While our processes and approach are based on the industry best practices that we helped create, we have immediately put in place additional processes and technical controls to eliminate the possibility of human error. We will continue to relentlessly evolve these best practices to ensure something like this does not happen again.

In addition, we discovered that a few outstanding employees, who had successfully undergone our stringent on-boarding and security trainings, failed to follow our policies. Despite their best intentions, this failure to follow policies has led to their termination after a thoughtful review process. Because you rely on us to protect the digital world, we hold ourselves to a “no compromise” bar for such breaches. As a result, it was the only call we could make.

A Tough Day as Leaders [Quentin Liu/Symantec]

Improved Digital Certificate Security [Google Online Security]

Notable Replies

  1. In the environments where I have worked a great deal of effort has been taken to ensure that data for software testing never resembles the real operational environment. For this reason we would never have created test google certs. We would have used or similar.

  2. If you are testing properly you would have a simulator for that.

  3. I'd actually have felt better if it said "...this failure to follow policies so enraged us that we became a wrathful, unreasoning berserker and fired them immediately. Did they try to explain themselves? We don't know. We couldn't hear them over our bestial screams of rage. When we came to our senses half of our employees were gone and there was blood on our shirts that we're pretty sure didn't come from us."

  4. KarlS says:

    The details are a bit sparse, but the way I read it, it wasn't necessarily meant to do any harm, but it was an egregious breach of proper procedures. Imagine a bank employee borrowing money from Google's account to try something and then putting it back. Some things are just not done in industries built on reputation and doubly not to major players, even if there is no specific damage.

    A certificate issued by a major CA is such a digital nuclear weapon that allowing that to happen doesn't make Symantec look good at all.

  5. So I used to work at symc. I also used to be a key officer and key custodian at a huuuge financial company (those are neat titles for the boring job of making, maintaining, verifying encryption material).

    This is as symc says, a monumental brain fart. And you know how they know? At no time is there any less than three people, all digital activity is logged and stored, there is always a camera on you, and there is always a person literally writing down in a physical ledger every detail of what happens.

    For EV certs there also has to be either a public Dunn and Bradsteet record that is provable you checked, or an attestation letter from a lawyer. You screw up any one of several hundred steps and it will be caught.

    It's like when I took down an entire production, customer facing backend because of a lapse of double checking. (But this is waaaaaay worse).

Continue the discussion

31 more replies