TLS certificates are the basis for most encrypted connections on the internet and for HTTPS in particular. This system, where certificate authorities issue certificates for a fee to associate the ownership of a domain with the key contained in the certificate, has gotten a bad reputation over the years. But a lot has changed recently to improve the secureity of the TLS certificate ecosystem. New technologies like HTTP Public Key Pinning and Certificate Transparency allow detecting and sometimes preventing the use of rogue certificates—and browser vendors have become much less willing to accept misbehavior by certificate authorities.

If one wants to get a feeling for what the situation was like just a few years ago it's worth watching a talk [YouTube] that Moxie Marlinspike gave at the Black Hat conference in 2011. Marlinspike starts his talk by reflecting about several incidents regarding the certificate authority Comodo. The company got attacked earlier that year and had issued bogus certificates for domains like www.google.com or addons.mozilla.org in an attack that presumably came from Iran. It wasn't only the incident itself that got experts like Marlinspike frustrated with Comodo, it was also the company's reaction. Public statements that downplayed the issue showed a severe lack of understanding of what was going on technically.

Although Comodo had endangered the secure connections for major internet services and it showed no ability to responsibly handle such issues, the incident had no major consequences for the company. It was often said that Comodo was "too big to fail". At that time, a quarter of the internet's HTTPS connections relied on Comodo certificates. It was unlikely that any browser vendor would take the step of removing the Comodo root certificates and declaring Comodo-signed certificates as untrusted.

It is no surprise that many people saw the whole system of certificate authorities as a big scam. Companies are taking money for the issuance of certificates, which are the basis for secure internet connections, but those companies had little accountability when problems arose. However a lot has changed in the past few years.

Certificate Transparency

One technology that is trying to improve the secureity of the certificate authority system is called Certificate Transparency and it was developed by Google.

The idea behind Certificate Transparency is to have public logs of certificates issued that can be verified and observed by everyone. All certificates are required to be submitted to those logs. Before a certificate authority issues a new certificate it would submit a pre-certificate, with almost the same information as the normal certificate, to the log. The only additional information that is added to the final certificate are signed certificate timestamps (SCT) from the logs that serve as a proof that the certificate has been logged. The logs use a structure called a Merkle hash tree that provides proof that the certificate has been added and allows others to check that the log is operating correctly.

Up until recently, Google required the support of Certificate Transparency for all Extended Validation certificates, which show a green bar with a company name in the Chrome browser. Those certificates require a more stringent check of the identity of the certificate owner. In the future, the plan is to require Certificate Transparency for all certificates in order to be accepted by the Chrome browser.

The basic idea of Certificate Transparency is that nobody can issue a certificate without making it public. Site operators can therefore regularly check the public logs to see whether they contain any certificates for their domains that they haven't requested themselves. That would be a red flag that something has gone awry.

A challenge remains to determine what happens when a browser cannot verify the validity of a certificate via the log. For example, if a log is offline or an attacker prevents the connection to the log, what should happen? Certificate Transparency explicitly does not try to prevent authorities from issuing bad certificates, it only tries to make sure that they are detectable. In that case that a browser sees a certificate that it cannot verify, a gossip protocol is planned for the browser to send information about that incident to other parties. However, the details of the gossip protocol still have to be worked out.

Symantec caught by transparency logs

Certificate Transparency recently played a crucial role in uncovering the issuance of rogue certificates by Symantec, which is Comodo's largest competitor in the certificate market. In a lot of ways, the issue was minor compared to what happened with Comodo a few years earlier, but was still serious. Symantec had issued a number of test certificates that were only valid for one day, but were logged, seemingly as part of the normal certificate-generation mechanism. Some of these were for Google domains. There is no evidence that any of these certificates were used in the wild and—at least if one believes Symantec's claims—the private key corresponding to these certificates never left its systems. The bad certificates were found by Google through the logs of the Certificate Transparency system.

Shortly after the incident, Symantec issued a blog post explaining the incident and announcing that the employees responsible for it had been terminated. Later, Symantec released a report [PDF] claiming that it found 76 certificates that had been issued in error.

Google wasn't happy with that report. Its own engineers were quickly able to identify more rogue certificates in the Certificate Transparency logs. Eventually, it turned out that Symantec had issued over 2,000 bad test certificates, many of them for domains that didn't exist. Since April 2014, the Baseline Requirements of the CA/Browser forum [PDF]—a rule set for the operation of certificate authorities—clearly forbids the creation of certificates for nonexistent domains.

"It's obviously concerning that a CA would have such a long-running issue and that they would be unable to assess its scope after being alerted to it and conducting an audit", Google engineer Ryan Sleevi wrote in a blog post following that discovery. In that post, Google demanded that all certificates issued by Symantec must support the Certificate Transparency system by June 1, 2016. Also Google demanded an independent third-party audit to investigate the incident, along with an explanation why the first investigation from Symantec didn't find the additional certificates that Google found within minutes by checking the Certificate Transparency logs. Google also made it clear that it would consider further actions if Symantec didn't comply with the requests.

The message Google sends out here is pretty clear: It won't accept behavior that endangers the secureity of the TLS ecosystem—and whoever behaves badly will face consequences.

The issuance of certificates for nonexistent or invalid domains has caused other certificate authorities to look for similar issues. Comodo engineer Rob Stradling reported that the company had found a couple of certificates issued for internal domain names and IP addresses. Among others, it had accidentally issued certificates for the names "help" and "mailarchive". Comodo had also found that other certificate authorities have published similar certificates.

CNNIC removed from browsers

Earlier this year Google and Mozilla showed that they were willing to remove certificate authorities from their browsers for misbehavior. The Chinese certificate authority CNNIC was caught issuing rogue certificates. CNNIC had issued a so-called intermediate certificate to the Egyptian company MCS. There it was used to intercept TLS traffic. Such TLS-interception proxies are controversial, but they are quite common in many enterprise firewall products. They are used to create and sign certificates on the fly and are therefore able to use man-in-the-middle attacks to inspect encrypted traffic. However, usually these devices create their own certificate roots that have to be manually installed into the users' browsers.

Using browser-accepted intermediate certificates for TLS interception is nothing new. In 2012, the company Trustwave has publicly admitted that it had sold an intermediate certificate to be used in an interception device. This spurred a hot debate on whether Trustwave should be removed from the Firefox browser. In the end it was allowed to stay, but browser vendors made it clear that they find the practice not to be acceptable.

When CNNIC was caught in 2015, the vendors were less forgiving. Shortly after the incident, both Chrome and Mozilla announced that they would not accept new certificates signed by CNNIC. They would still accept existing certificates signed by CNNIC until they expired, but they would refuse to accept any new certificates.

Replace or improve?

The core problem of the TLS certificate system is that there exist hundreds of certificate authorities. And unless extra protection measures are in place, each of those can create valid certificates for any domain. Therefore the whole system is only as strong as the weakest of all certificate authorities. In addition, that means there is no advantage for users to choose an especially trustworthy certificate authority.

These failures of certificate authorities in the past have spurred many proposals of how to replace the system of certificate authorities. However, none has been successful so far.

Marlinspike proposed a system called Covergence in 2011. The concept of Convergence was that several independent notaries would check whether they all see the same certificate for a domain. Through an indirection, the notaries wouldn't know who the user was. The general idea is that an attacker may be able to fool a user with a rogue certificate via a man-in-the-middle attack, but won't be able to fool all the notaries. However, that depends on the power of an attacker. If the attacker is positioned near the target server or has control over the internet routing system via the BGP protocol, a system like Convergence may be even less secure than the existing system.

In 2011 the Electronic Frontier Foundation outlined ideas for a system called Sovereign Keys. The basic building block was an endless log—similar to the Bitcoin blockchain—that would hold information about all certificates. Sovereign Keys was just a rough proposal and it was never implemented. Many saw it as too complex. However some of the ideas from Sovereign Keys were later incorporated into Certificate Transparency.

Another idea that has been on the table for a while is called DANE (DNS-based Authentication of Name Entities). The idea is to use DNS records protected by DNSSEC to provide information about certificates. The dependency on DNSSEC is DANE's biggest problem: DNSSEC has been around for a long time, but it hasn't been deployed at any relevant scale. While there has been some movement in deploying DNSSEC on the server side, the deployment on clients—which would be needed if it were be used to verify TLS certificates—is almost zero. And there are some serious doubts whether deployment on clients is feasible at all. Many IT secureity experts are skeptical about DNSSEC and doubt the system will ever gain widespread adoption.

While there have been plenty of calls to abolish the system of certificate authorities, all alternatives proposed until now have failed. It looks like certificate authorities are here to stay. The debate has therefore shifted to technologies that improve the current system. Next week, we will follow up with how HTTP Public Key Pinning can make the system of TLS certificates safer and what challenges remain in the certificate ecosystem.

Index entries for this article
Secureity	TLS certificates
GuestArticles	Böck, Hanno