Should PKI, Web Certificates, Private Keys Be Simple To Explain?

schoen · April 13, 2017, 9:15pm

The existing web PKI is quite complex and I'm not sure that there's a way to eliminate or even significantly reduce that complexity. We think we're on a good path toward getting integrations that will hide it from a lot of people; for example, the Caddy web server automatically gets and uses certificates from Let's Encrypt to enable HTTPS for every site it serves, with no configuration or user interaction, and a large number of hosting providers have also made it automatic or convenient for their users with provider-side integration.

We're hoping this will continue to become the norm everywhere. Providers like HostGator are making this more difficult than necessary. I have said in several other threads that HostGator customers in particular will not benefit financially from using Let's Encrypt certs. This is because our certs have a validity period of 90 days so they must be replaced at least that often; since HostGator charges $10 each time, customers using Let's Encrypt certs will have to pay at least $40 in certificate fees per year, which is more than they might pay if they got a paid certificate from another certificate authority and paid the fee only once every 1 or 2 years. There is nothing that we can do about this except to highlight that other hosting providers (even some which don't directly integrate Let's Encrypt!) are doing a better job with HTTPS integration.

HostGator's lack of integration is also an inconvenience in another regard: a big part of the Let's Encrypt vision is automation of certificate requests and deployment, which several clients and tools that interoperate with Let's Encrypt do. So we have millions of sites that automatically renew their certs every 2-3 months with no human intervention at all. Of course, our shorter renewal period is more annoying in environments where this automation isn't applicable, and HostGator, by requiring manual intervention to deploy certs, means this automation won't happen for their customers. For both of these reasons, Let's Encrypt might not be a good choice for you since you're committed to staying with them.

Of course, we do have people using Let's Encrypt successfully with HostGator and you're very welcome to do so if you decide to. I think the information in the Wikipedia pages @TCM linked to is very helpful and I think it's plausible to say that many of these technologies are becoming an important part of the web platform and that some degree of familiarity with them is an important part of web developers' skill set today. Notably, more than half of all web requests are now made over HTTPS, and, as you may be aware, some browser features are now only available on HTTPS origins. Reading other people's overviews of these topics can be a great way to learn, and in my experience, Wikipedia's coverage of this kind of topic tends to be excellent.

In terms of some of the specific concepts you might encounter:

Making a secure connection requires cryptographic keys, which are used for many purposes and come in many varieties. Many of these are ephemeral and used internally by software in the course of a connection, but some of them are long-lived and are created and used explicitly by administrators in the course of setting up sites and services.
Environments like the web where people interacting don't have a prior relationship normally use public-key cryptography, a set of technologies which allow entities with no direct prior communications or interaction to create a secure communication channel. In public-key cryptography, keys exist in pairs called public keys and private keys. The public keys are not secret and are used by other parties as part of the process of contacting you; the private keys should be kept secret and never posted publicly.
The most common algorithm used in public-key cryptography today is called RSA. When using RSA, you will have a key pair consisting of an RSA public key and RSA private key. These may exist in the form of text files, commonly using a format called PEM.
The process of ensuring that people can confirm the validity of keys they encounter, in order to establish secure connections with the parties they intended to, requires some kind of system for this purpose. On the web we use a system called the web public key infrastructure, or web PKI. This infrastructure is established through agreements between different kinds of entities, including browsers and certificate authorities. The certificate authorities are organizations that follow procedures to publicly confirm the match between a particular public key and some kind of "identity" (whether a legal identity or just something like a domain name or e-mail address), so that they can state publicly that a particular public key "belongs to" a particular identity.
Such statements are expressed in digital documents called digital certificates (which can also be represented as text files in PEM format). The digital certificates are statements by certificate authorities about other entities' identities and public keys. These certificates can, among other things, be verified by web browsers when connecting to a web site. This lets them confirm that they have established a secure communication channel with an entity that is the actual owner of the web site.
The web protocol HTTP can be used in conjunction with a security protocol called TLS (which was previously known as SSL) in order to make cryptographically secured connections that otherwise work much like ordinary HTTP connections. The secure connections are called HTTPS connections, and involve HTTP-over-TLS, usually using TCP port 443 on the web server instead of port 80. In an HTTPS connection, the browser performs a TLS session negotiation with the web server before sending an HTTP request.
Part of the TLS session negotiation, and hence part of the HTTPS process, is verifying certificates from the server. This is normally totally transparent to the browser user, but the web server administrator may have to do something to get and install the certificates.
The server serves a set of certificates called a certificate chain. This chain confirms not only the identity of the server itself, but also the identities of certificate authorities that were involved in the certification process. The chain consists of a single end-entity certificate, also called a leaf certificate, which is the certificate belonging to and describing the server itself, and one or more intermediate certificates (sometimes informally called "chain certificates" or "CA certificates"), describing delegations or grants of authority by certificate authorities to other certificate authorities.
The ultimate authority to issue browser-trusted certificates is held by root certificate authorities, which are trusted as a result of negotiations between the browser industry and the CA industry, and as a result of audits of the business practices and policies of the CAs. The root certificate authorities issue the intermediate certificates that form part of a certificate chain. The certificates describing these root authorities do not need to be made part of chains, because they are already known to the browsers (and, if they weren't, the browsers wouldn't believe them simply because of their presence in a chain).
In Let's Encrypt's case, the root certificate authority from which our authority to issue trusted certificates derives is caled IdenTrust, and it is identified as "DST Root CA X3". Let's Encrypt operates an intermediate certificate authority called "Let's Encrypt Authority X3"; there is an intermediate certificate showing that Let's Encrypt has been authorized by IdenTrust to issue certificates, and this intermediate certificate should be sent as part of the chain.
Every certificate authority performs some kind of verification of entities that request certificates in order to confirm their identity, or as much of their identity as is asserted in the certificate. Certificate authorities like Let's Encrypt perform a kind of verification called domain verification (or "DV"), which does not try to confirm the real-world or legal identity of the subject of a certificate. Instead, it just tries to confirm that the entity requesting the certificate also exercises control over the Internet domain name(s) that will be mentioned as the subject of the certificate.
Let's Encrypt performs DV in what can be a fully automated way, machine-to-machine, using a protocol called ACME, which has also been described as a certificate issuance API. Using ACME, a machine requests authorization for a certificate from Let's Encrypt and Let's Encrypt responds with challenges, which are tasks that can be completed in order to confirm that the requester really has control of the appropriate domain name(s). This would, for example, stop you from getting a certificate for someone else's web site which you don't actually operate. When the challenges are completed, Let's Encrypt confirms this and is willing to issue the certificate.
Some Let's Encrypt client applications, of which there are a few dozen, can perform this whole process with no human intervention, depending on the nature of the web server environment.
Other applications require some degree of human involvement or intervention.
To deploy a certificate, a web server needs the full certificate chain (which consists of the end-entity certificate, often just called "the certificate", and one or more intermediate certificates) and the private key which corresponds to the public key mentioned by the certificate. The public key itself is state in full by the certificate and so it does not need to be stored or given in a separate file. The Certbot client, which my organization, EFF, has written as part of the Let's Encrypt project, creates and refers to four different PEM files whenever you obtain a certificate from Let's Encrypt, namely
privkey.pem, the private key
cert.pem, the end-entity certificate
chain.pem, the intermediate certificates that should be served as part of the chain (currently consisting of just one intermediate certificate)
fullchain.pem, a combination of cert.pem and chain.pem into a single file
Depending on the server software or server environment, these PEM files can be provided or configured in different combinations. These filenames themselves are not industry standards but are conventions from Certbot, and might or might not be used by other client software.

If you can suggest particular parts of our documentation where you think this background information, or other information, would fit, we're always looking for opportunities to improve things.