I had some questions about how encryption is working on the How It Works page. If someone has the patience to help me understand this I'd be very grateful!
I made this graphic combining the two diagrams into one:
In the first diagram, could someone please describe exactly what the box in the middle-top is communicating? Some questions I have are:
What does "A" stand for?
Does this message include "9cf0b331" signed by the private key?
Does this message also include a key itself in the payload? I see that big blue key and think that this http message would include a key inside of it.
Has at some point Let's Encrypt received the agent's public key? I understand they would need this to verify the signature on "9cf0b331", but I haven't read in the text that the agent ever sent this to LE.
In the second diagram, what exactly does the box in the middle-top communicate? I have some more questions:
What does the "S" mean?
What does the "A" and outer-most box mean?
The text says the CSR is signed by the authorized key. Does signed specifically mean it was signed by the private key and will be verified by LE who has stored the agent's authorized key pair public key?
What does the key with the "S" mean? Is this a new public key just for example.com that is different from the public key in the authorized key pair?
What does the inner box labeled "S" mean? The text says the CSR includes a signature by the private key corresponding to the public key in the CSR. Is this box a signature by the new private key? If so, how does LE verify this if the public key needed for verification is inside of it? Or is it inaccurate to describe the included public key and domain as "inside" the signature? Sorry if that is a shockingly ignorant question. I think I have that perception because the orange "S" key is inside the box with the "S" label.
LetsEncrypt currently issues RSA and ECDSA certificates. They operate similarly to Diffie Hellman, and will be much easier to understand once you are more familiar with DH.
Try watching the video and reading those links, that should give you some clues about what is happening in the charts.
What does "A" stand for? "A" is the key that the client used to encrypt the nonce.[private key]
Does this message include "9cf0b331" signed by the private key? Yes.
Does this message also include a key itself in the payload? I see that big blue key and think that this http message would include a key inside of it. Never!
Has at some point Let's Encrypt received the agent's public key? I understand they would need this to verify the signature on "9cf0b331", but I haven't read in the text that the agent ever sent this to LE. Yes. [but not yet]
What does the "S" mean? "S" should be the signing request ("CSR").
What does the "A" and outer-most box mean? That means the contents where encrypted using the "A" (private) key.
The text says the CSR is signed by the authorized key. Does signed specifically mean it was signed by the private key and will be verified by LE who has stored the agent's authorized key pair public key? LE would never have the clients key pair (private part always remains private).
What does the key with the "S" mean? Is this a new public key just for example.com that is different from the public key in the authorized key pair? Think of "S" as "CSR".
What does the inner box labeled "S" mean? The text says the CSR includes a signature by the private key corresponding to the public key in the CSR. Is this box a signature by the new private key? If so, how does LE verify this if the public key needed for verification is inside of it? Or is it inaccurate to describe the included public key and domain as "inside" the signature? Sorry if that is a shockingly ignorant question. I think I have that perception because the orange "S" key is inside the box with the "S" label. There is no new private key - there is only the one/original private key. [if the client were to change the private key (at any point), things would fail] [if LE were to ever have the private key, things would be insecure] [if LE were to provide the client with a private key, things would be insecure]
Thanks @jvanasco for the background info and @rg305 for answering my questions!
I've tried to focus my questions a bit if it helps understand where I'm getting confused.
Starting with the validation step, I read that the agent signs the nonce with its private key and LE verifies it. But I didn't read that the agent ever shared its public key. How does LE verify the signed nonce at this time without the agent's public key?
For the Certificate Issuance step, I see the CSR is a domain and an orange key "S", contained by a box "S", contained by a box "A". It comes along with this paragraph:
To obtain a certificate for the domain, the agent constructs a PKCS#10 Certificate Signing Request that asks the Let’s Encrypt CA to issue a certificate for example.com with a specified public key. As usual, the CSR includes a signature by the private key corresponding to the public key in the CSR. The agent also signs the whole CSR with the authorized key for example.com so that the Let’s Encrypt CA knows it’s authorized.
Does the outer box "A" represent that the entire CRA is signed by the private key?
Does the inner box "S" represent a smaller part of the CRA also signed by the private key?
Are the contents of the inner "S" box the domain to provide the certificate for and the agent's public key?
Assuming the S and A boxes are both signatures done with the agent's private key, how does LE verify these and access the contents within? I think LE would need the agent's public key to do this but I haven't read that the agent ever shared that with LE except for right now deep inside the CRA.
To be frank, I don't think the how it works page is well written. It seems to try to give a general simplified overview, but I still had some trouble trying to understand this myself - and I know the ACME protocol spec moderately well. The page seems to expect some prerequisite knowledge about CSR-based issuance, which is confusing when combined with the unspecified terminology used in the page (i.e. it doesn't use the proper ACME terms in many places).
For one, the ACME protocol actually deals with multiple keypairs. The how it works page seems to do a poor job at clarifying this, or explaining which key is used when.
The ACME protocol primarily deals with two keypairs:
The account key
The certificate key
Both keys are generated and maintained by the ACME client (called agent in the how it works page). Both private keys never leave the agent/client's machine.
When dealing with the ACME protocol, the agent commonly signs messages it sends to the ACME server (the CA) using its account key. This is done to ensure that the transmission between the agent and the CA is authenticated and so that the CA can assert some state (history) about the agent. The how it works page seems to just call the account key "the key".
When a certificate order is to be finalized, the agent submits a Certificate Signing Request (CSR). The CSR contains a X.509 structure, which will eventually be transformed into the final certificate issued by the CA. The CSR contains, among other things, another public key: The certificate key. This key is entirely independent of the account key (in particular: An agent can request many certificates, all with different public keys, but the agent's account key is always the same*). By convention, the CSR itself is signed using the certificate key (not the account key).
*There's actually account key rotation in the ACME protocol, but discussing this seems a bit out of scope here.
The very basic flow of the ACME protocol is along these lines (simplified):
If the ACME client (using proper ACME terms from now on) is using the CA for the first time, the client will register a new account. This involves sending the CA a public key (the account key) and some metadata, like an email address to be associated with the account. The account is persistent, can be reused in the future. Account's also have a unique address, called the Account URL.
Next, the client creates a new (Certificate) Order. Here the client tells the CA which domain names it would like to order a new certificate for.
The CA will respond with the created Order object, and give the client things called authorizations. The client must fulfil these authorizations to be authorized to obtain a certificate.
Fulfilling a specific authorization depends on the type of the challenge that is to be performed. The ACME protocol defines multiple challenges (some are HTTP-based, others DNS based, some are something else entirely). Challenges usually revolve around placing a particular token at a specific location (related to the to-be-authorized domain name) where the CA can retrieve it. This process is called Domain Validation, as the CA verifies that the client possesses a certain amount of control over the domain.
The client will prepare the challenges, and then trigger the authorization objects (by notifying the ACME server via the ACME protocol) to start validation. The CA will report the success or failure of each authorization.
If enough authorizations (e.g. at least one for each domain name to be included in the final certificate) have been successfully validated, the client may do what's called order finalization. The client contacts the CA via the ACME protocol and instructs it to finalize the order object. This finalization request is signed by the client's account key (but so are various other requests, like the new order request as well). Finalization also includes the CSR, which contains the certificate key, and also yet another signature of the CSR, signed by the certificate key. The CA will then begin the process of issuing the final certificate (assuming all security checks are OK).
The CA then returns the final, issued certificate to the ACME client, which may immediately begin installing and using the certificate.
Hear hear. Also, I think that page is from the very beginning from Let's Encrypt and the ACME protocol. It would be nice if it would get a complete overhaul where it explains the ACME protocol simply for absolute beginners and further down with more complexity for the more expert readers.
First to clarify, when we say sign, does it mean creating a digital signature using a private key? I read up on this on this Stack Exchange answer, that using private keys create digital signature and using public keys create actual encrypted messages.
My remaining question is about the CRA's structure. I understand it to be:
account key signature
certificate key signature
certificate key itself (public key)
Is the above structure correct? If so, how does LE retrieve the certificate public key if it's inside a signature that needs the public key itself to verify/open?
Ah yeah, looks like we need to clarify digital signatures. No problem .
We're using asymmetric cryptography here, specifically asymmetric signatures. It's asymmetric, because each key consists of two halves: The public and the private key (together commonly called a keypair). The private key is supposed to remain secret, while the public key can be freely published or transferred over a network. The secret key is able to sign messages, which results in a signature. The signature can be verified using both the message and the public key.
In a network protocol, the signature is commonly transferred alongside the message. The signature is really just that: Like a signature someone put on a paper. The signature does not hide (or encrypt) the message. The message can still be read by anyone (unless we additionally encrypt it, alongside with signing).
For the CSR specifically, the CSR is the message. So when we sign the CSR, we get a signature that's valid for this specific CSR and keypair. We then transfer both, the CSR and the signature to the other party. So the CA can freely parse and read the CSR, extract the public key, and then use that public key to verify the entire CSR using all data: The entire CSR, the public key, and the signature send by the client.
To sum up, signing (authentication) doesn't involve encryption at all. The ACME protocol itself signs its messages, but doesn't encrypt them. However, the ACME protocol runs over HTTPS, which does use encryption. But this is independent of the inner protocol: All messages are decrypted by the HTTPS protocol engine before being processed by the ACME server (and vice versa).
Thanks @Nummer378, I think I understand this a lot better now. Your detail that signatures are transferred alongside the message, that they don't hide the message, was key. Additionally that the ACME protocol itself doesn't encrypt messages, just signs them.
The entire connection is encrypted using HTTPS/TLS of course. (But depending on your setup it might get intercepted by e.g. company security systems or your antivirus software.)
Let's jump back for a second, as you may be confusing a handful of things here.
1- The "Account Key" is a cryptographic key only used by a Subscriber (website owner), for the sole purpose of authenticating themselves to the LetsEncrypt ACME Server. It is not publicly related to any certificates. It's use within the ACME protocol is documented in RFC 8555 - Automatic Certificate Management Environment (ACME)
2- To get a Certificate, a Subscriber creates a CSR (Certificate Signing Request) which wraps together some data, along with the Public Key Component of an asymmetric key (RSA or ECDSA). The CSR is signed by the Private Key component of the same asymmetric key. This is submitted to a Certificate Authority (CA), for them to reformat the data and sign with the private component of their asymmetric key (which will usually be an Intermediate Certificate that derives trust from a publicly trusted root certificate. This has been standard for decades. The resulting payload is a Signed Certificate.
3- The new-ish ACME protocol automates generating the CSR, submitting it to the CA, performing the required validations, and downloading the resulting certificate.
4- Within the ACME protocol, all the relevant operations are happening under a HTTPS connection. The payloads sent to the ACME server are all signed by the private key component of the Account Key; if the payload contains a CSR, the CSR is signed by the Public Key component of the Certificate's asymmetric key (RSA or ECDSA).
Account keys rarely change. The ACME protocol supports a key rollover functionality.
Certificate keys are typically unique to each issuance - most ACME clients will generate a new keypair for each issuance and renewal. In some situations where a single entity is the sole administrator of a large number of domains/certificates, they may want to share/recycle a single key across multiple certificates. For example, on one of my systems we rotate the keys every week instead of every certificate (so all certificates obtained during a specific week will share the same key). This is only allowable when a single entity controls all the domains, as the private key can not be shared with others.
.