Currently, we're getting long chain files. How do I change it to getting short chain files instead?
1 Like
What's your ACME client?
Most have an option to choose.
5 Likes
In case the ACME client does not have this as an option:
1- If using your own client, the URL to download the alternate chains appears as a Link: header on the certificate download with rel="alternate". See RFC 8555 - Automatic Certificate Management Environment (ACME)
2- You can manually substitute the alternate chain, and manually create an alternate fullchain by cat'ing the alternate chain files with your end-entity/leaf certificate. This is because the long-chain and short-chain use the same private key to sign your Certificates.
5 Likes
Thanks. Will look into these options.
2 Likes
If applicable, the most easy option is to use the option in your ACME client to select the chain. E.g., with Certbot you could use --preferred-chain "ISRG Root X1" on the command line.
4 Likes
You can make the short chain by simply removing the last certificate from the long chain.
5 Likes
Be careful about just removing the last cert in the default chain. While that is currently the long chain early next year the default will be the shorter chain. Blindly removing the last cert on the shorter chain will cause problems.
5 Likes
Quite right, but then you wouldn't be removing the last certificate from the long chain anymore. 
5 Likes
Just to reiterate what folks have said: For each issuance, Let's Encrypt provides multiple chains, returning the default one from an order's finalize URL. Currently the long chain is default.
If you're writing your own ACME client, the client should fetch each of the alternates (indicated with Link: rel="alternate") and decide among them based on whatever heuristics you want. For instance, you could have a list of all the roots that are trusted by the OSes and browsers that you care about, and pick the shortest chain that validates against that list. Then the ACME client configures the webserver with the appropriate chain.
9 Likes
Is the last cert always the root? Can they flip the order, resulting in the last one being the actual cert?
No, it might be a cross signed root or an intermediate. You have no guarantees about which if you don't control which chain you get.
5 Likes
The certificates returned by Let's Encrypt are (and I believe always will be) listed from leaf at top down to whatever certificate (usually intermediate) needs to be verified via a stored trust anchor.
3 Likes
Adding to this- an ACME client should save each of the Certificates and Chains locally as well, so services can immediately switch to an alternate chain whenever needed. I recommend the general pattern of creating a local storage of an alternate chain on it's first encounter, and then saving metadata about the alternate chains with the certificate. That greatly reduces filestorage and allows the chains to be rebuilt on demand.
3 Likes
Interesting idea. I haven't seen this requested before or used by any deployments yet. I think re-downloading suffices most of the time? (I bet a lot of site owners would be confused to see multiple cert chains on their server when only 1 is used...) What use case would necessitate storing all the alt chains up front?
Posh-ACME does it to enabling changing the chain in use without needing to talk to the ACME server again.
3 Likes
Right, but what is the use case for that? (For not talking to the ACME server again, when downloading the alt chain -- in the very rare case that is wanted -- probably takes a quarter of a second or so)
I think it's mostly just about not having to repeat calls to the ACME server if necessary since those files aren't going to change for the life of the cert. My client is already storing the cert and primary chain as well as checking the alt chains. It's not a lot of effort to also store the alt chains if they exist and it simplifies the code if the user ever does want to change chains by having one less thing to do.
6 Likes
I don't think that the ACME spec guarantees that the certificate will be downloadable for the entire time that the certificate is valid. I think that the order can expire, and the server not host the certificate anymore, at any point the ACME server wants it to.
In practice I don't know if there are any servers that remove the certificate before it would expire, but I could see not wanting to rely on the CA continuing to be up and just have everything needed self-contained in the ACME client after issuance.
7 Likes
That's a good point I hadn't considered. I still don't know of any users who have needed to switch chains instantly, but I guess I will keep that in mind.
3 Likes
The ability to instantly switch chains can be incredibly important when you run into emergencies like an incompatibility disproportionately affecting your end users.
If you only have a handful of certificates, a later download by creating a new Order to get the alternate chain is a viable solution in many situations.
While the Order Object could allow for chain downloads at a later date, many ACME clients don't save the relevant info to reconstruct the order URL at all, and those that do simply have it in API logs that must be mined. Several ACME Servers, including LetsEncrypt, do not support listing or iterating over past orders, so I cautiously treat Order URLs as short-lived and ephemeral. On top of that, as @petercooperjr noted above, the lifetime of the order object may not be long lived.
If you have large deployments of nodes or certificates, the task of having to procure new certificates and coordinate service restarts can be a problematic bottleneck within your systems. If the issue is due to an expiration or revocation of one of the chain certs, you can expect a degradation of API performance and a bottleneck on the API server.
Again, also noting how this ties into my previous criticism of LE/ACME - the available Root Certificates and Chains are completely unknown to ACME Clients until the Left/EndEntity Certificate is downloaded and the alternates are iterated and downloaded as well. LE/ISRG has rotated chains a few times in the past, which can catch subscribers off-guard. Unless LE/ISRG were to somehow advertise the default + alternate chains up-front, IMHO every client should iterate and download all the alternate chains. The only reason I see for not downloading all chains, is when a subscriber specifically requests a root/chain and that is presented as the default.
I've found the best option for flexibility and storage to be iterating all presented chains, saving the first occurrence, and "decorating" the certificate with metadata about the compatible chains. If you have a corpus of chains, you can also analyze them to find compatible chains for existing certificates in an emergency.
The current chain offerings and implementation from LetsEncrypt are rather predictable (long vs short, single intermediates, cross-signed, etc) but they will be moving to multiple intermediates in the future, which will make this more complex.
Ultimately my motivations are 90% as a precaution for emergency situations and 10% as a "might as well" due to deficiencies in the ACME spec to clearly indicate the chains.
3 Likes