Hello,
Yesterday I renewed my SSL certificates using the acme-php program, as I've been doing for over 3 years now.
I noticed that the size of the .pem file had changed, it had decreased
After deployment on my web server, analysis by a tool like SSL Checker shows that there is one less CA.
Before i had CRT <- R3 <- ISRG Root X1.
Now i have directly CRT <- R3.
Is this normal? Wasn't the ISRG Root X1 valid until 2024?
Thanks a lot,
Guldil
There is more than one ISRG Root X1. There's the actual root certificate, valid until 2035:
Root certificates should never be sent explicitly, since clients already know them. Sending them explicitly just wastes bandwidth and does not increase compatibility.
Next, there's ISRG Root X1 cross-signed by DST Root CA X3:
This cross sign is included by default to enhance Android compatibility. The cross sign does expire next year. Therefore the plan is to phase it out starting early next year:
You should still be getting the cross sign though, since the above change should not have happened yet. However, it is possible to opt-in to the shorter chain for a few years now. Perhaps your ACME client explicitly requests the short chain?
Otherwise, it is also possible that web tools do not show the true chain send by server. To validate this we need to know your domain, so that we can check the chain using other tools.
Thanks to take time to reply.
my acme client hasn't changed since September, it's the same one (acme-php is a compiled phar file)
I forgot to mention that we need compatibility for older Android devices.
Do you think i can just add the part about ISRG Root X1 from september pem into the november file ?
We are investigating
Apologies, it looks like this was actually a small bug on our end. We should have the fix deployed in a few hours, and will update this thread again when it has been deployed.
Thanks a lot, i'll wait 
The correction has rolled out to production now. Again, my apologies for the error.
If you renew your certificate now (or simply re-download the existing certificate) it should come with the chain that you expect.
That said, @Nummer378 is right: the short chain you got this time will become the default in Feb 2024, and and longer compatibility chain you expected will go away entirely in June 2024. If you truly need the longer chain to support users on Android 7.0 or below, then you will need to configure your client to request the alternate chain before Feb, and you will need to find some other solution (e.g. encouraging your users to upgrade) before June.
Thanks, I restarted my script this morning and was able to obtain the full certificate, everything is ok.
We are just on the subject of the end of the X1 in February 2024, we have very old devices (e-reader) and customers are having trouble changing.
@guldil, There may be other free CAs that chain to older roots.
Not a permanent fix, but it might buy you more time [beyond Feb 2024] before you have to upgrade those ancient devices.
I understand accidentally deployed shorter chain default has been reverted in Production.
We checked in Staging and we get the longer chain in PEM.
When is Staging going to default to shorter chain? It would be nice, if client authors could test it there before Feb '24.
Sure, but what would the test reveal? The client would try to use the presented chain as happened here.
It would test that we do not fall under one of the issue buckets under If you are an ACME client author
here Shortening the Let's Encrypt Chain of Trust - Let's Encrypt
We had a short form of this debate yesterday, and I decided to go with keeping 'staging should look like production' for now.
Staging's easy to change, it's just a question of how long we want it to be divergent and, to @webprofusion's point, what value comes from it?
In the case of Staging my question is just whether client authors would find the chain length divergence more useful than confusing. I'm open to arguments for, but maybe we should make a new topic like, "Change Staging R3 to Short Chain Early" and discuss it there?
I think staging should be focused on being as close to production as possible for the benefit of subscribers who need to test their setups or do dry-runs of integration that will not affect their rate limits.
I think client authors should be pushed to use automated testing against local versions of Pebble or Boulder.
Failure modes we have seen in the past include a) never downloading the chain at all and only serving the end-entity certificate; b) never downloading the chain and instead serving a hard-coded chain; and c) only downloading the chain at first issuance and not re-downloading during renewals. Please ensure that your client does not fall into any of these buckets.
Is the change in question already available in Pebble or Boulder? Could you explain the difference between them please? In general which one should client authors use to keep up with latest API announcements? Is there a change history for Pebble or Boulder that we could somehow map to LE API announcements?
Boulder is the same server that LetsEncrypt runs.
Pebble is a lightweight test server that LetsEncrypt developed. Wherever possible it makes a different implementation decision than Boulder, to ensure ACME clients do not hardcore behavior to only LetsEncrypt.
A good client might run test systems against both. Pebble is lightweight enough it can be used for unit tests.
IIRC, the changes that caused this hiccup were basically configuration changes and unrelated to code. The errors described in the passage above are all due to clients not being correctly programmed to the RFC’s specification and ignoring both the RFC and the payloads that LetsEncrypt serves.
In this case, the default chain was swapped and clients failed to correctly process the payloads.
The last step of the RFC is to download the certificate. A first certificate using the default chain is directly provided, and the location header will contain value(s) for certificates with alternate chains. (The certs are actually the same, but that is not guaranteed). A proper client will process the certificate download and extract the correct chains. Robust clients will grab and process the alternate chains.
Improper clients will not process the download and make an assumption on what the chain is, expecting it to be a hard coded value. Improper integrations might use a proper client, but ignore the (correct) chain from the client and make their own assumption on what the chain is.
many of the errors that people experienced are due to an improper client or integration, which ignored the payload from LetsEncrypt and assumed another chain. errors also likely happened due to the client’s operating system (or trust store) not having the root of the default chain.
While simulating these changes on staging or a test server can be helpful, they’re not required for testing. All that client needs to do is follow the rfc and analyze the downloaded certificate to determine the chain and act appropriately. (Note: pebble uses a new chain on every invocation which reinforces the need for clients to analyze the payload and make no assumptions about the chain).
IMHO, a lot of these issues could be avoided or at least more easily detected if the ACME server advertised which chains are currently configured as default and alternate before trying to renew instead of requiring analysis after renewal.
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.
