Help thread for DST Root CA X3 expiration (September 2021)

Woops, you're right, there is a typo in the error message, I copied/pasted the wrong one.
Here's the correct one :
ERROR: Alternative chain with CN = ISRG Root X2 not found, available options: DST Root CA X3, ISRG Root X1

Thank you for the information regarding ECDSA/RSA certificates.
I completely missed this point... so another option to add to the command line
--algo (-a) rsa|prime256v1|secp384r1

However it does not solve the problem
dehydrated -c -x -d --preferred-chain "ISRG Root X2" --algo secp384r1
[...]

    • Requesting certificate...*
      ERROR: Alternative chain with CN = ISRG Root X2 not found, available options: DST Root CA X3, ISRG Root X1

I retrieved the full string from valid-isrgrootx1.letsencrypt.org via the site you provided but I get the same result with "DST Root CA X3" instead of "ISRG Root X1"

I think I'm missing something here....

2 Likes

ISRG Root X1 and ISRG Root X2 are roots. Roots never sign subscriber certificates ("leafs") directly. Instead, they sign intermediate certificates which in turn sign leafs.

The intermediate certificate currently in use (by default) is R3, which is signed by ISRG Root X1. ISRG Root X1 in it's current form is additionally signed by DST Root CA X3. There's an alternate chain available which does not include this last signature and therefore terminates at ISRG Root X1.

ISRG Root X2 is a brand new root certificate. It has signed an intermediate certificate called "E1". It is possible to get a leaf certificate signed by E1, but only under two conditions:

  1. Your leaf must use an ECDSA keypair (ISRG Root X2 and E1 are both ECDSA themselves).
  2. As this feature is currently in a "sort-of" beta, only allow-listed accounts can get signed by E1. To hop onto the allowlist, you need to fill out a form: ECDSA availability in production environment

With both conditions fulfilled, you get a leaf certificate signed by E1, which is signed by ISRG Root X2, which is signed by ISRG Root X1.

ISRG Root X2 is not in any root program as of today, therefore it is impossible to request a chain from LE terminating at X2 - it would just result in errors. It will take a bunch of years until X2 is widely supported (X1 is in root programs for almost 5 years, and people are still concerned).

valid-isrgrootx1.letsencrypt.org is an example for a site that terminates at ISRG Root X1 and does not include the final signature up to DST Root CA X3. However, some TLS implementations (e.g browsers) tend to build their own chains, which can differ from the chain send by the server - this causes seemingly "incorrect" results in their displays.

PS: You might want to read Production Chain Changes too.

6 Likes

Thank you for your help.

So... if I understand well, I do not need to worry about anything as E1, E2, R3, and R4 intermediate certificates will be renewed and resigned by ISRG Root X1 or ISRG Root X2 (depending on the key type) instead of DST Root CA X3

5 Likes

If I have a certificate signed by this expiring DST Root CA X3 root certificate, what should I do between 30/9/2021 and my certificate's expiration date, for example 21/10/2021 ?

Re-apply a new certificate, or this certificate is available between 30/9/2021 and 21/10/2021 ?

1 Like

If we apply certificate through ACME protocol from Let's Encrypt after 30/9/2021 , which root CA would signed this certificate?

1 Like

Unless you have special needs (needing to support older OpenSSL/LibreSSL/GnuTLS/embeddedTLS versions), nothing. (OpenSSL Client Compatibility Changes for Let’s Encrypt Certificates)

Root CA's do not sign subscriber certificates ("leafs"). Roots sign intermediate certificates which in turn sign leafs.

The certificate chain currently in use is:

End-entity certificate ("leaf") ← R3 ← ISRG Root X1 ← DST Root CA X3

(Production Chain Changes)

This chain will continue to be used, even after DST Root CA X3 has expired. This ensures Android compatibility, but will break some clients named above.

You can request an alternate chain*, terminating at ISRG Root X1 (instead of DST Root CA X3), but this will cause problems with Android versions < 7.1 (and will fix other clients).

*How to do this depends on your ACME client. Some modern clients have an option, usually called preferred-chain.

4 Likes

When comes the time that the AIA-URL "http://r3.i.lencr.org/" will serve the R3 Version that is signed by ISRG Root X1 and not the old R3 that is signed by DST Root CA X3 and is only valid until 29.09.2021?

This can become a problem with improperly configured servers where the client browser tries to load the intermediate from the AIA Server after September 2021.

7 Likes

I doubt, that a specific app will still work on older android devices after the expiration date of DST Root CA X3. We tried this: Renew of the Let'sEncrypt-SSL-Certificate for the backend-server of the app (bayern-app-test.bayern.de). It is now valid until 05 Oct 2021. Now I set the date on a Lollipop-Tablet manually to 02 Oct 2021. The app then gives an error message and in the log I find

error: HandshakeException: Handshake error in client (OS Error: CERTIFICATE_VERIFY_FAILED: certificate has expired(handshake.cc:359))

Setting the date to 29 Sep 2021 makes the app work normal again.
It doesn't match with what you were saying, that android doesn't check actively for expiration, or I misunderstood some part.

4 Likes

Hi everyone! This thread has become really long! :slight_smile:

Just want to confirm that the same applies, that old Androids will continue to be OK until Sep 2024, but iOS < 10 will get certificate errors. Is that correct?

I want to check with you before I go ahead and put a warning on my website advising iPhone users to upgrade their devices.

What about MacOS < 10.12.1 - will those will be affected too?

Thank you!

3 Likes

Hello,

The test website for ISRG Root X1, https://valid-isrgrootx1.letsencrypt.org/ is also serving a second certificate with DST Root. This makes it difficult to test compatibility of ISRG Root X1. Can we have the website deliver only ISRG Root X1 issued certificates?

image

2 Likes

The test site does only send the R3-signed-by-ISRG-Root-X1 certificate over the TLS connection. But that doesn't mean that's what your browser will show when you look at it. You need to use a tool like openssl or an online tool designed for looking at the sent certificate chains in order to see what's actually getting transmitted. Browsers and operating systems will just show you a way they found a valid chain, even if it's not related to the intermediates that were sent over the connection.

9 Likes

This is one of the best tools I've found for viewing the actual chain being sent:

https://www.digicert.com/help/

6 Likes

Nice addition to the toolbox.
Thanks Griffin

3 Likes

I think the trick is to delete the old R3 (the one issued by DST Root X3) from your Windows machine under "manage user certificates" > Intermediate certification authorities (possibly under "manage computer certificates" as well), this forces windows to look at the newer R3. If you don't have that one in your "intermediate" store you can get it from https://letsencrypt.org/certs/lets-encrypt-r3.der but try it without.

I'm testing this stuff today as well (by setting a bunch of machines to be in the future!).

5 Likes

We were thinking that we'd do this around the time that DST Root CA X3 expires, but there's no reason not to do it earlier, really. It's now done, and http://r3.i.lencr.org is serving the version of R3 issued by ISRG Root X1, rather than the version signed by DST Root CA X3.

12 Likes

So I'm having an issue with my certs still showing R3 as an intermediate cert which expires 9/21.
When I pull up my full chain - the intermediate is a cert that expires in 2025, but when I click on the certs in Firefox or Chrome - I see the old R3.
I'm probably stupid and missing something, but can someone explain and reassure me all my stuff isn't gonna break next month?

2 Likes

Welcome to the Let's Encrypt Community :slightly_smiling_face:

You likely have the retired R3 (signed by DST Root CA X3) stored in your cache. Try clearing it.

You can use the following tool to see exactly what certificate chain your webserver is serving:

https://certlogik.com/ssl-checker/

3 Likes

thank you for your reply!
When I run checkers such as those found at ssllabs or digicert, they all throw errors about having an invalid chain and having to download the old R3 cert.
I want to believe it's a my browser issue, but I'm really nervous as I've got this in production and not sure if I should bail and go get a different cert for the next year to bridge this gap...

3 Likes

Modern browsers build their own chains. There's lots of logic involved including caching, pre-validation of CA signatures, preloading of CA certificates and many more.

All of this has the effect that the chain displayed by the browser has almost zero relationship with the chain send by the server - the browser effectively does what it wants. In fact, many browser configurations do not even care if you send the intermediates at all*. To diagnose certificate chains, browsers are the wrong tool nowadays.

Older, or more simple TLS clients still rely on the server sending a correct and valid chain - which you're probably doing.

*Some browsers not doing CA preloading need to "learn" the intermediate first, by receiving them once and then remember them (caching). In theory this doesn't need a certificate chain send by server, but can also be done via AIA loading, though I don't have knowledge how this is currently implemented in diffferent browsers.

5 Likes

If you used the tool I gave you and it says that your webserver is not serving the full chain then the problem is that your webserver is not serving the full chain. Your webserver should be serving all of the following (in this exact order):

your leaf cert
https://crt.sh/?id=3334561879
https://crt.sh/?id=3958242236

3 Likes