Thank you for your answer.
Our Zimbra server is very productive and actually the mail server one of our client.
Unfortunately we have to wait until next cert renew and see what will happened.
Zimbra forum don't have yet something similar. I'll try there and came back for new information.
Thank you.
4 Likes
I have one question, which is not answered on the docs page: for now, it looks like my certificates are signed by R3 which indicates an issuer of DST Root CA X3; when will the new chain be used in renewals?
Those details seem to be available in this other topic: https://community.letsencrypt.org/t/production-chain-changes/150739; would it be possible to add those details on the help page?
4 Likes
If you have received a notification email containing an unfamiliar domain name, please see the Let's Encrypt staff message here:
5 Likes
Welcome to the Let's Encrypt Community 
Great suggestion! I've submitted changes to reflect this to DST Root CA X3 Expiration (September 2021) - Let's Encrypt that are awaiting review.
5 Likes
Does this issue affect all vendor-supplied versions of OpenSSL 1.0.x? Or have some vendors patched theirs? I'm curious in particular about Ubuntu's 1.0.2g-1ubuntu4.19 which comes with 16.04
We're a bit behind with upgrading our systems unfortunately (16.04 was supported till April this year).
5 Likes
There are some details on how OpenSSL will break when DST Root X3 expires if a server is sending the default chain in this thread:
And a questions for it thread, though it looks like it's since been closed:
I haven't heard of any specific vendor having backported a fix for it to older OpenSSL, but that doesn't mean it hasn't happened.
If your OpenSSL 1.0.x-based system is only connecting to servers you control, and those servers don't need to maintain compatibility with old Android, then you probably want your servers to be using the "alternate" chain that doesn't include the expiring DST Root.
9 Likes
I've heard that OpenSSL 1.0.2 can be manually fixed by setting the flag X509_V_FLAG_TRUSTED_FIRST. This flag is not set by default in 1.0.2, but you might be able to do this in your client applications (it can also potentially be done via config file nope, doesn't look like it).
I haven't tested this myself, as all my systems have OpenSSL 1.1, where the flag is set by default.
7 Likes
I've requested that topic be reopened to allow further questions (as I believe it was intended to be).
7 Likes
Hello,
"If you run a typical website, you won’t notice a difference - the vast majority of your visitors will still accept your Let’s Encrypt certificate."
Based on my Analytics, I still have a good % of users on Android < 7.1 and iOS < 10.
Sure, "vast majority" of the users will be accepting the new certificates, but a good significant amount will be affected.
"To make sure the certificates we issue are trusted on older devices, we also have a “cross-signature” from an older root certificate: DST Root CA X3."
"DST Root CA X3 will expire on September 30, 2021."
I understand your position, and that this is the reason.
However (and I apologize for the ignorance), is there no workaround for this, such as cross-signing to some other older widely accepted root CA that is still active?
Are other certificates such as Comodo, Verisign, etc, being affected by the same problem?
Thank you!
2 Likes
We hear your pain and wish we could provide broader compatibility for longer, but we concluded it wasn't feasible for us. This workaround was requested and discussed recently in this thread which offers some more detail and options.
In the next few years, other certificates authorities will have their roots expire. This is a deep dive on the problem space that offers information about upcoming root expirations: The Impending Doom of Expiring Root CAs and Legacy Clients
6 Likes
To clarify, the users on Android < 7.1 will be fine, due to the special cross-sign we got. But it's true the users on iOS < 10 will have problems. In theory all iPhone 5 and later can upgrade to at least iOS 10, but I know there are a variety of reasons people may not install all updates.
5 Likes
Do you have any recommendations for testing client connectivity prior to the certificate expiring?
For example, is there a test web server available using only the ISRG Root X1 root certificate? Or a method to request a certificates without the DST Root CA X3?
Thanks.
2 Likes
Isn't that special cross-sign with DST Root CA X3, which is expiring this year?
I don't understand how in 2020, the article says: "IdenTrust has agreed to issue a 3-year cross-sign for our ISRG Root X1 from their DST Root CA X3.", if that root is expiring in Sep 2021.
Did I misunderstand something?
Thank you!
3 Likes
I'm not sure if it's misunderstanding or you simply stopped reading. The text gives an explanation why this works.
3 Likes
My apologies. My answer is there!
"Android has intentionally chosen not to use the notAfter field of trust anchors."
Thank you very much.
5 Likes
You can test what happens with a chain that only leads to ISRG Root X1 today, your ACME client might support an option to have it sort this out for you, or you can manually construct the chain, the individual certificates are signed documents, so you cannot change those, but the "fullchain.pem" file for example is just several certificates concatenated together, so if you remove a certificate from the file now it's not in your chain.
However, it will not be possible to check what happens exactly in September 2021, without travelling to September 2021. If you tell a computer today that it's already September 2021, it will say your certificates aren't trusted because they're out-of-date - you got them back in May (ie now). We can do some better experiments on the details in July and I believe there are plans for just that.
3 Likes
You can already test this today, if you have control over both client and server that you're testing -
You can issue a cert from Let's Encrypts staging enviroment. Staging is specifically configured to serve a chain up to a root that is already expired. However, the staging certificates are not trusted, so you need to temporarily add the Let's Encrypt staging certificates to your trust store do test. (Remove them after you're done).
The test certificates used in staging are documented here: Staging Environment - Let's Encrypt
I host a test server that is configured to serve a staging chain, if you don't want to set that up yourself: https://expired-root-ca-test.germancoding.com. Remember to add the staging certificates to your trust store when testing, and don't forget to remove them after that.
PS: As I just saw that the LE staging documentation is a bit confusing, here's the staging root certificates you need to add for testing.
https://letsencrypt.org/certs/staging/letsencrypt-stg-root-x1.pem (This is a simulated "ISRG Root X1" certificate)
https://letsencrypt.org/certs/staging/letsencrypt-stg-root-dst.pem (This is a simulated "DST Root CA X3" that is already expired. Adding this to your trust store enables testing on how your trust store handles expired root certificates)
9 Likes
There is a test server that sends only the information to chain up to the ISRG Root X1 root: https://valid-isrgrootx1.letsencrypt.org/. It sends the leaf and the R3-signed-by-ISRG-Root-X1 intermediate, and so should be trusted if your client trusts ISRG Root X1. (But note that browsers may build other chains, so if you look in your web browser's certificate info you might also see the certificate of ISRG-Root-X1-signed-by-DST-Root-CA-X3 because your browser had seen that certificate elsewhere, even though the server itself didn't send it.)
There's also a test site that's configured with the "normal" chain rooted in DST Root CA X3, https://helloworld.letsencrypt.org/, but it hasn't renewed since the May 4 chain change so it's still using R3-signed-by-DST-Root-CA-X3 rather than the combo of R3-signed-by-ISRG-Root-X1 plus ISRG-Root-X1-signed-by-DST-Root-CA-X3.
Yes, if you request the "alternate" chain in your ACME client, it will send the chain without the ISRG-Root-X1-signed-by-DST-Root-CA-X3 certificate, so that your "root" can be the self-signed ISRG Root X1 in your trust store. This actually hasn't changed, and isn't expected to change for quite some time. The "valid-isrgrootx1" site mentioned above is set up the same way as a site that uses this approach. (See the great Production Chain Changes announcement that summarizes the plan.)
For the production chain, that is pretty true. But there is a staging environment, which is set up with a "mirror" of the real certs, except that the certificate corresponding to the DST Root CA X3 (called "(STAGING) Doctored Durian Root CA X3") is already expired.
So, what one could do, is have whatever system you're testing have both "(STAGING) Doctored Durian Root CA X3" and "(STAGING) Pretend Pear X1" in its trust store (be sure to remove after testing, as certificates from staging aren't audited or anything), or whichever combination of roots corresponds to the scenario you want to test. Then you can try using it to connect to systems that have been issued certificates from the staging environment (either the "normal" or "alternate" chain) to see what happens.
(Looks like @Nummer378 already said this while I was typing this up, but I already typed it so I'm leaving this here. 
)
@jple: Just bugging you again. Why isn't "(STAGING) Doctored Durian Root CA X3" mentioned or downloadable from the Staging Environment documentation? I'd think that needing to deal with it being or not being in one's trust store is the kind of thing that one would often be using the staging environment to test.
9 Likes
Is https://valid-isrgrootx1.letsencrypt.org/ an example of the new cert chain, including the Android workaround? I'd be interested to develop a Javascript routine that could reliabily test clients so that we might be able to show warnings to human users browsing our websites before Sep 30. (We did something similar for TLS 1.2 support in advance of disabling TLS <= 1.1 on our httpds.)
Ideally there'd be an https site up now using the new chain (with the Android workaround) hosting either a CORS-compatible URL returning a simple JSON object like {"success":true} that we could test with XmlHttpRequest or a 1x1 pixel image that could be tested with event handlers on an IMG element.
Thanks so much for all your work!
4 Likes
No, valid-isrgrootx1 is an example of the "alternate" chain, with ISRG Root X1 as the root.
You're unlikely to be able to reliably test web browsers, including mobile web browsers, because they go through extraordinary efforts to "guess" and "find" intermediates that aren't actually being sent by the web server, in order to work around that a lot of servers have incorrect configurations. (See for instance, what Firefox does to work around poorly-configured servers, and other web browsers do similar and different things.) So you can have two identically-configured devices, but one happened to go to a web site in the past to help it "know" about an intermediate, but the other didn't, and so when going to a server that should send that intermediate but doesn't, one will work but the other will fail. (At least, until the intermediate falls out of its "cache" on the first device. Maybe.) Testing from within a browser kind of requires it to be a "fresh" factory-reset device to have any sense of repeatability, and then of course you're not going to see much about how "real-world" usage will work.
The only "official" site that I know of is https://helloworld.letsencrypt.org, or maybe if you're looking for JSON you could try getting to https://acme-v02.api.letsencrypt.org/directory (I'm not sure what chain the API will be using long-term, though). It should be easy enough to spin up your own, though, since after all getting Let's Encrypt certificates are free. The default chain is, and will continue to be, rooted in DST Root CA X3. The tricky part is just that it's hard to test what will happen once that certificate expires later this year, without using the mirrored certificates from the staging environment as described above and messing with your device's trust store.
9 Likes