Help thread for DST Root CA X3 expiration (September 2021)

wouldn't we have the same problem at some point in the future if we switched to another CA? or was there a problem the way letsencrypt implemented this?

getting pressure from my higher-ups to make our site work for older clients and it sounds like the only way is to switch CAs?

1 Like

Yes, this can happen to any CA that experiences a root expiry.
Switching to a newer root implies that clients made/unmodified from before the replacement root was issued would have no knowledge of that second root (only the first one - which has then expired).

But of course this example/lesson will serve plenty to all that may come after.
The first time it is a mistake.. unavoidable/unknown situation.
The second time - lord help them!

Older clients should be able to work with the default chain (through "DST Root CA X3").
But if you had to switch off that (to "ISRG Root X1") because newer clients had problems...
Then you can't really satisfy them both with the current LE chains.
I'd switch to an alternate (free) CA that supports ACME protocol. Like: ZeroSSL.com or BuyPass.com

2 Likes

Inevitably yes, probably. In an ideal situation, you have multiple certificates in all the root stores, old and new, overlapping each other with much overlap.

However, Let's Encrypt is relatively new. The ISRG Root X1 is already in all major root stores for more than 5 years. (!!!) Unfortunately, not every piece of software has been updated.. And LE didn't get a new cross-sign: it was time to stand on their own.

Now, you can blame Let's Encrypt for that. But IMO that's not fair. And it's probably also a matter of taste. But in my opinion if your software is SO old it doesn't work properly at the moment, that device shouldn't be on the internet anyway.

4 Likes

yes totally agree the client should be updated or taken offline. problem is if my competitors allow traffic from these older clients and we do not then we lose customers. trying to find a way in haproxy to identify these clients and redirect them to a webpage but not having much luck.

1 Like

Inconvenient but a pre-connection page with two continue buttons might do the trick.
One button served from an FQDN that has cert trust path #1
Other button served from another FQDN that has cert trust path #2
If they can see the button, they can reach the site (that serves it).

Hi sorry for the construction inconvenience, but we are working on the site and you only need to enter it via the door that is open/working for you today.
[button one] [button two]
Again, sorry for all the dust and noise too...

:jackhammer:
:drill:
:wood:
:sheetrock:
LOL

2 Likes

Well! I have very serious issues with connecting from iOS devices (ver 14 and 15) to adobe connect servers. I have tried many ways and yet no sucess. As we we are providing online education medium, an urgent answer is a huge help.
Looking forward to hearing from you.

1 Like

Hi @sherrikalak,

ISRG Root X1 has been supported since iOS 10 and so in principle these devices should still be able to interoperate with sites using Let's Encrypt certificates. An error in connecting would probably mean that the intermediate certificate sent by the server is incorrect. (For example, it could have hardcoded an expired intermediate certificate, or fail to send any intermediate certificate at all.)

Could you give an example of a domain name that you're having problems connecting to? Alternatively, you could test with https://ssllabs.com/ and look for any errors in the "Chain issues" section, which would likely provide a useful diagnosis.

4 Likes

Hi @sherrikalak welcome to the LE community forum :slight_smile:

That is key to understanding the problem.

2 Likes

A bit off topic but...

It's unfortunate that IdenTrust didn't consider the issues they would cause when they issued their root certificate on September 30, 2000.

That aside, this is a good thing for web PKI. Many root certificates were issued in the early 2000s, this type of expiry is going to happen much more frequently.

Because of this unavoidable event, Let's Encrypt is shaking things up a bit and seeing what bolts fall out. We can learn from this event so future root expiration are handled more gracefully.

EDIT: I do not appreciate my posts being edited without my knowledge. Please see the original quoted context here Help thread for DST Root CA X3 expiration (September 2021) - #1208

3 Likes

Thanks Schoen! Your tip about ssllabs.com saved me! I directed the admins to repair the chain and it seems it worked!

5 Likes

The reissue problem with acme.sh drove me nuts.

According to https://github.com/acmesh-official/acme.sh/wiki/Preferred-Chain the default is still 'DST Root CA X3'.

But even when I force reissued with --preferred-chain "ISRG Root X1" or Le_PreferredChain='ISRG Root X1' I got the fullchain with the now expired 'DST Root CA X3' in the chain

Certificate chain
 0 s:/CN=domain.tld
   i:/C=US/O=Let's Encrypt/CN=R3
 1 s:/C=US/O=Let's Encrypt/CN=R3
   i:/C=US/O=Internet Security Research Group/CN=ISRG Root X1
 2 s:/C=US/O=Internet Security Research Group/CN=ISRG Root X1
   i:/O=Digital Signature Trust Co./CN=DST Root CA X3

Somebody said that they had to re-issue the certificate. Since force didn't work. I tried to delete and issue again only to learn that acme.sh now uses zerossl by default.
https://community.letsencrypt.org/t/the-acme-sh-will-change-default-ca-to-zerossl-on-august-1st-2021/144052

Came across the switch to continue using letsencrypt

acme.sh --issue --domain 'domain.tld' --dns dns_pdns --debug --server letsencrypt

Trying to switch back with this is still unconfirmed since I had to give up because of the rate limits.

I really would have appreciated LE raising those limits during such vital changes.

For now I had to switch to zerossl to keep everything running

And I still don't know if it will be fine next time since I can only continue my tests next week.

1 Like

did you upgrade to the verison:

acme.sh --upgrade
3 Likes

Yes, I did that after the first trying to just re-issuing it. I mean the certificate chain get's generated by the API from letsencrypt, they would know what goes into the chain, right :wink:

With the upgraded version I tried to delete (rename the conf file) the certificate and reissue it. This came up with a certificate from zerossl. This might be fine for many. I didn't want to research this switch right now and wanted back into the letsencrypt ecosystem.

1 Like
acme.sh  --set-default-ca  --server letsencrypt
4 Likes

Yes, I hope this is going to be it. Can't try it right now for this domain, because of rate limiting.

1 Like

Hi! Newbie in this forum. Kindda need some help which I think is related to this topic. Our users keep on seeing Connection not private.

We're experiencing this also with our learning management systems. Good thing that it's not happening in the majority of our users. We've tried some troubleshooting which only worked on a couple of users.

Those troubleshooting are Clear Cache and Cookies, uninstall and reinstall their main browser, double checking if the time & calendar of their desktop is sync, made sure that their browsers are up to date.

I know this isn't much related to the clients OS but those users of our system that affected by it are the ones using windows 7 with a bit old PC, and a handful of windows 10 with updated PCs.

But we still have many users which uses windows 7 with old PCs which didn't encounter this connection not private issue.

1 Like

The fact that there is still the expired X3 certificate in the full chain causes .NET / Mono / Xamarin classes like WebClient, HttpWebRequest and others to fail.

An attached ServerCertificateValidationCallback receives sslPolicyErrors=RemoteCertificateChainErrors as a parameter with 4 certificates in the "chain" parameter (my own, the intermediate, the ISRG Root X1 and the expired DST Root CA X3).
Microsoft's example code on RemoteCertificateValidationCallback Delegate (System.Net.Security) | Microsoft Docs shows to just check for sslPolicyErrors.
There is no way for my application (and probably .NET/Mono/Xamarin) to know which of the certificates in the chain are trusted root certificates and when it MUST early-exit before it fails on expired roots.

Our server APIs are currently not accessible from mobile Android/Xamarin clients due to this and I have no idea how to fix this other than moving to a different CA without expired roots in the chain.

1 Like

3 posts were merged into an existing topic: I have IdenTrust 1 and my Mac OSX is still screwed

7 posts were split to a new topic: Long chain gives warnings in Qualys SSL Server Test

So, that is showing only path #1. That may indeed work for us as we have a ver specific list of browser versions we support for out sites. How do I change my certs to only use/serve the good chain? And if need be, how to switch back?

1 Like