Help thread for DST Root CA X3 expiration (September 2021)

Help
1209

Actually I did. Start with this article: DST Root CA X3 Expiration (September 2021) - Let's Encrypt

You must not be getting Windows updates (as is the case with me) for it to break. Otherwise an update would have already downloaded the ISRG Root X1 root cert. Chrome, strangely, uses Windows's root certificates, while Firefox manages its own. So FF works no problem. Here is what I did: In FF go to: about:preferences#privacy and scroll to Certificates. Click View and then find ISRG Root X1 root cert. Export it, it will be saved to, say, your Desktop. Then just import it to your Trusted Root Certificate Authorities store in Windows (it's easy, just google it quick). Worked like a charm for me.

3 Likes
1210

Thanks, man!
Gonna try the update and if not the FF method.

2 Likes
1211

It worked! I exported it from another PC that had it and imported it on the problematic PC.
Important confusion is not to double click, but to use the IMPORT function.

3 Likes
1212

Precisely. I double clicked first also.

1 Like
1213

I had some trouble using Python 3.9 on Windows 10
The standard way (up to 2021-09-30) was:

Python 3.9.5 (tags/v3.9.5:0a7dcbd, May  3 2021, 17:27:52) [MSC v.1928 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib3
>>> http = urllib3.PoolManager()
>>> ret = http.request("GET", "https://community.letsencrypt.org")

which now produced the exception

Traceback (most recent call last):
  File "C:\dev\git\playground-python\env\lib\site-packages\urllib3\connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "C:\dev\git\playground-python\env\lib\site-packages\urllib3\connectionpool.py", line 382, in _make_request
    self._validate_conn(conn)
  File "C:\dev\git\playground-python\env\lib\site-packages\urllib3\connectionpool.py", line 1010, in _validate_conn
    conn.connect()
  File "C:\dev\git\playground-python\env\lib\site-packages\urllib3\connection.py", line 411, in connect
    self.sock = ssl_wrap_socket(
  File "C:\dev\git\playground-python\env\lib\site-packages\urllib3\util\ssl_.py", line 449, in ssl_wrap_socket
    ssl_sock = _ssl_wrap_socket_impl(
  File "C:\dev\git\playground-python\env\lib\site-packages\urllib3\util\ssl_.py", line 493, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
  File "C:\dev\Python\Python39\lib\ssl.py", line 500, in wrap_socket
    return self.sslsocket_class._create(
  File "C:\dev\Python\Python39\lib\ssl.py", line 1040, in _create
    self.do_handshake()
  File "C:\dev\Python\Python39\lib\ssl.py", line 1309, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1129)

Note: On Ubuntu 20.04 the above snippet still works out of the box, it was just Windows that caused trouble.

For Windows using certifi solved my problem.

>>> import certifi
>>> import urllib3
>>> http = urllib3.PoolManager(cert_reqs="CERT_REQUIRED", ca_certs=certifi.where())
>>> ret = http.request("GET", "https://community.letsencrypt.org")
>>> ret.status
200
6 Likes
1214

Hi @mpapenbr, welcome to the LE community forum :slight_smile:
[and thank you for bringing the information of that problem and your solution to it into this forum]

6 Likes
1215

Thank you so much everyone.

Using snap to run a newer version of certbot, and telling it to use --preferred-chain "ISRG Root X1" worked like a charm. Now my phone is back to happily running DOT through Stunnel.

The preferred chain option was automatically added to /etc/letsencrypt/renewal/MyServer.conf as:
preferred_chain = ISRG Root X1

So on next renewal it will not break the chain.

2 Likes
1216

So then what did you do after getting rid of ca x3, recreate the certificate?

Wayne Sallee

1 Like
1217

I got:
update-ca-certificates
/usr/share/ca-certificates/mozilla/DST_Root_CA_X3.crt not found, but listed in /etc/ca-certificates.conf.

So in /etc/ca-certificates.conf I changed
mozilla/DST_Root_CA_X3.crt
to
!mozilla/DST_Root_CA_X3.crt

Then re-ran
update-ca-certificates

The fact that my server is still running debian 9 is playing into the problem. Time to upgrade.

Wayne Sallee

2 Likes
1218

My /etc/ca-certificates.conf is dated 25th September and the mozilla/DST_Root_CA_X3.crt entry already was commented out with a '!'.

However, it is also listed in /snap/core18/2128/etc/ca-certificates.conf and I've already found that trying to change that results in it being marked 'broken'.

1 Like
1219

If it can help anyone, the problem of wrong chain response for asp.net application in my case resolved deleting certificates cache, on linux machine /home/username/.dotnet/corefx/cryptography/x509stores/ca
and restarting application

2 Likes
1220

A post was split to a new topic: Nginx with preferred-chain "ISRG Root X1" - still showing "DST Root CA X3"

1221

A post was split to a new topic: Errors using FPDF to make PDFs from websites

1222

A post was split to a new topic: Alpine Linux issue

1223

A post was split to a new topic: Splitting Wishes

1224

wouldn't we have the same problem at some point in the future if we switched to another CA? or was there a problem the way letsencrypt implemented this?

getting pressure from my higher-ups to make our site work for older clients and it sounds like the only way is to switch CAs?

1 Like
1225

Yes, this can happen to any CA that experiences a root expiry.
Switching to a newer root implies that clients made/unmodified from before the replacement root was issued would have no knowledge of that second root (only the first one - which has then expired).

But of course this example/lesson will serve plenty to all that may come after.
The first time it is a mistake.. unavoidable/unknown situation.
The second time - lord help them!

Older clients should be able to work with the default chain (through "DST Root CA X3").
But if you had to switch off that (to "ISRG Root X1") because newer clients had problems...
Then you can't really satisfy them both with the current LE chains.
I'd switch to an alternate (free) CA that supports ACME protocol. Like: ZeroSSL.com or BuyPass.com

2 Likes
1226

Inevitably yes, probably. In an ideal situation, you have multiple certificates in all the root stores, old and new, overlapping each other with much overlap.

However, Let's Encrypt is relatively new. The ISRG Root X1 is already in all major root stores for more than 5 years. (!!!) Unfortunately, not every piece of software has been updated.. And LE didn't get a new cross-sign: it was time to stand on their own.

Now, you can blame Let's Encrypt for that. But IMO that's not fair. And it's probably also a matter of taste. But in my opinion if your software is SO old it doesn't work properly at the moment, that device shouldn't be on the internet anyway.

4 Likes
1227

yes totally agree the client should be updated or taken offline. problem is if my competitors allow traffic from these older clients and we do not then we lose customers. trying to find a way in haproxy to identify these clients and redirect them to a webpage but not having much luck.

1 Like
1228

Inconvenient but a pre-connection page with two continue buttons might do the trick.
One button served from an FQDN that has cert trust path #1
Other button served from another FQDN that has cert trust path #2
If they can see the button, they can reach the site (that serves it).

Hi sorry for the construction inconvenience, but we are working on the site and you only need to enter it via the door that is open/working for you today.
[button one] [button two]
Again, sorry for all the dust and noise too...

:jackhammer:
:drill:
:wood:
:sheetrock:
LOL

2 Likes