Help thread for DST Root CA X3 expiration (September 2021)

? ? ?

I'm not arguing about what that file contains or not.
What I've said, and the test show, is that the system is NOT serving a trusted chain.
This is all of what I get directly from it:

-----BEGIN CERTIFICATE-----
MIIFIjCCBAqgAwIBAgISA6bJd9Q/QePvDBKsI+penHAdMA0GCSqGSIb3DQEBCwUA
MDIxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQD
EwJSMzAeFw0yMTA5MzAxMDQwMzVaFw0yMTEyMjkxMDQwMzRaMBkxFzAVBgNVBAMT
Dmh1YjEuY2V0LmNsb3VkMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA
2QKa90ruzeJvuDpsz3xLGZecF8xIwOhF2WCgWpj1b0rk7m9eWHJ/CnyvL2bkPYgj
xwIJ1SnhoIhRoUg2acVmw7HeP8/2ixJ4w+B4SBcsfnHk+yF/W0PY11hSCyikDmoH
FAFygZPKjsW6pcvoYvkaH5TjkoKLoAatCELPgA8+bDGvbus4KwKFuqclvHxH4602
2m1IVgdUrm0Yod4bhVzJVjz8Mbn/q9E9lEUZAZNH5nDOdF9Xgjb1YdQJdNFwUfaH
hm01XFv1huYTb6IhhYXyOU8xUxuK4pKGSFIhVdmlnjVZwsWNzm01vgy5/F13mnGa
IwNiSlsHFKcF46dJ1YXPGwIDAQABo4ICSTCCAkUwDgYDVR0PAQH/BAQDAgWgMB0G
A1UdJQQWMBQGCCsGAQUFBwMBBggrBgEFBQcDAjAMBgNVHRMBAf8EAjAAMB0GA1Ud
DgQWBBScx2SaR2DlHI8kNlJzXHTkSkIkVDAfBgNVHSMEGDAWgBQULrMXt1hWy65Q
CUDmH6+dixTCxjBVBggrBgEFBQcBAQRJMEcwIQYIKwYBBQUHMAGGFWh0dHA6Ly9y
My5vLmxlbmNyLm9yZzAiBggrBgEFBQcwAoYWaHR0cDovL3IzLmkubGVuY3Iub3Jn
LzAZBgNVHREEEjAQgg5odWIxLmNldC5jbG91ZDBMBgNVHSAERTBDMAgGBmeBDAEC
ATA3BgsrBgEEAYLfEwEBATAoMCYGCCsGAQUFBwIBFhpodHRwOi8vY3BzLmxldHNl
bmNyeXB0Lm9yZzCCAQQGCisGAQQB1nkCBAIEgfUEgfIA8AB2AH0+8viP/4hVaCTC
wMqeUol5K8UOeAl/LmqXaJl+IvDXAAABfDaBmJgAAAQDAEcwRQIhAJY0gQ/vQEWR
1fwn+ML/RA1BKvqTtis7pVUnK/JMtVlZAiAGgtsPJ/IvB66eidubtJK5yHM9aIoo
CZsB3bLNUx0KxwB2AG9Tdqwx8DEZ2JkApFEV/3cVHBHZAsEAKQaNsgiaN9kTAAAB
fDaBmUIAAAQDAEcwRQIhAP9nXqL7HtpVBYBSbcPb2kiWyDpGzJe1rSubYd/r/pHA
AiA0bI92IbJx4wW3P6lPrs/c1feK4P7qH2pDnZelCboz8zANBgkqhkiG9w0BAQsF
AAOCAQEApasjJEtVS+k/xon8uzzY2r9e3ogwKOShEX1y+O/8q8y2adaVNh6/R8rz
/A7Fja4EwnMfEBIlew6SzLWBKWoJ3oF2TXee2/E0Z6/gBC0imlBG/po1RwVrldck
Zh7TSAAlqjNZfPsz+CSuMXqZ4/D30lgLpB0nrqL1rNxJrhLphNSOzkOmNRF3xokM
dzCsOl5+9k2MXWQqAZSEXXKdt/qVgc2CVzD3Q5KODsPv8nNNsQcq0+Bj5VAdO3Nv
nrD5lTVs31Ykf4BXslw8vEma5SDxnhNXOoW7TJOtMLE+f2mdLa+6mNZazyBv8Uza
tGgku+XQSB2XAEH0xY6+2uEbGCxQxQ==
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIEZTCCA02gAwIBAgIQQAF1BIMUpMghjISpDBbN3zANBgkqhkiG9w0BAQsFADA/
MSQwIgYDVQQKExtEaWdpdGFsIFNpZ25hdHVyZSBUcnVzdCBDby4xFzAVBgNVBAMT
DkRTVCBSb290IENBIFgzMB4XDTIwMTAwNzE5MjE0MFoXDTIxMDkyOTE5MjE0MFow
MjELMAkGA1UEBhMCVVMxFjAUBgNVBAoTDUxldCdzIEVuY3J5cHQxCzAJBgNVBAMT
AlIzMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAuwIVKMz2oJTTDxLs
jVWSw/iC8ZmmekKIp10mqrUrucVMsa+Oa/l1yKPXD0eUFFU1V4yeqKI5GfWCPEKp
Tm71O8Mu243AsFzzWTjn7c9p8FoLG77AlCQlh/o3cbMT5xys4Zvv2+Q7RVJFlqnB
U840yFLuta7tj95gcOKlVKu2bQ6XpUA0ayvTvGbrZjR8+muLj1cpmfgwF126cm/7
gcWt0oZYPRfH5wm78Sv3htzB2nFd1EbjzK0lwYi8YGd1ZrPxGPeiXOZT/zqItkel
/xMY6pgJdz+dU/nPAeX1pnAXFK9jpP+Zs5Od3FOnBv5IhR2haa4ldbsTzFID9e1R
oYvbFQIDAQABo4IBaDCCAWQwEgYDVR0TAQH/BAgwBgEB/wIBADAOBgNVHQ8BAf8E
BAMCAYYwSwYIKwYBBQUHAQEEPzA9MDsGCCsGAQUFBzAChi9odHRwOi8vYXBwcy5p
ZGVudHJ1c3QuY29tL3Jvb3RzL2RzdHJvb3RjYXgzLnA3YzAfBgNVHSMEGDAWgBTE
p7Gkeyxx+tvhS5B1/8QVYIWJEDBUBgNVHSAETTBLMAgGBmeBDAECATA/BgsrBgEE
AYLfEwEBATAwMC4GCCsGAQUFBwIBFiJodHRwOi8vY3BzLnJvb3QteDEubGV0c2Vu
Y3J5cHQub3JnMDwGA1UdHwQ1MDMwMaAvoC2GK2h0dHA6Ly9jcmwuaWRlbnRydXN0
LmNvbS9EU1RST09UQ0FYM0NSTC5jcmwwHQYDVR0OBBYEFBQusxe3WFbLrlAJQOYf
r52LFMLGMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggrBgEFBQcDAjANBgkqhkiG9w0B
AQsFAAOCAQEA2UzgyfWEiDcx27sT4rP8i2tiEmxYt0l+PAK3qB8oYevO4C5z70kH
ejWEHx2taPDY/laBL21/WKZuNTYQHHPD5b1tXgHXbnL7KqC401dk5VvCadTQsvd8
S8MXjohyc9z9/G2948kLjmE6Flh9dDYrVYA9x2O+hEPGOaEOa1eePynBgPayvUfL
qjBstzLhWVQLGAkXXmNs+5ZnPBxzDJOLxhF2JIbeQAcH5H0tZrUlo5ZYyOqA7s9p
O5b85o3AM/OJ+CktFBQtfvBhcJVd9wvlwPsk+uyOy2HI7mNxKKgsBTt375teA2Tw
UdHkhVNcsAKX1H7GNNLOEADksd86wuoXvg==
-----END CERTIFICATE-----

That second cert expired yesterday!

2 Likes

Hi there,

we are using IIS 10.0 and have to support old Android devices, so it is important that our server provides both paths, including the "New Default Chain" with IdenTrust DST Root CA X3.

Sadly, the IIS removes the path including the expired IdentTrust Certificate.

The Workaround we found was putting the "ISGR Root X1" in the Untrusted Store of the Windows Certificate Manager as described here and reboot the server.

Is there a better way to do this? This way we are unable to renew the certificates using win-acme.

Thanks!

1 Like

It is astounding, but explains perfectly what happens. Please, can you tell me how you have obtained that response (two certificates, second being probably the obsolete R3)?

1 Like

openssl s_client -connect EXAMPLE.COM:443 -servername EXAMPLE.COM -showcerts
[simply copy out the certs shown]

2 Likes

Hello, we are using centos 7. I updated the ca-certificates package and checked to make sure it's removed trust list | grep -C3 'DST Root CA X3', however, I'm still seeing the root certificate listed when I check our sites:

Is a server reboot required, or is there another way to clear this certificate from the chain?

99% of our users can access the site without any issue. We are having issues with users that have older device (mostly old MacBooks)

1 Like

To all webhosts looking to continue supporting older clients, I suggest that

  1. you confirm the root certificate you want to use is on the system you want to support, example:
    OS X Mavericks: List of available trusted root certificates - Apple Support (IL)

  2. you configure your ACME client to use a different CA server

example: SSL | Buypass Go SSL – Technical information | Buypass CA

when an embedded certificate expires, if the device isn't getting updated, that's that. check the expiration date of the embedded root certificate on the older system you want to support, and the browser is probably deriving it's trusted root from the operating system.
exception: firefox, but firefox accepts expired trusted root certs, unlike chrome, so there's that.

1 Like

we're using HAProxy 2.4.4 and seeing some older macs and ubuntu 15 seeing cert expired messages when they go to docucopies.com is there a config change I can make in haproxy that will fix that?

1 Like

Thank you, I will try to find which subject is substituting R3 certificate. Thank you again.

2 Likes

Hi @deckberg-qd, welcome to the LE community forum :slight_smile:

That site is using the exact same trust path (chain) that this forum is using.
You have only three choices:

  • upgrade the clients to work with this trust path (chain)
    [may not even be possible now, or ever (for some no-longer-supported clients)]
  • modify all the servers to use a different trust path (chain)
    this isn't very likely; as you can't possibly control the very large number of systems on the Internet that are using this same trust path (chain)
  • put all your troubled clients behind a proxy (one that terminates the TLS connections
    that may keep those clients within your trust path (chain) control; as they would only see the proxy

[there may be other ways... I'm running low on :zzz: ]

2 Likes

for your 2nd bullet. am I thinking correct that I can update certbot to 1.12 and put in the correct whatever for --preferred-chain ?

sorry, I'm not trying to make the clients work everywhere on the net, I just want my older customers to be able to get to my website docucopies.com without the expired cert warning

1 Like

For the servers under your control, sure.
"a different trust path" can includes using other FREE, or paid, CAs [all valid choices]

Many ways to slice that pie!

2 Likes

Looks like SSL labs is saying there is and issue with one of the paths. So I need to include the cross signed cert somewhere?

https://www.ssllabs.com/ssltest/analyze.html?d=docucopies.com

1 Like

Your server is already sending the cross-signed cert. It's #3 in the second section in your screenshot. That's part of the "default" chain Let's Encrypt sends to your ACME client.

The --preferred-chain option you alluded to would be the way to get the "alternate" chain that removes that cross-signed cert and also then breaks compatibility with old Android devices. Whether that is an improvement for your overall userbase, only you can determine.

5 Likes

so there's nothing I can do server side to have those older clients be able to connect? it's simply up to the client to update their trust store if they are able?

1 Like

If those older clients do not trust the ISRG Root X1 cert, the only thing you can do server side is change to a different Certificate Authority that they do trust. Personally, I would try switching to the shorter chain first and seeing how those clients respond. But again, that will definitely break old Android devices.

6 Likes

For those looking for a fix with NodeJS 8, i've opened an issue on the github, but probably not going to do anything.

In the post i've writen the fix, but you gonna have to build your own version of Node unless they fix it.

3 Likes

I can Confirm a fix that worked on both a 10.9.5 & 10.11.6 Mac OS users. Simply set the DST Root CA X3 to "Always Trust" on several Mac's I manage in an office and home's this fix work for 4 websites that previously had issues with this CERT ERR.

https://www.nynewspapers.com/

Directions for fix:

  1. Open ~/Applications/Utilities/Keychain Access.app
  2. From View menu select "Show Expired Certificates"
  3. On the Left Sidebar pick System Root
  4. In search bar top-right type DST
  5. Double-click "DST Root CA X3"
  6. In pop-up, turn down "Trust" arrow and set "When using this certificate" to "Always Trust"
  7. Close the pop-up and put in an Administrator user/password info.
  8. Close all open Browsers & Keychain you should be good to go after that.

5 Likes

Hello Everyone!

This expiry of DST Root caused me to really look at and understand my own setup :slight_smile: And thank you all for helping each other here!

I had two issues:

  1. I had to remove the DST Root from my system store on the server because openssl s_client kept failing my tests.
  2. The existing fullchain.pem files still had the DST Root CA X3 certificate in them which caused my webserver to continue to send it, causing failures on some clients. The solution was to force a renewal of the certificate.

I'm not using the apache installer and authenticator, but instead use the webroot athenticator as I found it better when using a mix of services:

# certbot --webroot --dry-run -w /var/www/domains/wiki.tnonline.net/htdocs -d wiki.tnonline.net -v certonly

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Certificate not yet due for renewal

You have an existing certificate that has exactly the same domains or certificate name you requested and isn't close to expiry.
(ref: /etc/letsencrypt/renewal/wiki.tnonline.net.conf)

What would you like to do?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: Keep the existing certificate for now
2: Renew & replace the certificate (may be subject to CA rate limits)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate number [1-2] then [enter] (press 'c' to cancel): 

I found that certtool from the GnuTLS package is better to look at the full chain than with openssl:

certtool -i --infile /etc/letsencrypt/live/wiki.tnonline.net/fullchain.pem |less
1 Like

14 posts were merged into an existing topic: Ubuntu Android problem

This is causing lftp to fail.

Certificate: C=US,O=Internet Security Research Group,CN=ISRG Root X1
Issued by: O=Digital Signature Trust Co.,CN=DST Root CA X3
ERROR: Certificate verification: Not trusted

Wayne Sallee
Wayne@WayneSallee.com

1 Like