We had our certs generated from vancluever/acme and running on nginx using Nginx Ingress Controller on Kubernetes and it failed today, after DST -> R3 cert expired, and now I have recreated the cert to use ISRG Root X1 as preferred chain and applied it, the issue is on chrome, website is still showing cert failed, if i see the chain it is DST Root -> R3 > cert, but if I open in Firefox, it gives ISRG -> R3 -> cert which works fine, also I tried the cert on openssl,
openssl s_server -accept 8080 -www -cert tls.crt -key tls.key -CAfile ca.crt
it shows correct chain, what am i missing?
its not working on Chrome for most of the users but working fine for some?
I am also having issues with the change that was made today.
In my case I seem to somehow lose the parent "DST Root CA X3" in the certificates path when I convert it from pem to pfx.
I had the same problem under MacOS Big Sur 11.6. Found that the Intermediate CA Let's Encrypt R3 certificate (Cert) was missing from the Keychain. Using Firefox, which uses its own Cert store, I was able to download the R3 Cert .pem file. Added the R3 Cert to the MacOS Keychain by simply double-clicking on the file.
If you trust me to give you the right PEM file I've uploaded it here.Lets-Encrypt-R3.pem (1.8 KB)
Although I suggest you really do this by chasing down the Cert yourself. Brief instructions:
Using Firefox go to the site having the issue (does not work in Chrome or Safari)
Click on the Padlock in the Address Bar
Click on Connection Secure
Click on More Information
Click on the View Certificate Button
Under the Certificate section click on the R3 Tab
Scroll down to the Miscellaneous Section
From the Download Item click on PEM (cert)
Allow the download - note it will name the file with the FQDN of the site visited in step 1.
The file is really the Let's Encrypt R3 Cert in PEM format - simply double click the file
Verify that the Cert has been added to the Keychain - find them by entering R3 in the Keychain seach-bar at the upper right
There should be two:
R3 (this is the Let's Encrypt Cert you just added)
GTS Root R3 (preinstalled Google R3, not at all related)
Both Safari and Chrome have problems as they both rely on the MacOS Keychain for Root and Intermediate Certs. Adding the Let's Encrypt R3 Cert will fix both. However, I found I needed to restart Chrome for the Cert lookup to work.
If there is a better way to do this, please by all means contribute.
@bill.gertz Thanks for the detailed answer, however previously i was creating certs from terraform vancluever/acme module with preferred_chain as "ISRG Root X1" and it wasn't working but now I moved to cert-manager and let it provision certs for me, and it worked fine, I am really not sure what I was doing wrong... but that seems to be a simple solution for anyone else who face this issue
I got this problem too. Worse, in some web browsers it would work and in some others it would not.
My context: Linux, nginx server, getssl as the let's encrypt client
Thanks to this thread, I was able to understand what was going on. Getssl installs only the certificate in nginx, not the full chain. So depending on my luck, the web browser still had the expired R3 certificate or not:
The solution turned out to be easy: installing the full chain certificate in nginx for each web site.
in the domain's getssl.cfg config file, make sure that you have PREFERRED_CHAIN="ISRG Root X1" and FULL_CHAIN_INCLUDE_ROOT="true"
regenerate your certificate using the force mode: getssl -f yourdomain.com
copy the fullchain.crt file in you nginx ssl folder, renaming it at the same time: cp -a fullchain.crt /etc/nginx/your_ssl_folder/yourdomain.com.crt
reload nginx
Et voilà ! The root CA is now "ISG Root X1" and the R3 intermediate certificate is valid (at least until 2025).
I hope this can help someone,
Thank to everyone who wrote in this thread, you saved my day! I was running in circles.
Cheers,
Yann
Good point, if someone was trying to use certbot to get a cert for windows they probably then convert to PFX and expect the chain to be preserved, which it wont be.
@neik if you are indeed on Windows, see some general useful info here: Let's Encrypt DST Root CA X3 expiry Sept 30th 2021 | Certify The Web Docs but the first thing to do is reboot if you can, or recreate your https binding if you can't reboot. Windows can be quite determined to hang onto the old R3 chain. If you scan your site with SSL Checker you want the R3 intermediate to be the unexpired one issued by ISRG Root X1.
Nope, I'm on Ubuntu 20.04 LTS and using Emby which needs the pem files being converted to a pfx.
Interestingly if I put nginx in front and use the pem files (fullchain + privkey) then it is working just fine.
I'm not the only one having this, on the Emby forums one user brought up it might be a Emby specific problem.
I think for the time being I will simpy activate nginx again and see how this goes.
Check your Apache config.
Highly likely that is isn't doing exactly what you want it to.
I'd start with the output of: sudo apachectl -t -D DUMP_VHOSTS
Thank you for the link to this good article!
I e-mail the link to the Apple product security team, as they sure have a lot to learn from it. Building the certificate trust chain on macOS 11.6 is obviously broken.
Forcing renewal is NOT necessary for chain issues. The certificate and the signing intermediate would be IDENTICAL. Please don't force renewal for this kind of issues.
How would you resolve this issue then? My website is sending faulty certificate chains (with the expired DST Root CA) to some users and the correct one (with ISRG Root) to others.