R3 Intermediate certificate has expired

Thx a million!!!!!

Everything seems to be up to speed now :slight_smile:

Hi,

We had our certs generated from vancluever/acme and running on nginx using Nginx Ingress Controller on Kubernetes and it failed today, after DST -> R3 cert expired, and now I have recreated the cert to use ISRG Root X1 as preferred chain and applied it, the issue is on chrome, website is still showing cert failed, if i see the chain it is DST Root -> R3 > cert, but if I open in Firefox, it gives ISRG -> R3 -> cert which works fine, also I tried the cert on openssl,

openssl s_server -accept 8080 -www -cert tls.crt -key tls.key -CAfile ca.crt
it shows correct chain, what am i missing?
its not working on Chrome for most of the users but working fine for some?

I am also having issues with the change that was made today.
In my case I seem to somehow lose the parent "DST Root CA X3" in the certificates path when I convert it from pem to pfx.

Command I'm using:

openssl pkcs12 -export -out certificate.pfx -inkey privkey.pem -in cert.pem -certfile chain.pem

Already tried fullchain.pem instead of chain.pem but result unfortunately is the same.

Any hint is highly appreciated!

@kahootali

I had the same problem under MacOS Big Sur 11.6. Found that the Intermediate CA Let's Encrypt R3 certificate (Cert) was missing from the Keychain. Using Firefox, which uses its own Cert store, I was able to download the R3 Cert .pem file. Added the R3 Cert to the MacOS Keychain by simply double-clicking on the file.

If you trust me to give you the right PEM file I've uploaded it here.Lets-Encrypt-R3.pem (1.8 KB)

Although I suggest you really do this by chasing down the Cert yourself. Brief instructions:

  1. Using Firefox go to the site having the issue (does not work in Chrome or Safari)

  2. Click on the Padlock in the Address Bar

  3. Click on Connection Secure

  4. Click on More Information

  5. Click on the View Certificate Button

  6. Under the Certificate section click on the R3 Tab

  7. Scroll down to the Miscellaneous Section

  8. From the Download Item click on PEM (cert)

  9. Allow the download - note it will name the file with the FQDN of the site visited in step 1.

  10. The file is really the Let's Encrypt R3 Cert in PEM format - simply double click the file

  11. Verify that the Cert has been added to the Keychain - find them by entering R3 in the Keychain seach-bar at the upper right

  12. There should be two:

    • R3 (this is the Let's Encrypt Cert you just added)
    • GTS Root R3 (preinstalled Google R3, not at all related)

Both Safari and Chrome have problems as they both rely on the MacOS Keychain for Root and Intermediate Certs. Adding the Let's Encrypt R3 Cert will fix both. However, I found I needed to restart Chrome for the Cert lookup to work.

If there is a better way to do this, please by all means contribute.

Good luck!

1 Like

@bill.gertz Thanks for the detailed answer, however previously i was creating certs from terraform vancluever/acme module with preferred_chain as "ISRG Root X1" and it wasn't working but now I moved to cert-manager and let it provision certs for me, and it worked fine, I am really not sure what I was doing wrong... but that seems to be a simple solution for anyone else who face this issue

1 Like

Hi there,

I got this problem too. Worse, in some web browsers it would work and in some others it would not.
My context: Linux, nginx server, getssl as the let's encrypt client

Thanks to this thread, I was able to understand what was going on. Getssl installs only the certificate in nginx, not the full chain. So depending on my luck, the web browser still had the expired R3 certificate or not:
R3 expired
The solution turned out to be easy: installing the full chain certificate in nginx for each web site.

  • in the domain's getssl.cfg config file, make sure that you have PREFERRED_CHAIN="ISRG Root X1" and FULL_CHAIN_INCLUDE_ROOT="true"
  • regenerate your certificate using the force mode: getssl -f yourdomain.com
  • copy the fullchain.crt file in you nginx ssl folder, renaming it at the same time: cp -a fullchain.crt /etc/nginx/your_ssl_folder/yourdomain.com.crt
  • reload nginx

Et voilà ! The root CA is now "ISG Root X1" and the R3 intermediate certificate is valid (at least until 2025).

I hope this can help someone,
Thank to everyone who wrote in this thread, you saved my day! I was running in circles.
Cheers,
Yann

3 Likes

Are you using Windows/IIS?

2 Likes

Good point, if someone was trying to use certbot to get a cert for windows they probably then convert to PFX and expect the chain to be preserved, which it wont be.

@neik if you are indeed on Windows, see some general useful info here: Let's Encrypt DST Root CA X3 expiry Sept 30th 2021 | Certify The Web Docs but the first thing to do is reboot if you can, or recreate your https binding if you can't reboot. Windows can be quite determined to hang onto the old R3 chain. If you scan your site with SSL Checker you want the R3 intermediate to be the unexpired one issued by ISRG Root X1.

Hi @phungld, welcome to the LE community forum :slight_smile:

You need to change the web service to serve the correct intermediate chain.
This is what is being served now:

---
Certificate chain
 0 s:/CN=itourlink.com
   i:/C=US/O=Let's Encrypt/CN=R3
 1 s:/C=US/O=Let's Encrypt/CN=R3
   i:/C=US/O=Internet Security Research Group/CN=ISRG Root X1
---

This is the EXAMPLE to follow:

---
Certificate chain
 0 s:/CN=community.letsencrypt.org
   i:/C=US/O=Let's Encrypt/CN=R3
 1 s:/C=US/O=Let's Encrypt/CN=R3
   i:/C=US/O=Internet Security Research Group/CN=ISRG Root X1
 2 s:/C=US/O=Internet Security Research Group/CN=ISRG Root X1
   i:/O=Digital Signature Trust Co./CN=DST Root CA X3
---

Which ACME client are you using?

Note: Windows/IIS is a bit tricky and may require manually removing certs from the certificate store.

1 Like

Nope, I'm on Ubuntu 20.04 LTS and using Emby which needs the pem files being converted to a pfx.
Interestingly if I put nginx in front and use the pem files (fullchain + privkey) then it is working just fine.
I'm not the only one having this, on the Emby forums one user brought up it might be a Emby specific problem.

I think for the time being I will simpy activate nginx again and see how this goes.

I'm running apache httpd and i checked there are fullchain.pem in use.

        SSLCertificateFile      /etc/letsencrypt/live/myhost.com/fullchain.pem
        SSLCertificateKeyFile   /etc/letsencrypt/live/myhost.com/privkey.pem

However my chain looks like

openssl s_client -connect myhost.com:443 -servername myhost.com
CONNECTED(00000003)
depth=0 CN = myhost.com
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 CN = myhost.com
verify error:num=21:unable to verify the first certificate
verify return:1
---
Certificate chain
 0 s:/CN=myhost.com
   i:/C=US/O=Let's Encrypt/CN=R3
---

Last certbot renewal is made 5th of september.
Any ideas what should I check next?

What is the version of the apache web server?

2 Likes

Check your Apache config.
Highly likely that is isn't doing exactly what you want it to.
I'd start with the output of:
sudo apachectl -t -D DUMP_VHOSTS

1 Like
Server version: Apache/2.4.6 (CentOS)
Server built:   Nov 16 2020 16:18:20

When did you last restart Apache and are there multiple certificate entries in the fullchain.pem text file?

On that version (and earlier) you have to use the SSLCertificateChainFile statement. The SSLCertificateFile accepts only one certificate.

3 Likes

That is correct, linking a thread with more info about the topic:

2 Likes

Thank you for the link to this good article!
I e-mail the link to the Apple product security team, as they sure have a lot to learn from it. Building the certificate trust chain on macOS 11.6 is obviously broken.

I am encountering the same issue. How did you force a renewal?

Forcing renewal is NOT necessary for chain issues. The certificate and the signing intermediate would be IDENTICAL. Please don't force renewal for this kind of issues.

1 Like