No time is ever wasted, when it's spent with friends 
Cheers from Miami 
#FreeCUBA 
3 Likes
Thank you so much. This post saved my bacon this morning!
(beer & pizza for you)
2 Likes
I'll take the 
but hold the 
[too many calories for me - LOL]
2 Likes
By the way, the chain.pem should be updated by Certbot with the new R3 from Let's Encrypt (this should have happened a long time ago). I suspect that cases where it didn't appear to happen will usually involve people manually copying it to a different location on their systems, so that the copied version never gets updated when the original version is changed.
To check on this, people who are interested can run
openssl x509 -noout -dates -in /etc/letsencrypt/live/example.com/chain.pem
replacing example.com with your own domain name.
If you get a notAfter date which is before the present (or other than 2025), you may have found a bug in Certbot and it would be helpful to report it. 
I also think some people on the thread maybe were not serving a chain at all and so some clients had cached the old R3 and were using it automatically to build the trust path, which will no longer work with the expiry. In that case, updating the configuration to serve a chain (one effect of @rg305's solution!) will fix the problem because the clients will be told about the new R3. (It is possible to have a technically invalid configuration that works for a while with most clients because of the way that clients remember old intermediate certificates that they've seen before, which could be a common case for people experiencing this problem!)
2 Likes
it'd be radical idea but would bring up r4 would help this? nobody would cached r4 by dst cert
2 Likes
Yes, that's quite a clever thought—switching intermediates (and corresponding signing keys) would definitely help with that. On the other hand, I'm confident that the Let's Encrypt team wouldn't consider it anywhere close to worth the tradeoffs in terms of extra work for them and no longer having a backup for issuance in case of catastrophes. All of the issues caused by the expiration have been known and documented for a long time, and the workarounds are also pretty well-documented.
R4 is designed to be used for emergencies, and this isn't an emergency.
4 Likes
If you want to see a pretty technical description of this by someone who works with this all the time and is very frustrated by it, check out
One great insight there is that there isn't just one chain that will allow a particular certificate to be accepted by clients; there can be multiple different chains available, each of which can be independently valid (or not). Some problems can be caused by server configuration (like not serving a chain at all, or serving a chain that is expired or mismatched to the certificate), while other problems can be due to clients successfully building a chain that doesn't verify successfully (even though they could have built a different chain that does!).
5 Likes
Using fullchain.pem also fixed my IOS client problems with dovecot server but I don't understand why.
Hello. I have wildcard ssl from letsencrypt. Starting from 09/29/2021 all mobile browsers show me (connection to this site not secure) issuer DST root CA X3 (expired)
but with web browser everything is fine ISRG Root X1. My site obfog.com. Can anyone help me? Thank you.
Because, left to their own devices, things can make mistakes.
Using fullchain.pem removes the ambiguity (for those systems/software that will follow that lead).
2 Likes
Update the chain being served:
---
Certificate chain
0 s:/CN=obfog.com
i:/C=US/O=Let's Encrypt/CN=R3
1 s:/C=US/O=Let's Encrypt/CN=R3
i:/O=Digital Signature Trust Co./CN=DST Root CA X3
---
Should look like the EXAMPLE:
---
Certificate chain
0 s:/CN=community.letsencrypt.org
i:/C=US/O=Let's Encrypt/CN=R3
1 s:/C=US/O=Let's Encrypt/CN=R3
i:/C=US/O=Internet Security Research Group/CN=ISRG Root X1
2 s:/C=US/O=Internet Security Research Group/CN=ISRG Root X1
i:/O=Digital Signature Trust Co./CN=DST Root CA X3
---
For help with that, what ACME client and version are you using?
1 Like
Hi. Thanks for quick reply. It looks like I am using acme.sh v02. I'm not good at this. What do I need to do to change the chain?
You need to update the nginx config that is using the wrong file.
Let's have a look at:
sudo nginx -T | grep -Ei 'server_name|ssl_cert'
1 Like
OK, thanks you very much. I will write to the hosting provider support, since I cannot change the entry by my self.
1 Like
Same problem here. I'm googling trying to find where I change the certificate chain when using haproxy as a https front end offloader.
I'm using the acme client in OPNsense and letting haproxy use this as a https frontend.
Any ideas or pointers welcome. If I find it I'll post here
I have /etc/ssl/openssl.cnf for example but have no idea if that is used in this context
# ls -la /etc/ssl/
total 756
drwxr-xr-x 2 root wheel 512 Sep 30 07:58 .
drwxr-xr-x 27 root wheel 2048 Sep 30 09:08 ..
-rw-r--r-- 1 root wheel 691762 Sep 30 07:58 cert.pem
-rw-r--r-- 1 root wheel 10921 Sep 2 10:21 openssl.cnf
I have a the same problem, crt.sh | 5311774237.
this problem should be fixed from my computer or on server.
Thanks
Remove old expired X3 certs under System -> Cert Manager and renew your certficate.
1 Like
Could it be possible the problem with the root certificate actually gives me problems also renewing with acme? The acme has worked for years but not today. This is in the curl log
Can not init api for: https://acme-v02.api.letsencrypt.org/directory.
== Info: Trying 172.65.32.248:443...
== Info: Connected to acme-v02.api.letsencrypt.org (172.65.32.248) port 443 (#0)
== Info: ALPN, offering h2
== Info: ALPN, offering http/1.1
== Info: successfully set certificate verify locations:
== Info: CAfile: /usr/local/etc/ssl/cert.pem
== Info: CApath: none
== Info: (304) (OUT), TLS handshake, Client hello (1):
=> Send SSL data, 333 bytes (0x14d)
data stuff....
== Info: SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
== Info: ALPN, server accepted to use h2
== Info: Server certificate:
== Info: subject: CN=acme-v02.api.letsencrypt.org
== Info: start date: Sep 30 00:34:24 2021 GMT
== Info: expire date: Dec 29 00:34:23 2021 GMT
== Info: subjectAltName: host "acme-v02.api.letsencrypt.org" matched cert's "acme-v02.api.letsencrypt.org"
== Info: issuer: C=US; O=Let's Encrypt; CN=R3
== Info: SSL certificate verify result: certificate has expired (10)
== Info: Closing connection 0
== Info: TLSv1.2 (OUT), TLS alert, close notify (256):
Hahahahaha, ooohh nooooooo
-# curl https://acme-v02.api.letsencrypt.org/directory
curl: (60) SSL certificate verify result: certificate has expired (10)
More details here: https://curl.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
This is on a fully updated OPNsense box. What now 
1 Like
Thx a million!!!!!
Everything seems to be up to speed now 