Issue description
Couple of days ago, I noticed some issues at my automated curl requests. The issues appear after a renewal of some LE certificates. I was able to trace back the problems to an incomplete validation of my LE TLS certifications chain. A missing ISRG Root YR cert in my local CA trust store or within the provided cert chain causes the problem. This topic was also referenced in a post YE/YR are not in ca-certificates! by 532910.
I opened this new topic, because I think many people could come across this issue, because many(?) major linux distributions do not provide the ISRG Root YE/YR within ins ca-certificates packages. It's also possible to reproduce this situation with the test websites of the ISRG Root YE/YR certificates, provide by Let's Encrypt > Chains of Trust
- ISRG Root YE
[...]
- Test websites: valid, revoked, expired
- ISRG Root YR
[...]
- Test websites: valid, revoked, expired
(see also: The difficulty of making sure your website is broken - Let's Encrypt)
If these test websites are opened with a browser like firefox, everything is fine. A complete TLS chain validation is able.
(I am not complete sure, why e.g. firefox has no issues validating the chain, because ISRG Root YE/YR is neither inside the ff certificate store (or I missed it searching for it). FF may downloads the missing Root certificate on demand... I do not know ).
A incomplete validation behaivor can be seen using curl under different Linux distributions, because even the testing websites do not provide the complete certificate chain:
# openssl s_client -connect valid.yr.test-certs.letsencrypt.org:443 -showcerts </dev/null 2>/dev/null | grep -E "( s| i):"
0 s:
i:C=US, O=Let's Encrypt, CN=YR1
1 s:C=US, O=Let's Encrypt, CN=YR1
i:C=US, O=ISRG, CN=Root YR
Debian 13.5 (trixie)
root@e26f2be3e817:/# cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 13 (trixie)"
[...]
DEBIAN_VERSION_FULL=13.5
[...]
root@e26f2be3e817:/# dpkg -l | grep ca-certificates
ii ca-certificates 20250419 all Common CA certificates
root@e26f2be3e817:/# dpkg -L ca-certificates | grep ISRG
/usr/share/ca-certificates/mozilla/ISRG_Root_X1.crt
/usr/share/ca-certificates/mozilla/ISRG_Root_X2.crt
root@e26f2be3e817:/# dpkg -L ca-certificates | grep YR
root@e26f2be3e817:/# dpkg -L ca-certificates | grep YE
# ^_ YE/YR are not in ca-certificates trust store
Broken (insecure) TLS request:
root@e26f2be3e817:/# curl https://valid.yr.test-certs.letsencrypt.org/
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the webpage mentioned above.
root@e26f2be3e817:/# curl -v https://valid.yr.test-certs.letsencrypt.org/ 2>&1 | grep CAfile
* CAfile: /etc/ssl/certs/ca-certificates.crt
Ubuntu (26.04 LTS)
root@e46e8254f45e:/# cat /etc/os-release
PRETTY_NAME="Ubuntu 26.04 LTS"
[...]
root@e46e8254f45e:/# dpkg -l | grep ca-certificates
ii ca-certificates 20260223 all Common CA certificates
root@e46e8254f45e:/# dpkg -L ca-certificates | grep ISRG
/usr/share/ca-certificates/mozilla/ISRG_Root_X1.crt
/usr/share/ca-certificates/mozilla/ISRG_Root_X2.crt
root@e46e8254f45e:/# dpkg -L ca-certificates | grep YR
root@e46e8254f45e:/# dpkg -L ca-certificates | grep YE
# ^_ YE/YR are not in ca-certificates trust store
root@e46e8254f45e:/# find /usr/share/ca-certificates -iname "*isrg*"
/usr/share/ca-certificates/mozilla/ISRG_Root_X1.crt
/usr/share/ca-certificates/mozilla/ISRG_Root_X2.crt
Broken (insecure) TLS request:
root@e46e8254f45e:/# curl https://valid.yr.test-certs.letsencrypt.org/
curl: (60) SSL certificate OpenSSL verify result: unable to get local issuer certificate (20)
More details here: https://curl.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the webpage mentioned above.
Fedora (44)
[root@8fa12baac27c /]# cat /etc/os-release
[...]
PRETTY_NAME="Fedora Linux 44 (Container Image)"
[...]
[root@8fa12baac27c /]# dnf list installed ca-certificates
Updating and loading repositories:
Repositories loaded.
Installed packages (available for reinstall, available for upgrade)
ca-certificates.noarch 2025.2.80_v9.0.304-7.fc44 9f717d77be324992a27b71f74a042c00
[root@8fa12baac27c /]# dnf repoquery -l ca-certificates | grep ISRG
Updating and loading repositories:
[...]
Repositories loaded.
/etc/pki/ca-trust/extracted/pem/directory-hash/ISRG_Root_X1.pem
/etc/pki/ca-trust/extracted/pem/directory-hash/ISRG_Root_X2.pem
[root@8fa12baac27c /]# dnf repoquery -l ca-certificates | grep YR
Updating and loading repositories:
Repositories loaded.
[root@8fa12baac27c /]# dnf repoquery -l ca-certificates | grep YE
Updating and loading repositories:
Repositories loaded.
# ^_ YE/YR are not in ca-certificates trust store
Broken (insecure) TLS request:
[root@8fa12baac27c /]# curl https://valid.yr.test-certs.letsencrypt.org/
curl: (60) SSL certificate OpenSSL verify result: unable to get local issuer certificate (20)
More details here: https://curl.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the webpage mentioned above.
[root@8fa12baac27c /]# curl -v https://valid.yr.test-certs.letsencrypt.org/ 2>&1 | grep CAfile
* CAfile: /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem
[root@8fa12baac27c /]# grep ISRG /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem
# ISRG Root X1
# ISRG Root X2
Conclusion
I my opinion the ISRG Root YE/YR should be provided by the distribution ca-certificates packages as mentioned in YE/YR are not in ca-certificates! by 532910.
Providing two intermediates (e.g. YR1+Root YR) certificates for the fullchain valiadition is a "solution" until then.
