Can't verify certificates issued by letsencrypt

Help
1

I have a site that uses letsencrypt certificates and this morning I started getting certificate verify failed (unable to get local issuer certificate) errors when making calls to it from an Ubuntu 18.04 server. There doesn't appear to be anything wrong with the certificate, as I'm not having problems with other devices connecting, and the Ubuntu 18.04 server is now having problems connecting to multiple letsencrypt secured sites, including letsencrypt.org. I feel like this has something to do with the upcoming root certificate expiration, but I'm not getting anywhere with my troubleshooting. Things were working as recently as Thursday, but as of this morning they are not.

For example, I went to download the SRG Root X1 PEM file from letsencrypt.org thinking maybe I don't have the right certificates or something, but with wget it fails, not being able to verify.

ERROR: cannot verify letsencrypt.org's certificate, issued by ‘CN=R3,O=Let's Encrypt,C=US’:
Unable to locally verify the issuer's authority.

My server time is correct, and I have no available server updates. I appreciate any ideas. Thanks!

2

Hi @andyrue, welcome to the LE community forum :slight_smile:

I would first try:
sudo apt-get update
sudo apt-get upgrade

[as there have been some recent updates to Ubuntu regarding the expiring LE cert]

1 Like
3

Thank you, I did have updates available this morning, but I have since installed everything and it hasn't seemed to help.

2 Likes
4

OK, can you give an example request that throws an error?

Also, try:
update-ca-certificates

1 Like
5

wget https://letsencrypt.org/certs/isrgrootx1.pem results in:

ERROR: cannot verify letsencrypt.org's certificate, issued by ‘CN=R3,O=Let's Encrypt,C=US’:
  Unable to locally verify the issuer's authority.
To connect to letsencrypt.org insecurely, use `--no-check-certificate'.
6

I can do that just fine on:
Ubuntu 18.04.5 LTS

7

There were no updates available for that. ca-certificates is already the newest version (20210119~18.04.2)

8

Are you sure there are no pending updates?
sudo apt update
sudo apt-get update
[yes try both]

sudo apt list --upgradable

9

There are 4 packages being held back, but that appears to be all. I'm on 18.04.6 LTS

libnss-systemd/bionic-updates 237-3ubuntu10.52 amd64 [upgradable from: 237-3ubuntu10.29]
libpam-systemd/bionic-updates 237-3ubuntu10.52 amd64 [upgradable from: 237-3ubuntu10.29]
libsystemd0/bionic-updates 237-3ubuntu10.52 amd64 [upgradable from: 237-3ubuntu10.29]
systemd/bionic-updates 237-3ubuntu10.52 amd64 [upgradable from: 237-3ubuntu10.29]
10

I doubt they will fix this issue...
Please show:
ls -l /etc/ssl/certs/ca-certificates.crt

I show:
-rw-r--r-- 1 root root 199113 Sep 27 15:32 /etc/ssl/certs/ca-certificates.crt
[size 199113]

11

-rw-r--r-- 1 root root 131535 Sep 27 11:33 /etc/ssl/certs/ca-certificates.crt

Definitely a size difference.

12

Reinstall it.

apt install ca-certificates

1 Like
13

I actually did try that already. :smiley: I'll do it again.

1 Like
14

Then uninstall and reinstall it.

15

Right, that's what I mean. I just did it again. (apt remove and apt install) When finished it says 20210119~18.04.2

If you apt install ca-certificates, does it list a different version for you? 20210119~18.04.2 seems kinda old.

16

I show:

apt install ca-certificates
Reading package lists... Done
Building dependency tree
Reading state information... Done
ca-certificates is already the newest version (20210119~18.04.2).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
17

Again now:

18

-rw-r--r-- 1 root root 131535 Sep 27 11:45 /etc/ssl/certs/ca-certificates.crt

Same

19

That is weird!

20

Let's compare:

apt update
Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease
Hit:3 http://ppa.launchpad.net/certbot/certbot/ubuntu bionic InRelease
Get:4 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]
Get:5 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB]
Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main Sources [517 kB]
Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [2,237 kB]
Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,749 kB]
Fetched 4,755 kB in 5s (875 kB/s)
Reading package lists... Done
Building dependency tree
Reading state information... Done
All packages are up to date.

apt-get update
Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease
Hit:3 http://ppa.launchpad.net/certbot/certbot/ubuntu bionic InRelease
Hit:4 http://archive.ubuntu.com/ubuntu bionic-updates InRelease
Get:5 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB]
Fetched 163 kB in 1s (112 kB/s)
Reading package lists... Done