Curl gives certificate expired on Ubuntu 20.04 LTS

Solved

Thank you Profile - Nummer378 - Let's Encrypt Community Support.
I would also like to thank rg305 and Osiris

After a long journey I found the file that caused the problem. Just as Nummer378 pointed out I had misstakenly added the intermediate certificate to the trust store indirect and manually.

Below I have written down the steps I used to track it down. It could probably be done in a more effective way, but I'm learning. For example the man page for update-ca-certificates pointed out that files in the the directory /usr/localshare/ca-certificates was included in to ca-certificates.crt and a file link created to those files.

So if you don't need to know the steps I took you can stop reading now.

I repeated the following:

cd /etc/ssl
rsync -av certs/  certs_bak
rm -rf certs
mkdir certs
nano /etc/ce-certificates.conf  # Removed the line !mozilla/DST_Root_CA_X3.crt 
# Checked that the file /usr/share/ca-certificates/mozilla/DST_Root_CA_X3.crt didn't exist
update-ce-certificates
curl  -v https://acme-v02.api.letsencrypt.org/directory
....
curl: (60) SSL certificate problem: certificate has expired
...
openssl s_client -connect acme-v02.api.letsencrypt.org:443 -showcerts
CONNECTED(00000003)
depth=1 C = US, O = Let's Encrypt, CN = R3
verify error:num=2:unable to get issuer certificate
issuer= O = Digital Signature Trust Co., CN = DST Root CA X3
verify return:1
...

Same false result even after reboot. I also rechecked that openssl used the /etc/ssl/certs/ca-certificates.crt et.c. As I didn't have a copy of the expired DST root CA X3 certificate (it probably could be extracted by openssl), I thought maybe it was stored under a different filename.

awk -v cmd='openssl x509 -noout -subject' '
    /BEGIN/{close(cmd)};{print | cmd}' < /etc/ssl/certs/ca-certificates.crt  | grep DST

Nothing.

Then I executed the above without the grep and was lucky the last row was Letsencrypt ...X3. I did some mangling of the ca-certificates.crt, e.g. backuped the file, extracted the last certificate into a separate file, 'false.crt'. I also excluded the last cert from ca-certificates.crt.
Re run the curl and openssl commands, still false.

I then used grep to identify if there was another file in /etc/ssl/certs that contained the content of the false.crt.

cd /etc/ssl/certs
grep -R -F -f false.crt .

And it was, I found a link to another file and noted the filename. I removed the link and re executed the curl and openssl commands and it worked.

I tracked down the file and I had created the file in the begining of the year as my CA and ii was included by the update-ca-certificates command but it was not in the /etc/ca-certificates.conf file.

4 Likes