Curl says: unknown CA

Help
1

My domain is: www.phcomp.co.uk

I ran this command on the server & Debian client (Debian 12 & Rocky Linux 8.10):
curl -v https://les-test-b.phcomp.co.uk/index.html

It produced this output:

  • Trying 46.43.0.112:443...
  • Connected to les-test-b.phcomp.co.uk (46.43.0.112) port 443 (#0)
  • ALPN: offers h2,http/1.1
  • TLSv1.3 (OUT), TLS handshake, Client hello (1):
  • CAfile: /etc/ssl/certs/ca-certificates.crt
  • CApath: /etc/ssl/certs
  • TLSv1.3 (IN), TLS handshake, Server hello (2):
  • TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
  • TLSv1.3 (IN), TLS handshake, Certificate (11):
  • TLSv1.3 (OUT), TLS alert, unknown CA (560):
  • SSL certificate problem: unable to get local issuer certificate
  • Closing connection 0
    curl: (60) SSL certificate problem: unable to get local issuer certificate
    More details here: curl - SSL CA Certificates

However I am able to view the web site via web browsers firefox and brave

My web server is (include version): Apache

The operating system my web server runs on is (include version): Debian 12 & Rocky Linux 8.10

I can login to a root shell on my machine (yes or no, or I don't know): yes

I'm using a control panel to manage my site : no

The version of my client is: I use acme_tiny.py

This seems to be related to the following which does not come to a conclusion:

If I run curl under strace I see it looking in /etc/ssl/certs/ for 31dfb39d.0 and 8a24e8ea.0 but not finding anything.

So: it seems that the client operating system (ie where I run curl) is missing a file or two. Is this a Debian/RedHat problem or has Let's Encrypt not got these files out ?

Updating my individual client machines will not help as I cannot update the other machines that might try to use my server.

The file returned to me contains 2 certificates. The first with:
CN = les-test-b.phcomp.co.uk
Issuer: C = US, O = Let's Encrypt, CN = R11
The second:
Subject: C = US, O = Let's Encrypt, CN = R11
Issuer: C = US, O = Internet Security Research Group, CN = ISRG Root X1

So my client machine does not have the second issuer certificate (X1).

Creating a certificate with the cross signed certificate (isrgrootx1.pem) does not help as that is self signed (ie Subject = Issuer).

What can I do ?

Thanks

2

The problem is that the server les-test-b.phcomp.co.uk is sending a hardcoded intermediate. It's currently sending the following chain:

# openssl s_client -connect les-test-b.phcomp.co.uk:443
CONNECTED(00000194)
depth=0 CN = les-test-b.phcomp.co.uk
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 CN = les-test-b.phcomp.co.uk
verify error:num=21:unable to verify the first certificate
verify return:1
depth=0 CN = les-test-b.phcomp.co.uk
verify return:1
---
Certificate chain
 0 s:CN = les-test-b.phcomp.co.uk
   i:C = US, O = Let's Encrypt, CN = R11
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Jun 29 10:33:43 2024 GMT; NotAfter: Sep 27 10:33:42 2024 GMT
 1 s:C = US, O = Let's Encrypt, CN = R3
   i:C = US, O = Internet Security Research Group, CN = ISRG Root X1
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Sep  4 00:00:00 2020 GMT; NotAfter: Sep 15 16:00:00 2025 GMT
---

It is sending the intermediate certificate R3 (which is an old intermediate Let's Encrypt retired a few weeks ago), while it should be sending the R11 intermediate.

Note that Let's Encrypt has randomized the issuing intermediate, so hardcoding the intermediate will no longer work. The ACME server provides the proper intermediate certificate on each renewal. You must configure your webserver to serve that intermediate.

How are you currently configuring acme_tiny and what certificate files are you serving via Apache?

3 Likes
3

Drat ... I had left up a test version, this is what it is now:

Certificate chain
 0 s:CN = les-test-b.phcomp.co.uk
   i:C = US, O = Let's Encrypt, CN = R11
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Jun 29 10:33:43 2024 GMT; NotAfter: Sep 27 10:33:42 2024 GMT
 1 s:C = US, O = Let's Encrypt, CN = R3
   i:C = US, O = Internet Security Research Group, CN = ISRG Root X1
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Sep  4 00:00:00 2020 GMT; NotAfter: Sep 15 16:00:00 2025 GMT
4

That's still the exact same issue: You're sending a retired intermediate (R3 vs R11 in this case).

As acme_tiny only hands out a "full chain" file (I believe), this may be caused by you crafting your own certificate chain manually. What commands are you using to issue via acme_tiny? Were you perhaps manually building the "short chain" in the past by adding the R3 intermediate manually?

4 Likes
5

That is interesting ... I see different Subject & Issuers depending on if I use openssl s_client -connect les-test-b.phcomp.co.uk:443 (on another machine) or if I decode the certificates on the machine itself using openssl x509 -inform pem -noout -text.
(The latter command being run twice on the output from Letsencrypt but split into 2 files as the LE output contains 2 certificates).

CONNECTED(00000003)
depth=0 CN = les-test-b.phcomp.co.uk
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 CN = les-test-b.phcomp.co.uk
verify error:num=21:unable to verify the first certificate
verify return:1
depth=0 CN = les-test-b.phcomp.co.uk
verify return:1
---
Certificate chain
 0 s:CN = les-test-b.phcomp.co.uk
   i:C = US, O = Let's Encrypt, CN = R10
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Jun 29 17:16:03 2024 GMT; NotAfter: Sep 27 17:16:02 2024 GMT
 1 s:C = US, O = Let's Encrypt, CN = R3
   i:C = US, O = Internet Security Research Group, CN = ISRG Root X1
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Sep  4 00:00:00 2020 GMT; NotAfter: Sep 15 16:00:00 2025 GMT
---
Server certificate
... chop ....```

```openssl x509 -inform pem -noout -text < first-certificate
        Serial Number:
            03:20:83:a2:60:33:3f:9f:2d:f3:c5:98:aa:d6:7f:4e:61:9c
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = US, O = Let's Encrypt, CN = R10
        Validity
            Not Before: Jun 29 17:16:03 2024 GMT
            Not After : Sep 27 17:16:02 2024 GMT
        Subject: CN = les-test-b.phcomp.co.uk


       Version: 3 (0x2)
        Serial Number:
            4b:a8:52:93:f7:9a:2f:a2:73:06:4b:a8:04:8d:75:d0
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = US, O = Internet Security Research Group, CN = ISRG Root X1
        Validity
            Not Before: Mar 13 00:00:00 2024 GMT
            Not After : Mar 12 23:59:59 2027 GMT
        Subject: C = US, O = Let's Encrypt, CN = R10

I have checked the Apache config ... it is using the correct .crt file - which I have put at https://www.phcomp.co.uk/tmp/2024-06-29-les-test-b.crt

1 Like
6

That cert file looks correct. It has your leaf which was issued by R10 and it has the R10 intermediate (issued by and leads to ISRG Root X1)

Your Apache must have some config problem then because from the public internet you are definitely sending the wrong intermediate (R3 and not R10). (see SSL Checker link here). Note R10 or R11 are used randomly now and others are possible if LE uses backups. So, the intermediate in your chain must be as provided by LE when you requested the cert so that it pairs with your leaf.

Can you show us output of below and we'll look at your Apache

sudo apache2ctl -t -D DUMP_VHOSTS
5 Likes
7

@addw have you restarted Apache since you're renewal? Apache just loads the files your config file points to and it should ideally just be pointing to your "full chain" file and your "private key file". I suspect you probably have an extra line of config that's loading the CA chain as a separate file, which you don't need as your full chain (the .crt file you showed us) already includes everything.

As you can see by opening your crt file as text and decoding with any online pem decoder, you have your actual cert and it has the correct intermediate, so your config in apache should be pointing only to this file and the private key file.

2 Likes
8

It is amazing how different things look when you come back after a couple of days off ...

In the apache config I had SSLCertificateChainFile that gave lets-encrypt-r3.pem
This was needed when R3 was in use. I removed it and it all works properly.

Many thanks all for the help provided, I now just need to update config of other sites that still have SSLCertificateChainFile in there.

1 Like
9

Actually, that was never required and was never the correct way to handle the chain. You were just getting lucky that LE never had to use its backup intermediates.

One of the benefits of Let's Encrypt now using R10 and R11 "randomly" is to expose such problems so people use the cert and chain as intended.

Glad you corrected the problem. Hopefully my explanation helped.

5 Likes
10

Oh, and Apache versions 2.4.8 and later should look something like:

SSLCertificateFile    /etc/letsencrypt/live/example.com/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/example.com/privkey.pem
3 Likes
11

Yes, thank you.

2 Likes
12

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.