ERROR: cannot verify x's certificate, issued by ‘CN=R3,O=Let's Encrypt,C=US’:

Help
1

I have a git server running with a valid let's encrypt certificate.

Now I created an Ubuntu server:

Distributor ID: Ubuntu
Description: Ubuntu 22.04.3 LTS
Release: 22.04
Codename: jammy

When trying to connect to my server having the Let's Encrypt certificate using

wget -p https://git.dev.dynamic-biosensors.com

I receive the following message

--2023-08-22 19:40:40--  https://git.dev.dynamic-biosensors.com/
Resolving git.dev.mydomain.com (git.dev.dynamic-biosensors.com)... 192.168.70.14
Connecting to git.dev.mydomain.com (git.dev.dynamic-biosensors.com)|192.168.70.14|:443... connected.
ERROR: cannot verify git.dev.dynamic-biosensors.com's certificate, issued by ‘CN=R3,O=Let's Encrypt,C=US’:
  Unable to locally verify the issuer's authority.
To connect to git.dev.dynamic-biosensors.com insecurely, use `--no-check-certificate'.

I already ran

sudo apt-get install --reinstall ca-certificates

dpkg-reconfigure ca-certificates

update-ca-certificates

and all came back without error, but the issue persists.

The certificate was issued Monday, 14. August 2023 at 21:25:48 Central European Summer Time and the new Ubuntu server's timedatectl is

Local time: Tue 2023-08-22 19:47:49 UTC
Universal time: Tue 2023-08-22 19:47:49 UTC
RTC time: Tue 2023-08-22 19:47:49
Time zone: Etc/UTC (UTC, +0000)
System clock synchronized: yes
NTP service: active
RTC in local TZ: no

Any ideas what is going on? The connection should just work, should it not?

2

We won't be able to help you without being able to test the actual certificate. What's the actual FQDN?

6 Likes
3

Almost certainly the problem is that you're not serving a certificate chain. A webserver needs the certificate, as well as an intermediate or two which "chain" back to the long-lived roots in the ca-certificates package.

What is the webserver (eg, nginx), and how is the certificate configured in it? Unfortunately there is some inconsistency in how this is set up.

A typical source of this error is giving software the cert.pem file, when it needs fullchain.pem -- to use the filenames certbot produces by default. Some other software takes the cert.pem and chain.pem seperately. (fullchain.pem is both of them combined)

5 Likes
4

You can get some (very) verbose debugging help with this command:

echo | openssl s_client -showcerts -connect helloworld.letsencrypt.org:443
(put your domain name in)

You should see a message that says Certificate Chain followed by multiple certificates.

It should look something like this:

---
Certificate chain
 0 s:CN = helloworld.letsencrypt.org
   i:C = US, O = Let's Encrypt, CN = R3
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Aug  4 21:44:10 2023 GMT; NotAfter: Nov  2 21:44:09 2023 GMT
-----BEGIN CERTIFICATE-----
MIIFATCCA+mgAwIBAgISA68n+eszOwhe183DMJSiutIvMA0GCSqGSIb3DQEBCwUA
........
gD/L98SVuvuFIySmXapy8obqndEzUEJPBJjNGNvKRCNdd4Qeng==
-----END CERTIFICATE-----
 1 s:C = US, O = Let's Encrypt, CN = R3
   i:C = US, O = Internet Security Research Group, CN = ISRG Root X1
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Sep  4 00:00:00 2020 GMT; NotAfter: Sep 15 16:00:00 2025 GMT
-----BEGIN CERTIFICATE-----
MIIFFjCCAv6gAwIBAgIRAJErCErPDBinU/bWLiWnX1owDQYJKoZIhvcNAQELBQAw

If you only see a single certificate, ie, you don't see a line like i:C = US, O = Internet Security Research Group, CN = ISRG Root X1, then you've got the problem I just described and aren't serving a complete chain.

This is probably the most common TLS misconfiguration, and unfortunately it's largely due to inconsistencies between how different webservers are configured. Web browsers like Chrome and Firefox have workarounds for this issue because it's so common, so your website may work in them still.

6 Likes
5

You ran the command but you didn't say exactly what you did, did you make sure that X1 was actually checked?

image
image1289×237 12.1 KB

(show some love for X2 as well even though it's not relevant to your current certificate)

1 Like
6

Whatever server is running https://git.dev.mydomain.com is probably pointing to the certificate file instead of the full chain file, so the intermediate R3 certificate is not being served as part of the chain.

So, it's likely not wget that's the problem, it's the web server you're connecting to.

1 Like
7

I adjusted the original request to show the fully qualified domain name (FQDN).

8

The domain name git.dev.dynamic-biosensors.com is unresolvable. Is it a real domain name, or there is a typo?

2 Likes
9

I ran the command you suggested:

echo | openssl s_client -showcerts -connect git.dev.dynamic-biosensors.com:443

and received the following output:

CONNECTED(00000003)
depth=0 CN = git.dev.dynamic-biosensors.com
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 CN = git.dev.dynamic-biosensors.com
verify error:num=21:unable to verify the first certificate
verify return:1
depth=0 CN = git.dev.dynamic-biosensors.com
verify return:1
---
Certificate chain
 0 s:CN = git.dev.dynamic-biosensors.com
   i:C = US, O = Let's Encrypt, CN = R3
   a:PKEY: id-ecPublicKey, 256 (bit); sigalg: RSA-SHA256
   v:NotBefore: Aug 14 18:25:48 2023 GMT; NotAfter: Nov 12 18:25:47 2023 GMT
-----BEGIN CERTIFICATE-----
MIIEQDCCAyigAwIBAgISA5FM1lSBazHYOm2PCFfycWlNMA0GCSqGSIb3DQEBCwUA
MDIxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQD
EwJSMzAeFw0yMzA4MTQxODI1NDhaFw0yMzExMTIxODI1NDdaMCkxJzAlBgNVBAMT
HmdpdC5kZXYuZHluYW1pYy1iaW9zZW5zb3JzLmNvbTBZMBMGByqGSM49AgEGCCqG
SM49AwEHA0IABP/O8SciVLBYKrNq6yH5z70J40cw0IiLJxLpJ+RXYULhn1UtATY2
p7QDHTu3z3kU08NJdlC1qW9iaDZQKld/oM2jggIiMIICHjAOBgNVHQ8BAf8EBAMC
B4AwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMAwGA1UdEwEB/wQCMAAw
HQYDVR0OBBYEFIM32Y/Uah1K6+E8AUpdOV4PvBNaMB8GA1UdIwQYMBaAFBQusxe3
WFbLrlAJQOYfr52LFMLGMFUGCCsGAQUFBwEBBEkwRzAhBggrBgEFBQcwAYYVaHR0
cDovL3IzLm8ubGVuY3Iub3JnMCIGCCsGAQUFBzAChhZodHRwOi8vcjMuaS5sZW5j
ci5vcmcvMCkGA1UdEQQiMCCCHmdpdC5kZXYuZHluYW1pYy1iaW9zZW5zb3JzLmNv
bTATBgNVHSAEDDAKMAgGBmeBDAECATCCAQYGCisGAQQB1nkCBAIEgfcEgfQA8gB3
ALc++yTfnE26dfI5xbpY9Gxd/ELPep81xJ4dCYEl7bSZAAABifWC+hkAAAQDAEgw
RgIhAKqgjXIRsmv11kVvjWkoAqSt4wqfL11IAWlSLhdbwqp3AiEAlDK9vwHlGN39
4WeoJEvg3q+sLrLHFwopl4Scq2GexUYAdwDoPtDaPvUGNTLnVyi8iWvJA9PL0RFr
7Otp4Xd9bQa9bgAAAYn1gvnTAAAEAwBIMEYCIQCSrN3BoRTzTKlOA5vjBAR+gsOh
auFKxsfeLA332De6HAIhAIL5RgUsWltkUdopG0fBnHbcj8ENr3sf7ySPIUDuKcIH
MA0GCSqGSIb3DQEBCwUAA4IBAQAPmgqos/vc3g0QBq6+cxbkLLGNatbDNdmvAa1l
cg6sQvhfWfRh77rHuZ/eHgddjQ/n9k5jGSH7JACAdSYIQpud89rmjtL6VU893h2T
SRNhqGxyUcjIBuMe7SyvOKzBvEo7SU5MwF4Mm+GrxqQyEW1+9g6SKYJ3hrA2yhgf
Jk90zrVhsiu5lu98i1LfWGOSfrELC/AVIBR1rBoFMymoqIcRvazOYSO/I0fH/HOM
aEmZHxq5i43CL8c25Qrd43sYdxEDVZUGErWgiYFnIRULSvZCuGVk3XI6PQFT/xpX
5tlXeB/eNW67zdw3P1ZPhTWqPZB0vhJNtjbERZ209cq/ReT1
-----END CERTIFICATE-----
---
Server certificate
subject=CN = git.dev.dynamic-biosensors.com
issuer=C = US, O = Let's Encrypt, CN = R3
---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: ECDSA
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 1495 bytes and written 425 bytes
Verification error: unable to verify the first certificate
---
New, TLSv1.2, Cipher is ECDHE-ECDSA-AES128-GCM-SHA256
Server public key is 256 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-ECDSA-AES128-GCM-SHA256
    Session-ID: E6EF2F2A023A6F0DC91293A748444A0F520B8500C45C1F91AC53838A6706306C
    Session-ID-ctx: 
    Master-Key: F159177F70E2A809D5A7C4F724FA03E196B5430BD2A94F72E72B5FF6716040C7FAE3FF84A68712291FA4125450B99EA5
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket:
    0000 - 73 ec 3c e4 83 81 19 f1-61 a8 60 0a 74 42 15 1f   s.<.....a.`.tB..
    0010 - 97 e3 3a 36 31 5d 7d 56-cc 6e 04 6b 57 8c 7e ba   ..:61]}V.n.kW.~.
    0020 - 4e ba ed 15 1c 45 05 2f-64 e8 0a 80 e1 5c 28 80   N....E./d....\(.
    0030 - b7 f6 07 44 33 f5 af db-e0 f9 e4 9d ad d4 87 a6   ...D3...........
    0040 - 4a 80 d3 53 1c 06 a5 57-32 ae d5 a7 58 9f 49 69   J..S...W2...X.Ii
    0050 - 25 c2 ce 2b aa d8 56 70-2c fb c0 7c 53 d6 d3 37   %..+..Vp,..|S..7
    0060 - e9 e2 b3 fb db 96 e2 0a-cf a0 8b 3f c1 7c 71 c4   ...........?.|q.
    0070 - 7f b3 22 3a 67 cd 25 f9-0e 90 4b 19 08 0d 0e b3   ..":g.%...K.....
    0080 - cf                                                .

    Start Time: 1692770511
    Timeout   : 7200 (sec)
    Verify return code: 21 (unable to verify the first certificate)
    Extended master secret: no
---
DONE
10

It is an internal server, not reachable from the internet

11

This shows an incomplete chain: only the leaf certificate is served by the webserver. It should also be sending at least one intermediate.

3 Likes
12

Yes, it is selected

image

13

You need to modify the web servers' config so that it serves the left cert and an intermediate.
If using certbot, that means:
Don't use "cert.pem"
Use "fullchain.pem"

5 Likes
14

as @mcpherrinm and @rg305 suggested, it was the incomplete chain.

using the fullchain.pem instead of the cert.pem resolved the issue.

Thank you guys so much for your help.

This is really an amazing community!!

6 Likes
15

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.