Verification failing using certbot

Help
1

My domain is:docker.vaultara.com

I ran this command:docker run --rm --name certbot -v "/home/ubuntu/docker_cache/certs:/etc/letsencrypt" -v "/home/ubuntu/docker_cache/letsencrypt:/var/lib/letsencrypt" -p 80:80 certbot/certbot certonly -d docker.vaultara.com

It produced this output:successfully received certificate

My web server is (include version):docker registry v2

The operating system my web server runs on is (include version):Ubuntu 18.04.6

My hosting provider, if applicable, is:AWS

I can login to a root shell on my machine (yes or no, or I don't know):yes

I'm using a control panel to manage my site:no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):(latest container downloaded at run time)

As you can see, I'm using the certbot container in order to obtain a certificate which I'm then using within docker's registry container (so that I can host my own "hub.docker.com"). Everything works perfectly (or so it seems) and I get a certificate, the chain and full chain files.

However, when I attempt to use the certificate I get errors on the client, so after some searching, I decide to verify what's been generated and I see an issue which I'm totally unsure how to overcome.

When I run openssl verify -CAfile chain1.pem cert1.pem I get the following

C = US, O = Internet Security Research Group, CN = ISRG Root X1
error 2 at 2 depth lookup: unable to get issuer certificate
error cert1.pem: verification failed

Inspecting my certificate itself using openssl x509 -noout -text -in cert1.pem shows

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            03:eb:6a:04:83:61:8d:32:b6:df:d7:ee:cb:a2:1a:93:b8:b0
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = US, O = Let's Encrypt, CN = R3
        Validity
            Not Before: Jul 11 14:50:53 2022 GMT
            Not After : Oct  9 14:50:52 2022 GMT
        Subject: CN = docker.vaultara.com
        Subject Public Key Info:
            Public Key Algorithm: id-ecPublicKey
                Public-Key: (384 bit)
                pub:
                    04:b0:0e:c2:ee:98:6e:73:2a:a0:fd:24:f2:72:3d:
                    a9:5c:a8:f3:8c:c0:41:61:af:cb:a0:87:4b:26:5e:
                    f3:1a:72:d3:8b:17:be:7c:ce:ee:6d:bf:98:ed:20:
                    f6:3e:e2:e5:36:02:d8:0d:3e:79:ec:b2:3f:80:45:
                    a6:ba:1e:2b:85:8c:53:e9:17:47:9c:42:0c:d7:32:
                    a2:7d:f5:a4:14:61:48:c8:9d:c0:51:da:2c:95:ca:
                    78:47:f6:d4:dd:bf:2a
                ASN1 OID: secp384r1
                NIST CURVE: P-384
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature
            X509v3 Extended Key Usage: 
                TLS Web Server Authentication, TLS Web Client Authentication
            X509v3 Basic Constraints: critical
                CA:FALSE
            X509v3 Subject Key Identifier: 
                35:F9:9A:EA:7E:FD:CA:91:9E:E5:BF:BD:83:CD:F8:FF:8A:79:C6:F5
            X509v3 Authority Key Identifier: 
                keyid:14:2E:B3:17:B7:58:56:CB:AE:50:09:40:E6:1F:AF:9D:8B:14:C2:C6

            Authority Information Access: 
                OCSP - URI:http://r3.o.lencr.org
                CA Issuers - URI:http://r3.i.lencr.org/

            X509v3 Subject Alternative Name: 
                DNS:docker.vaultara.com
            X509v3 Certificate Policies: 
                Policy: 2.23.140.1.2.1
                Policy: 1.3.6.1.4.1.44947.1.1.1
                  CPS: http://cps.letsencrypt.org

            CT Precertificate SCTs: 
                Signed Certificate Timestamp:
                    Version   : v1 (0x0)
                    Log ID    : 46:A5:55:EB:75:FA:91:20:30:B5:A2:89:69:F4:F3:7D:
                                11:2C:41:74:BE:FD:49:B8:85:AB:F2:FC:70:FE:6D:47
                    Timestamp : Jul 11 15:50:53.278 2022 GMT
                    Extensions: none
                    Signature : ecdsa-with-SHA256
                                30:45:02:21:00:AF:3B:01:CF:72:B3:A5:1B:D7:D6:31:
                                9C:00:70:2B:C5:DB:7C:E7:A6:76:A4:8A:5F:E5:42:E5:
                                A3:6E:9D:6D:09:02:20:2A:6F:36:51:8D:57:74:E6:FC:
                                47:C0:A4:3C:CC:81:EC:CA:C4:35:D9:54:F4:64:B3:79:
                                8C:9A:88:CD:75:2F:21
                Signed Certificate Timestamp:
                    Version   : v1 (0x0)
                    Log ID    : DF:A5:5E:AB:68:82:4F:1F:6C:AD:EE:B8:5F:4E:3E:5A:
                                EA:CD:A2:12:A4:6A:5E:8E:3B:12:C0:20:44:5C:2A:73
                    Timestamp : Jul 11 15:50:53.740 2022 GMT
                    Extensions: none
                    Signature : ecdsa-with-SHA256
                                30:45:02:21:00:B1:81:AD:E7:D1:D4:75:F8:6F:16:60:
                                C7:A8:32:28:3E:E9:8B:AC:5D:83:CB:33:65:C7:18:73:
                                EE:2C:69:15:07:02:20:03:07:C3:8D:BE:08:18:BE:0C:
                                D2:1F:FE:84:E7:3A:69:6E:9D:A7:8E:83:E7:75:FB:8B:
                                04:E8:76:B3:05:66:EC
    Signature Algorithm: sha256WithRSAEncryption
         27:20:d9:cb:a7:dd:ec:00:43:f7:6c:23:13:0b:48:2d:a0:c2:
         70:a7:36:9f:68:e2:b7:5f:ed:49:97:07:4e:de:eb:9c:87:74:
         44:7d:e7:c7:e2:1e:4d:8b:23:e6:47:29:97:0d:ba:b3:bc:12:
         01:82:12:5d:a6:bd:cd:44:6c:56:43:a1:78:7f:58:f5:c1:43:
         19:6b:76:62:9a:22:6a:8d:47:71:00:c7:2f:81:35:0c:9e:8a:
         dd:ab:9b:84:ae:78:9d:3f:3d:a4:60:7f:35:90:d0:46:b0:44:
         16:ba:5c:0a:8f:88:fc:09:12:9c:f7:2a:25:54:14:bd:de:94:
         21:b2:d9:69:fe:67:33:44:e1:cb:71:86:69:39:c6:df:6c:cf:
         4e:d1:f2:ef:9a:92:2a:b5:93:f1:d9:cc:06:7a:16:fe:a6:fd:
         84:41:c3:36:e1:cc:5f:ae:62:bd:19:39:97:a5:88:c3:8d:88:
         a2:49:87:48:78:e5:30:e9:aa:ee:ee:f5:fc:fa:12:12:03:3a:
         6f:9c:30:2f:59:bc:93:f1:f9:01:c1:75:39:bb:af:bd:f5:bb:
         01:f3:a1:47:83:c1:80:74:79:23:4d:02:0a:fe:f5:a8:85:9c:
         ed:8d:5c:a7:56:fb:10:cb:f2:88:d4:34:a0:04:25:be:54:35:
         32:56:ab:79

Some searches state that the files may have been incorrectly put together, especially the fullchain file, but looking through them, I think this is a red herring: they all appear to be constructed correctly.

I'm really not doing much and I find it hard to believe that the certbot container I'm using is really at fault, but that's all I can think of...?

2

So far this all looks fine. The openssl verify command is rarely helpful and often behaves in unexpected ways - errors from that command are common and not generally an issue.

Do you have a specific error message generated by your client? Verification errors for Let's Encrypt certificate are often related to old libraries or outdated trust stores.

8 Likes
3

I think we'd need to know what client you're using and what the error you're getting is in order to be able to help.

That command isn't doing what you think it is. In order to validate the same way your client does, you'd need -CAfile to point to something that's the same root store as your client uses, and add -untrusted for a file of all other certificates that your server is sending in the chain. It's usually much more complicated to try to verify things that way than to just use whatever client you're actually trying to use.

I can't connect to docker.vaultara.com myself; I assume that it's not actually publicly available?

9 Likes
4

See:

6 Likes
5

I have some Go code that initially mimicks a docker login, so I'm manually replicating that in the command line:

docker login --username USERNAME docker.vaultara.com
Password: *****
Error response from daemon: Get "https://docker.vaultara.com/v2/": x509: certificate signed by unknown authority

It's obviously the "unknown authority" that's puzzling me.

When I try on my client, the messages that appear in the (server-side) container:

07/11 21:30:43 http: TLS handshake error from MY_EXT_IP:38382: remote error: tls: bad certificate
07/11 21:31:33 http: TLS handshake error from MY_EXT_IP:38382: remote error: tls: bad certificate
1 Like
6

I have some Go code that initially mimicks a docker login, so I'm manually replicating that in the command line:

docker login --username USERNAME docker.vaultara.com
Password: *****
Error response from daemon: Get "https://docker.vaultara.com/v2/": x509: certificate signed by unknown authority

It's obviously the "certificate signed by unknown authority" that's puzzling me

7

Just for complete transparency, the initial system was based on a FIPS kernel; I've just bought up an alternative that's using a non-FIPS kernel and I'm seeing the same thing...

Somewhat frustrating.

Are there any commands that I can run that would verify the files that the certbot container created?

8

Well, I'd guess that either your server isn't sending the full chain of intermediates, or your client doesn't have ISRG Root X1 in its trust store. But I don't know enough about docker or your setup to be able to give you much more detail than that.

7 Likes
9

I'm (attempting) to combine both the certificate for my domain and the fullchain into one, and configuring docker to use that ... in the hope that this will get me a little further forward. I'll let you know how I get on

1 Like
10

You do know that fullchain.pem contains cert.pem.

7 Likes
11

That apparently did the job! Thanks for everyone's feedback

3 Likes
12

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.