Unable to get local issuer, even with intermediate certs

CharmingMidnight8191 · May 23, 2021, 6:51pm

My domain is: clic.jsilber.ca

I ran this command: openssl s_client -connect clic.jsilber.ca:33333

It produced this output:

CONNECTED(00000003)
depth=0 CN = clic.jsilber.ca
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 CN = clic.jsilber.ca
verify error:num=21:unable to verify the first certificate
verify return:1
---
Certificate chain
 0 s:CN = clic.jsilber.ca
   i:C = US, O = Let's Encrypt, CN = R3
 1 s:C = US, O = Internet Security Research Group, CN = ISRG Root X1
   i:O = Digital Signature Trust Co., CN = DST Root CA X3
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIFIzCCBAugAwIBAgISBFlFx9pqv1IX336hqs0Up1ocMA0GCSqGSIb3DQEBCwUA
MDIxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQD
EwJSMzAeFw0yMTA1MjMwNDU3MDZaFw0yMTA4MjEwNDU3MDZaMBoxGDAWBgNVBAMT
D2NsaWMuanNpbGJlci5jYTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEB
AMnDubousYPfADgQWkTaBRVSO8XJ/lwOuhSAoL98B1qb9cZM6r4mu0vDi32Dtqzv
dLAUDYWEkpS0dROYouBk3nEMSomnS6oOpd9MVpJtEMy1+Ns2CEA8phGS5sij8EfG
GXYph50vNOFLZueEK3BpxgH5PlL6O18LdgFcGiUl3DsQtYaMJWjp4gN9hEH7CTj8
+RY566zpuTg04o57K7cPqypod5iKHAndQuQvIxeg2tSs4y/kYTGevidowOGUOwk3
GQf5U686SBozPWp7d/hGkifDuILego3H8fs+YV9Gg9CGQKjzkg+igRvUIZJK0+OQ
gMnpOMfBp8tYDH7o3W9pLNsCAwEAAaOCAkkwggJFMA4GA1UdDwEB/wQEAwIFoDAd
BgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwDAYDVR0TAQH/BAIwADAdBgNV
HQ4EFgQUnKgJw3zdJm7rn94Akd6A5Wd1ZvkwHwYDVR0jBBgwFoAUFC6zF7dYVsuu
UAlA5h+vnYsUwsYwVQYIKwYBBQUHAQEESTBHMCEGCCsGAQUFBzABhhVodHRwOi8v
cjMuby5sZW5jci5vcmcwIgYIKwYBBQUHMAKGFmh0dHA6Ly9yMy5pLmxlbmNyLm9y
Zy8wGgYDVR0RBBMwEYIPY2xpYy5qc2lsYmVyLmNhMEwGA1UdIARFMEMwCAYGZ4EM
AQIBMDcGCysGAQQBgt8TAQEBMCgwJgYIKwYBBQUHAgEWGmh0dHA6Ly9jcHMubGV0
c2VuY3J5cHQub3JnMIIBAwYKKwYBBAHWeQIEAgSB9ASB8QDvAHUARJRlLrDuzq/E
QAfYqP4owNrmgr7YyzG1P9MzlrW2gagAAAF5l8xq8wAABAMARjBEAiBi9gPkZZxh
fSTlfoJeu/Vq/BBc4a2yOdn8vFguyfr7QQIgHSwOhRFY/kZd7R/9Noe4uY23z6C0
1GfVh/jmloZusTwAdgB9PvL4j/+IVWgkwsDKnlKJeSvFDngJfy5ql2iZfiLw1wAA
AXmXzGsIAAAEAwBHMEUCIQDcsumzcXP2eUd9osoXFHOK/g2UekM0X9070PAg8Yvd
2wIgbb8sbuYc2W8gdG+w6NIhuVEBVB0Qao4/2YAgGawWdzAwDQYJKoZIhvcNAQEL
BQADggEBAB/NxX9gwDSqtAXnxsjTVKYH5JAWUnGC1ItWQYsSYrO7Bew6vEWQ92OX
VFbpDNJPqVRphW1dpapzX1AF8EqDcrw1Jxcr8+ZBUfg/+4dsAhiTizkff31LNKhC
5IdDcFpervQZB2MXC7NEpFM4wwcba4OoCJirXQMBce0/gTbnvVEe9RVGHIJSgT2z
xOM7FAQE9wZFmWP4OcMgYzBTPJ4GpqKf75nbyV867fQiIlfecPbdXcfMlJDalnGd
fxXAhpnktORpZ8RX6avda+bzBo2vUp9juv8G5I/AGr6q1Fnln7FqRYR0eHUYIw/a
DhE8L9wBXVFvL8tgLJFv4nQwTwv6wmo=
-----END CERTIFICATE-----
subject=CN = clic.jsilber.ca

issuer=C = US, O = Let's Encrypt, CN = R3

---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 3260 bytes and written 387 bytes
Verification error: unable to verify the first certificate
---
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
Server public key is 2048 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 21 (unable to verify the first certificate)
---
---
Post-Handshake New Session Ticket arrived:
SSL-Session:
    Protocol  : TLSv1.3
    Cipher    : TLS_AES_256_GCM_SHA384
    Session-ID: 675398C03924A5FDAD7D4613AFA739A9F00B50AFE8767161F2EC62901B816278
    Session-ID-ctx: 
    Resumption PSK: 05141D02E4BD40D02F068EAEF8F8847324C7DE53C8F6C8E6A08050D6550A9C7B2D042E34B04B800C87AD86910BF84E8A
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 7200 (seconds)
    TLS session ticket:
    0000 - 2f 8d 11 a1 12 75 78 db-cb a8 6b 0d 7e 8e 28 f1   /....ux...k.~.(.
    0010 - 4a 62 aa fb 02 b7 11 8f-e9 f8 72 14 77 92 71 d8   Jb........r.w.q.
    0020 - 5f 08 c4 0e 07 34 11 7d-24 4f 5b da d3 d5 3f 85   _....4.}$O[...?.
    0030 - 00 a6 6e 46 6f 32 06 ac-5c 6a 28 dd 9e af 05 36   ..nFo2..\j(....6
    0040 - 9e b0 39 fe b6 b7 37 d5-c6 13 1b 46 8f 19 30 83   ..9...7....F..0.
    0050 - b6 6f a6 d8 93 05 65 d8-89 1b 73 0a d0 d4 e5 84   .o....e...s.....
    0060 - 9a 89 a1 6d 5a 00 09 82-5f 9c 61 61 6f d6 d2 55   ...mZ..._.aao..U
    0070 - d3 04 81 0d 95 98 b3 a2-e9 44 bd 37 ef c4 ad a1   .........D.7....
    0080 - 28 1e 92 78 dc f2 25 01-a3 9a 90 88 7e 13 77 5d   (..x..%.....~.w]
    0090 - 39 e1 b3 95 fb a0 2b 10-cf 43 91 b1 2b 7a 7d a5   9.....+..C..+z}.
    00a0 - 17 b5 09 93 3c 64 75 31-62 d7 95 20 5d 64 7d c6   ....<du1b.. ]d}.
    00b0 - 4c c7 d5 35 91 6d d1 28-94 88 4a d7 21 90 14 12   L..5.m.(..J.!...
    00c0 - 61 f3 ca 5f 74 53 b6 5f-f1 38 f3 ac 65 df ba eb   a.._tS._.8..e...

    Start Time: 1621794903
    Timeout   : 7200 (sec)
    Verify return code: 21 (unable to verify the first certificate)
    Extended master secret: no
    Max Early Data: 0
---
read R BLOCK
---
Post-Handshake New Session Ticket arrived:
SSL-Session:
    Protocol  : TLSv1.3
    Cipher    : TLS_AES_256_GCM_SHA384
    Session-ID: 2B653CC1E1544EEA8EFD9C473E612CC0B589BCFE112F9572E1A80B77FC1ED7AB
    Session-ID-ctx: 
    Resumption PSK: 579697D8BA91D210E26B3F613EC11452059D1945EADEE5D2F74231112EF61FD233190E0E02D14440847559056B2E2D06
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 7200 (seconds)
    TLS session ticket:
    0000 - 2f 8d 11 a1 12 75 78 db-cb a8 6b 0d 7e 8e 28 f1   /....ux...k.~.(.
    0010 - 78 d8 c1 3d 00 8e 5e ee-1e 8a 3e b6 5c 86 d3 e1   x..=..^...>.\...
    0020 - 45 22 64 7c 50 c7 d6 a3-1c 0b db f8 87 3c 26 67   E"d|P........<&g
    0030 - 43 4e 0c 9a d9 51 56 32-db 5e 59 ad 8b fc 93 11   CN...QV2.^Y.....
    0040 - 74 c0 57 3d 47 c6 2f 1c-4f 2a 6c 44 9a 4e cd ac   t.W=G./.O*lD.N..
    0050 - ef b6 61 b0 cf 49 36 03-67 c8 8e 3d 0f 5c fd 23   ..a..I6.g..=.\.#
    0060 - 80 cb 8a de d8 d3 a6 18-67 40 a9 ab 99 2c 42 6f   ........g@...,Bo
    0070 - 2b c4 32 4a d1 fb aa a1-b9 4e b4 bf 1a 30 62 fa   +.2J.....N...0b.
    0080 - 82 b5 09 29 1d 88 67 c0-c2 0b 96 79 7c 11 60 30   ...)..g....y|.`0
    0090 - 72 b3 88 19 66 f6 fb e3-80 9c 94 ad 98 c0 fd 11   r...f...........
    00a0 - e6 ac 93 61 61 b3 ad d6-2e 33 1b 2d ac 71 ea f7   ...aa....3.-.q..
    00b0 - 87 61 3b 72 4c de 77 e4-89 2c 35 a2 07 40 0a d2   .a;rL.w..,5..@..
    00c0 - 44 fd fa 97 0a d3 93 ad-b6 50 12 66 db 3f 69 bd   D........P.f.?i.

    Start Time: 1621794903
    Timeout   : 7200 (sec)
    Verify return code: 21 (unable to verify the first certificate)
    Extended master secret: no
    Max Early Data: 0
---
read R BLOCK

My web server is (include version): Not a web server, a chat server I wrote in Python3. The server's context is:

self.context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH) # Context wrapper to apply TLS over sockets
self.context.load_cert_chain(certfile='acme_chain.pem', keyfile="acme_key.pem")

The LetsEncrypt cert was generated using the Terraform 'vancluever/acme' ACME module

The operating system my web server runs on is (include version): Ubuntu Server 20.04 Docker container, running on an Ubuntu Server 20.04 instance.

My hosting provider, if applicable, is: Self hosted on Linode.

I can login to a root shell on my machine (yes or no, or I don't know): Yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): No

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): Terraform ACME module: https://registry.terraform.io/providers/vancluever/acme/latest/docs/resources/certificate

Hi everyone, I've written a server and client for a CLI chat program in Python and deployed it using Ansible, Terraform, and Docker. To secure it I've generated a cert from LetsEncrypt, but when verification is enabled the client keeps failing with, "Unable to get local issuer certificate". Apparently that's often caused by not including the full cert chain, but when I look that the cert chain PEM file it includes my cert, Let's Encrypt's R3 cert, and the ISRG Root X1 cert. I thought the my problem might be that the DST Root CA X3 cert wasn't on there, but it turns out Stack Overflow's website has the same basic chain and theirs stops at ISRG Root X1 too.

This is the generated cert chain:

Perhaps there's something I need to do with the cert key other than include it in the context? I've gone over the Python SSL module docs but haven't found anything I've missed yet.

petercooperjr · May 23, 2021, 6:58pm

CharmingMidnight8191:

Certificate chain
 0 s:CN = clic.jsilber.ca
   i:C = US, O = Let's Encrypt, CN = R3
 1 s:C = US, O = Internet Security Research Group, CN = ISRG Root X1
   i:O = Digital Signature Trust Co., CN = DST Root CA X3

You're missing the certificate for R3 signed by ISRG Root X1 here. That is, there should be three certificates in the chain, not two (for any certificate issued after May 4). I'm not familiar with your ACME client, but I'm guessing it should be putting the complete chain (all three certificates) into that acme_chain.pem file you're using, but maybe it isn't?

Osiris · May 23, 2021, 7:05pm

There is an extra space before the start of the first intermediate certificate. My OpenSSL also complains about it when I leave that space in a copy/paste. Perhaps if you'd remove the space it could work?

There are three certs in the chain.

petercooperjr · May 23, 2021, 7:17pm

Heh. Good eye. I was just looking at the openssl s_client output and saw that it only showed two of them. You're right, it's probably that the format of the file isn't quite right.

Osiris · May 23, 2021, 7:25pm

It either was indeed the extra space or something else @CharmingMidnight8191 did, as I see a good chain now when I connect to the service

CharmingMidnight8191 · May 23, 2021, 7:26pm

Geez, you're quick, haha!

Yup, it looks good. now.

Now I just need to figure out how to make Terraform not generate the space. I probably one somewhere weird in a variable, so that'll fun to find XD

Thank you so much!

Osiris · May 23, 2021, 8:14pm

Does Terraform actually provide the full chain as a single file? In the documentation you've linked in the OP it says it exports certificate_pem and issuer_pem, but not a concatenation of both. Did you concatenate those two into a single full chain file yourself?

CharmingMidnight8191 · May 23, 2021, 8:22pm

Yup, that's correct. Originally I just tried to concatenate them, but it didn't like the two variables directly next to each other, which is how I ended up with the space, ie: ${var1} ${var2}. To get around that I changed the code to use an 'output' command where I called join on a list containing certificate_pem and issuer_pem:

output "cert_certificate_pem" {
    value = acme_certificate.cert.certificate_pem
}
output "cert_intermediates" {
    value = acme_certificate.cert.issuer_pem
}
output "cert_full_chain" {
    value = join("\n", ["${acme_certificate.cert.certificate_pem}", "${acme_certificate.cert.issuer_pem}"])

To actually retrieve the output from Terraform I need to use the -raw flag, eg. terraform output -raw cert_full_chain, otherwise it appends extra characters.

system · June 22, 2021, 8:23pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.