Chain incomplete since feb 8 2024

Since feb 8 the policy of LetsEncrypt changed: see Shortening the Let's Encrypt Chain of Trust. The fullchain is not a full chain anymore.

Since then certificates on my system are invalid, because the chain is incomplete. See e.g. SSL Server Test: adm-oldenhage.nl (Powered by Qualys SSL Labs)

For some other sites I added the missing part of the chain by hand, but feel that is not the correct solution. What can I do to prevent this error in a more robust way?

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is:
adm-oldenhage.nl

My web server is (include version):
Apache/2.4.56 (Debian)

The operating system my web server runs on is (include version):
Debian 11.8

I can login to a root shell on my machine (yes or no, or I don't know):
yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):
No

your site sends just leaf certificate without any chain, as ssllabs says

5 Likes

True, and this is since feb. 8th. Before this date the certificate chain request at letsencrypt gave the full chain including the R3 intermediate. But now the R3 is left away, and only the certificate itself is returned upon requesting a chain. I don't know if this has to do with the policy change since feb 8th, and if manually adding the R3 intermediate might give problems after September 30th.

Think you were removing last cert from chain as script? What's your client?

5 Likes

The software didn't change for at least 2 years. I have a custom client in our CMS, based on this code: GitHub - skoerfgen/ACMECert: PHP client library for Let's Encrypt and other ACME v2 - RFC 8555 compatible Certificate Authorities.

didn't found obvious problem from them: have you have more verbose log while getting a certificate

5 Likes

Thanks for your help so far. Will make a log and post it here!

1 Like

This is the log. This exact sequence used to give both the certificate and the R3 intermediate up to feb 8th. After that it only gave the certificate without the R3.

2024-02-19 13:24:01 - Getting account info
2024-02-19 13:24:01 - Initializing ACME v2 live environment
2024-02-19 13:24:01 - Using cURL
2024-02-19 13:24:01 - https://acme-v02.api.letsencrypt.org/directory [200] (0.54s)
2024-02-19 13:24:01 - Initialized
2024-02-19 13:24:02 - https://acme-v02.api.letsencrypt.org/acme/new-nonce [200] (0.16s)
2024-02-19 13:24:02 - https://acme-v02.api.letsencrypt.org/acme/new-acct [200] (0.17s)
2024-02-19 13:24:02 - AccountID: https://acme-v02.api.letsencrypt.org/acme/acct/2896845
2024-02-19 13:24:02 - Account info retrieved
2024-02-19 13:24:02 - Creating Order
2024-02-19 13:24:02 - https://acme-v02.api.letsencrypt.org/acme/new-order [201] (0.29s)
2024-02-19 13:24:02 - Order created: https://acme-v02.api.letsencrypt.org/acme/order/2896845/245751290147
2024-02-19 13:24:02 - All authorizations already valid, skipping validation altogether
2024-02-19 13:24:02 - Using provided CSR
2024-02-19 13:24:02 - Finalizing Order
2024-02-19 13:24:04 - https://acme-v02.api.letsencrypt.org/acme/finalize/2896845/245751290147 [200] (1.47s)
2024-02-19 13:24:04 - Requesting certificate-chain
2024-02-19 13:24:04 - https://acme-v02.api.letsencrypt.org/acme/cert/031fd740f78f86186087083a79d707ca8797 [200] (0.17s)
2024-02-19 13:24:04 - Certificate-chain retrieved

There are two certificates in that file. The end leaf cert first and the intermediate after it.

If your ACME client malfunctions, it probably means there's a bug in the client.

Not an issue with LE anyway.

4 Likes

Thanks, I see what you mean. Must be something like a timing-issue in the client then, maybe requesting the cert to soon. I will investigate. The behaviour definitely (reproducibly) changed since feb 8th.

At least I'm glad that the intermediate is still delivered, thanks!

1 Like

I found it and solved it. It indeed was a problem with the client. @orangepizza and @Osiris Thanks a lot for helping me out!

5 Likes

I'm curious to hear what the problem was exactly. Because "timing issue" sounds a little bit strange to me: an ACME client would always retrieve that /acme/cert/ path for the certificate and Let's Encrypt always includes the entire chain: there's no way in the ACME protocol to download just the end leaf certificate or just the chain.

5 Likes

It was not a timing issue, but @orangepizza was right all the time. Before feb 8th LetsEncrypt sent 3 certs in the chain, presumably the X3/X1 compatibility cert. The code I used removed that one, i.e. the last one.

I didn't realise that that could be something in the code. After feb 8th, the chain consists of 2 certs, and the code still removed the last one, leaving an incomplete chain.

So to the contrary to what I thought at the moment, the second reply of @orangepizza really hit the mark.

4 Likes

Thanks for the explanation :slight_smile:

It's probably a good idea to make your client implementation future proof by planning for change: things can always change suddenly, so everything should not require manual intervention.

3 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.