Returned certificate was incomplete

We use Let’s Encrypt to issue certificates for customers, and we’ve been observing the issue that we see some incomplete certificate returned from the API time to time.
We use Ruby client (https://github.com/unixcharles/acme-client, v2.0.0), and we see this at “Downloading a certificate” step, and order.certificate there is incomplete: https://github.com/unixcharles/acme-client#downloading-a-certificate

Usually, we see something like:

-----BEGIN CERTIFICATE-----
certificate
-----END CERTIFICATE-----

-----BEGIN CERTIFICATE-----
chain certificate (ca_certificate)
-----END CERTIFICATE-----

However, when it’s incomplete, it looks like (recent failed one’s example, happened on Jan 10):

-----BEGIN CERTIFICATE-----
certificate
-----END CERTIFICATE-----

-----BEGIN CERTIFICATE-----
chain certificate, but cut in the middle, after line 21 (usually it has 25 line)

Usually, the second try with the same set of identifiers will go well. We noticed that this started happening Nov 2019 or so (it might have been happening before that too), and this happened 2-5 times per month so far (we issue a lot of certificates per month, so the percentage is not high). This tend to happen with the certificate with lots of identifiers (20+).

Any idea how this could be happening, or heard any similar issues before? Happy to provide more info.

2 Likes

You shouldn't have to repeat the whole order - just performing the download operation again should be sufficient.

The closest thing I can think of is that Let's Encrypt changed CDN providers to Cloudflare in September 2019. There were a couple of teething issues but nothing as drastic as this that I know of.

I'm surprised that some kind of network-based truncation is happening without any error being raised at all.

Maybe you can instrument acme-client's use of faraday (its HTTP client) to print the raw response body and headers like content-length, to try and narrow down where the truncation is happening.

3 Likes

Thanks for the reply! Interesting point of changing CDN provider.

You shouldn’t have to repeat the whole order - just performing the download operation again should be sufficient.

Very good point, I'll tweak the code to perform the download operation again instead of repeating the whole order.

Given how often this is happening and there is some workaround (perform the download operation again), I'll not do further debugging for now. I'll update here if I see this more frequently or find something else.

Thanks again!

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.