acme.errors.TimeoutError when renewing via ZeroSSL

Help
1

My domain is a subdomain for a high-profile customer whose domain gets treated exceptionally around the internet because the brand is so often used in fraud. They have have made a CNAME to our public dev server. We could not issue a cert through Let's Encrypt for them because they have already issued more than 50 themselves and reached some limit. So, we got a cert through ZeroSSL, which worked fine until it was time to be renewed.

I tried a certbot renew command, which seem to work fine at first and get a response back from ZeroSSL:

{"status":"processing","expires":"2024-10-22T18:36:24Z","identifiers":[{"type":"dns","value":"example.com"}],"authorizations":["https://acme.zerossl.com/v2/DV90/authz/OXaW-nBz55wBCRkpoAOlhA"],"finalize":"https://acme.zerossl.com/v2/DV90/order/x1830551LQxye9MqE8rvFQ/finalize"}

But then the what seems like the final stage, it times out:

2024-07-24 18:37:56,729:DEBUG:acme.client:Storing nonce: HNTk09WTrvYNqLr8QAMaQ5oLSejr6q0fakefakefake
2024-07-24 18:37:56,729:ERROR:certbot._internal.renewal:Failed to renew certificate example.com with error:
2024-07-24 18:37:56,730:DEBUG:certbot._internal.renewal:Traceback was:
Traceback (most recent call last):
File "/snap/certbot/3834/lib/python3.8/site-packages/certbot/_internal/renewal.py", line 540, in handle_renewal_request
main.renew_cert(lineage_config, plugins, renewal_candidate)
File "/snap/certbot/3834/lib/python3.8/site-packages/certbot/_internal/main.py", line 1550, in renew_cert
renewed_lineage = _get_and_save_cert(le_client, config, lineage=lineage)
File "/snap/certbot/3834/lib/python3.8/site-packages/certbot/_internal/main.py", line 131, in _get_and_save_cert
renewal.renew_cert(config, domains, le_client, lineage)
File "/snap/certbot/3834/lib/python3.8/site-packages/certbot/_internal/renewal.py", line 399, in renew_cert
new_cert, new_chain, new_key, _ = le_client.obtain_certificate(domains, new_key)
File "/snap/certbot/3834/lib/python3.8/site-packages/certbot/_internal/client.py", line 451, in obtain_certificate
cert, chain = self.obtain_certificate_from_csr(csr, orderr)
File "/snap/certbot/3834/lib/python3.8/site-packages/certbot/_internal/client.py", line 341, in obtain_certificate_from_csr
orderr = self.acme.finalize_order(
File "/snap/certbot/3834/lib/python3.8/site-packages/acme/client.py", line 275, in finalize_order
return self.poll_finalization(orderr, deadline, fetch_alternative_chains)
File "/snap/certbot/3834/lib/python3.8/site-packages/acme/client.py", line 259, in poll_finalization
raise errors.TimeoutError()
acme.errors.TimeoutError

We were trying nginx as the authenticator, which works fine for many other domaiins on the server.

I'm wondering if we might be running into geo-blocking for the domain or some other issue, but I didn't get many forum hits for this specific timeout issue.

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):

certbot 2.11.0.

I'm wondering if switching to DNS validation in this case might help, in which case I can loop in the customer's IT team and ask about that.

2

You'd really need to ask ZeroSSL about that. This is not the ZeroSSL support channel.

It seems Certbot times out while ZeroSSL is still in the process issuing your certificate. I have no idea how long that should take, but it might also be an issue with ZeroSSL.

There are also other free CAs besides LE and ZeroSSL out there, maybe try one of those. :man_shrugging:t2:

6 Likes
3

Note that there is a way to ask for a rate limit increase from Let's Encrypt (although this should really be filled out by the customer and not by you!). If you're actively hitting that rate limit on Let's Encrypt, it's somewhat likely that other departments at the customer are too, so other people might be able to benefit from this.

As @Osiris said, Let's Encrypt created this technology but ZeroSSL is a different entity using it with its own implementation and its own support channels for that (unless you can see specific reason to suspect a Certbot bug).

6 Likes
4

This is Finalization (order completed and validated, waiting for the CA to issue the actual cert), so it's not related to geoblocking, etc.

You need to contact ZeroSSL support but I've seen other complaints from users recently that ZeroSSL orders are timing out (e.g. take more than a minute to issue etc) and have also seen random errors from their Order endpoint etc. ZeroSSL don't have a supoprt community as far as I know so you need to speak with them directly but as far as I can see their system is just being unexpectedly slow.

Some ACE clients have configurable retry amounts/delays, some don't.

6 Likes
5

Certbot has an --issuance-timeout parameter that defaults to 90 seconds, I don't know if that's the timeout involved here but you can try increasing it.

Or, use another CA, like Let's Encrypt or Buypass Go. There are many CAs offering free certificates via ACME, and often all you need to do is change the --server that certbot is using, though some require setting up a separate free account first.

Some comparison charts:

6 Likes
6

Thanks for surfacing the rate limit increase form! I would prefer to use Let's Encrypt and will pass this suggestion back to the customer.

(unless you can see specific reason to suspect a Certbot bug).

There is an opportunity for a better user experience here. As it stands, it's not clear /what/ HTTP request timed out, or what the recommend resolution is.

Had I received a more useful error message, I would have had a much better chance of resolving the issue directly. Based on the feedback here, a certbot behavior like would have been better:

ERROR: The finalization step timed-out when waiting a response for the CA to issue the cert from example.com/some/path. Using the --issuance-timeout parameter with a higher value may help.

3 Likes
7

Sure, better messages are always nice. The best place is to suggest at the github EFF uses for Certbot

4 Likes
8

Thanks. Issue submitted. wish: Better error message when "acme.errors.TimeoutError" is thrown. · Issue #9972 · certbot/certbot · GitHub

2 Likes
9

I learned some more about the root cause here.

Our customer turns out to be using CAA DNS records which list specific vendor domains that it authorizes to issue SSL certs for it. Let's Encrypt is on the list, while Zero SSL is not (nor any other services that issue normally issue free certs).

So that could explain why Zero SSL consistently times out, although timing out seems like an odd way to handle that-- 403 or some other standard HTTP code seems like the way to go.

Best path forward seems to be get the Let's Encrypt quota raised, which I'm working on with them now.

10

If a CA is not allowed according to CAA, the error from the CA usually says so.

Also, the timeout is from Certbot, not ZeroSSL.

3 Likes
11

See also Google Trust Services. Cloudflare use them as well. [In general, having a multi-CA strategy is a good idea especially if you have many domain to cater for. Ideally falling back to another CA when one CA becomes problematic.]

2 Likes
12

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.