We’re a hosting provider who has been provisioning Let’s Encrypt certificates for use by our customers for about 2 years now. Everything has been working fine with provisions and renewals until earlier this month, when we started getting Too Many Pending Authorizations rate limiting errors.
We are running
certbot from another (proprietary) script to handle renewals and cert provisioning, using
webroot validation, and managing web server configuration with our own tool set. We’re using a single Let’s Encrypt account across all of our (1500+) servers.
I’ve had success using
LE_FIND_PENDING_AUTHZ.py to eliminate our pending authorizations. However, for some reason they are coming back quite quickly, and we’d like to track down the root of the problem. Canceling pending authorizations is troublesome due to the high number of authorizations we create in the 7 day window, so we’d like to figure out how to prevent this from happening in the first place.
Now that we’re having this problem, using a single account for all of our hosting servers feels like it may not have been the best choice. However, at least it makes it easier for us to clear out the authorizations centrally rather than having to manage it across many different accounts.
Regarding “too many pending authorizations”: I’ve read “That Should Never Happen” as long as we use
certbot and don’t issue certs with a high number of SAN domains (we only issue at most one SAN for www along with the main domain name); but it is happening.
My initial suspicion was: if our script that calls
certbot times out and kills off
certbot in the middle of issuance, could this leave pending authorizations open? What about when
certbot is processing renewals? Can authorizations be created by failed cert provisioning or failed cert renewal attempts when certbot completes without interrupotion?
We’ve drastically increased our timeouts and added instrumentation to help us determine if timeouts are the source of problems in the future, but I don’t have logs on whether this happened in the past.
If there is any other way that using
certbot might create pending authorizations, please tell me. If there is any way to get a list of pending authorizations via ACME, that would be much more efficient than sending requests for thousands of authorizations just to find the 300 that are still pending.
My domain is: (Various, we are a hosting provider)
I ran this command:
certbot certonly --webroot --webroot-path ... --account ... -d example.com -d www.example.com -n certbot renew --account ... --post-hook ...
It produced this output:
2019-05-28 01:10:06,201:ERROR:certbot.log:There were too many requests of a given type :: Error creating new order :: too many currently pending authorizations: see https://letsencrypt.org/docs/rate-limits/
My web server is (include version): Not relevant because we use
certonly? Apache 2.4 and NGINX
The operating system my web server runs on is (include version): Ubuntu, FreeBSD; various versions
The version of my client is (e.g. output of
certbot --version or
certbot-auto --version if you’re using Certbot): 0.34.2