We’ve written an automated service to issue/renew certs from Let’s Encrypt. So far, it’s been working really well. We also requested a rate limit increase (5K certs/week) which was approved. Over the weekend, we saw that the service was seeing the following error:
“acme-v01.api.letsencrypt.org\nAn unexpected error occurred:\nThere were too many requests of a given type :: Error creating new registration :: too many registrations for this IP”
Our service is distributed across multiple nodes.Each node uses it’s own account to do all issues/renewals.
the only rate limit you would have likely to have hit is this one:
We have two other limits that you’re very unlikely to run into.
You can create a maximum of 10 Accounts per IP Address per 3 hours. You can create a maximum of 500 Accounts per IP Range within an IPv6 /48 per 3 hours. Hitting either account rate limit is very rare, and we recommend that large integrators prefer a design using one account for many customers.
If you are using a provider such as DigitalOcean that have shared /48 spaces it might have been that lots of people registered or someone had buggy code that created lots of registrations.
The rate limit applied to the /48 has a different error message than "too many registrations for this IP" so we can rule out that rate limit. It must be the rate limit per IP.
How many nodes do you have? Are they using a unique external IP per node or is it behind NAT? Our recommendation is to use one account ID across your entire organization where-ever possible.
I'm seeing one IP (ending in .118, currently getting 429 rate limit responses to new-reg) creating a significant number of accounts. 176 over the last month. This is why you're hitting the rate limit.
Is it possible one of your nodes has gone off the rails and is recreating its own account too frequently?
@cpu thanks for the response. We have 4 nodes - 2 in US, 2 in EU. Each of them has it’s own public IP. They are GCE instances.
We are using certbot as the LE client. Isn’t certbot the one that does the account registration? Cause I am pretty sure the service itself doesn’t do any registration.
That's correct, Certbot does the registration. That said it shouldn't be creating so many registrations for one IP.
@amraks Were you able to investigate why the .118 node has been creating so many registrations? If this wasn't a problem before it indicates something might have changed.
@schoen@bmw Do you know of any condition that could cause Certbot to create registrations over and over?