Request to unblock rate limit

Hi

Running certbot renew produces:

There were too many requests of a given type :: Your IP, 2a03:b0c0:1:d0::65:b001, has been blocked due to ridiculously excessive traffic. Once this is corrected you may request this be reviewed on our forum https://community.letsencrypt.org

I have traced the problem to having both Nginx and Caddy webservers installed and this problem has now been corrected (in favour of Nginx). I have waited a week but still get the rate limit message.

Please could somebody advise the procedure for unblocking the server so I can renew the certificate?

many thanks

Hi @dmlogic,

Glad to hear you were able to sort out the problem that was causing you to submit so many requests!

As the error message mentions the only way out of this state is by opening a community forum thread (like this one!).

I’ve raised a ticket with operations to have 2a03:b0c0:1:d0::65:b001 cleared. I will let you know via this thread when the change has been made in production.

Thanks for addressing your misconfiguration! We appreciate it!

1 Like

That’s the first time that I’ve heard of the “ridiculously excessive traffic” rate limit. Should it be documented somewhere, or is it just meant as a failsafe for super-extreme situations that nobody should anticipate encountering? :slight_smile:

1 Like

It’s this - reserved for cases where someone has been hitting a 429 excessively for a considerable period. I think documenting it will be of limited value since the error message explains the only recourse already and the only way to find yourself in this position is from hitting an already documented rate limit in excess for long enough to have ops apply this block adhoc.

1 Like

Hi again @dmlogic,

The block on 2a03:b0c0:1:d0::65:b001 has been removed. Please let me know if you still experience the same error.

Thanks!

@dmlogic, can you provide more details on how the interaction between Nginx and Caddy produced such excessive traffic? I’d love to work with @mholt on reducing Caddy’s impact in error situations like this one. Was Caddy failing to start because it couldn’t bind a port it needed?

2 Likes

Yeah, pretty sure that was something like that. I had a report the site was down, found the server basically dead through lack of disk space caused by a full /tmp folder. Realised pretty quickly that an aborted experiment with installing Caddy had been left running and I think it was just constantly fighting Nginx for the ports. I didn’t attempt to investigate beyond that, just cleared out /tmp, got rid of the Caddy service and it all came back up.

Did you use Let’s Encrypt’s staging endpoint for your experimental setups?

Was this with systemd or init? If systemd, were you using the systemd scripts provided in the Caddy repo?

Thanks,
Jacob

1 Like

sorry I don’t remember exactly how it was set up. I dumped Caddy in a hurry when I realised what was going on.

From memory, It was using init.d based on examples included with Caddy, but I didn’t keep the script.

I’ve updated certbot and renewed the certificate with certbot --nginx certonly -d {domain}, but attempting certbot renew --dry-run produces the same error as before. Would it be best to just clear everything out and start again?

Interesting! Thanks for reporting that. Since --dry-run uses the staging environment I suspect our operations team only removed the block for the production environment. I’ll ask about getting this fixed and report back when I know more. Thanks for your patience.

Hi again @dmlogic,

I’m told the staging ban has been lifted. You should be good to go with --dry-run as well. Thanks again.

Lovely, thanks for all the help

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.