Question regarding rate limits and multiple servers

Hi,
I received "too many certificates (5) already issued for this exact set of domains in the last 168 hours" error and read Rate Limits - Let's Encrypt.
What is the proper way to renew certificates if I have a few servers serving the same set of domains?
Let's say I have 10 servers? Renewing on one of them and rsync to others seems me not so elegant solution...
Any suggestions?

Thanks, Vitaly

1 Like

Why not? As that is the solution.

Unless your duplicate certificates all have different private keys for some specific reason, I don't see why you wouldn't redistribute a single certificate among multiple servers?

4 Likes

Unless your duplicate certificates all have different private keys for some specific reason, I don't see why you wouldn't redistribute a single certificate among multiple servers?

I don't like this flow because in this case, I need some kind of "orchestration" - to choose one server as a master, schedule rsync, remember to update script when I'll retire master, and so on.

You can install a certificate on the load balancer alone, or you can do something really dirty where each server has a different certificate for two domain names, all of which include commondomain.com but each of which has a different server1.commondomain.com, server2.commondomain.com, etc.

This way it's not "the exact set of domain" anymore, and each server renews by itself (they all need to validate commondomain.com, eh)

2 Likes

Yes, it's more work, but, IMO, it's also way more elegant. Less abuse on the Let's Encrypt infrastructure too.

5 Likes

Yet instead, you preferred to implement an unsupported and ill-advised anti-pattern as your "solution".

There are numerous ways to correctly handle multi-node setups, which have been discussed repeatedly here and are easily searchable. Options include terminating on a single node as @9peppe suggested, using post-hooks in a "push" model, a cron-based daily "pull" model, storing the certificates in the cloud, storing the certificates on a shared volume, using an internal certificate manager to deploy within your lan, and so on.

6 Likes

For all systems that are reusing the same private key, they can also pull the latest cert from the Internet.

The "renewals" themselves can be done from any system - even from an unrelated system (via DNS auth).

4 Likes

To be able to pull the renewed cert from a certificate transparancy log aggregator (which is what you mean I believe when saying "the internet"), you do need access to the private key.. So either redistribute any newly generated private key (and there is no benefit what so ever to pull the cert from the web) or distribute the private key just once and re-use the private key at renewal.

Thought it might be worth mentioning that latter condition :wink:

4 Likes

Yes, "using" should have read "re-using" [the same private key].
[updated the post]

2 Likes

Getting the certificate that way is easy. Getting the right chain can be a problem.

2 Likes

An alternative approach (amongst many) is to use DNS validation instead of http validation and have a specific certificate renewals server (with centralised config for your ACME account, certificate configurations etc). So the important part is you don't just have to use http-01 validation from one of your web servers.

You can then distribute your certificates using either push or pull (rysnc, ssh/sftp, a key vault such as HashiCorp Vault etc). A 10 server load balanced farm is quite big, so I'd be looking at formalising a solution that's well understood and easily supported by the rest of your infrastructure team, with monitoring of the renewal and deployment processes etc.

You can use any acme tool for the actual certificate renewals, it's your deployment that you need a comfortable solution for. The app I work on (https://certifytheweb.com) can do the centralised renewals with sftp or vault deployment, it's currently Windows only but if linux is essential (for the central cert manager) we will shortly be offering it to alpha testers, so do message me if interested in testing that.

3 Likes

Many thanks to all! Vitaly

3 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.