Same Wildcard certificate on multiple servers

Hey!
I'm trying to understand if this situation is possible,
I've read lots of documentation and none seems to give me an answer.

I want to know if there is a possible way to make a wildcard certificate (*.exmple.com),
for instance on 30 different servers (different public IPs).
From what I understood there is a duplicate limit of 5 certificates, so I won't be able to renew all of the certificates, since that they would all make a renewal request to the same exact FQDN.

The only option is to make the request on one server and have centralized storage so everyone would be able to get it
Am I correct?
Thanks!

2 Likes

Not exactly.
The duplicate limit of 5 may prevent the last 25 from getting a duplicate cert during that same week.
But it wouldn't limit 5 from getting a cert on the following week.
And so on...
So that each week 5 of them can easily obtain a new and duplicate cert.
[90/7*5=] >40 certs should be allowed without any problem.
Furthermore, renewals are not part of the limit count, so the 5/week can scale up indefinitely.

1 Like

Just to make sure things are clear...

The main limit is Certificates per Registered Domain (50 per week). A registered domain is, generally speaking, the part of the domain you purchased from your domain name registrar. For instance, in the name www.example.com, the registered domain is example.com. In new.blog.example.co.uk, the registered domain is example.co.uk. We use the Public Suffix List to calculate the registered domain. Exceeding the Certificates Per Registered Domain limit is reported with the error message too many certificates already issued, possibly with additional details.

...

Renewals are treated specially: they don’t count against your Certificates per Registered Domain limit, but they are subject to a Duplicate Certificate limit of 5 per week. Note: renewals used to count against your Certificates per Registered Domain limit until March 2019, but they don’t anymore. Exceeding the Duplicate Certificate limit is reported with the error message too many certificates already issued for exact set of domains.

1 Like

With that many servers, I'd definitely support the approach of having the request in one place (probably using the DNS-01 challenge would be easiest) and centralized secured storage of the key that then gets distributed everywhere.

Presumably, you have some method already in place to securely keep that many servers in sync with whatever code/content they already use, and perhaps (though I'm speculating on your architecture), hacking into that process is probably already almost equivalent to hacking the private key (in the sense that rather than m-i-t-m'ing you an attacker could just put compromised code on your server directly). Adding the private key and certificate into that process (assuming proper controls/permissions around it) is probably not really exposing you to much more risk in practice.

I'm sure other people would disagree about the best approach, as it really depends on a lot of things. Engineering is all about making tradeoffs since there are very few perfect solutions to problems.

3 Likes

Though if you are paranoid about passing the private key around at each renewal, it's also possible to renew using the same private key as the previous cert. You'd still have to distribute the original key when the server is spun up, but the renewal process would only have to copy around the newly signed public certificate.

3 Likes

That was exactly my thought, @rmbolger.

One key to rule them all for all time... :old_key:

1 Like

Make sure all your ciphers use PFS.

2 Likes

so the thing is that I'm trying to automate the whole process, so I'd not have to deal with certificate renewal, but if this is the situation it's just as getting a regular certificate and installing it every time it expires

1 Like

You should be able to automate distributing the certificate to all your servers. I'm not sure quite what problem you're seeing. How do you keep a fleet of that size in sync (in terms of code, content, configuration, etc) today?

2 Likes

You can do it once (fully automated), and then copy that exact same process to all your servers.
As stated, (in the worst case scenario) you should be able to add 5 duplicate certs per week and bring the total number of servers to however many are needed. And they will all renew as expected from there on.

OR

Do it in one and sync all the changes to the rest (using only one cert).

1 Like

As a bonus, if you copy the same ACME account credentials to all the servers, only one would need to actually perform the dns-01 challenge every 28 days. None of the other servers would need to perform a dns-01 challenge and would thus automatically be issued certificates since the single ACME account would already have a cached validation from the actions of the one server.

1 Like

not sure I understand your intention with "you should be able to add 5 duplicate certs per week and bring the total number of servers to however many are needed"
can you please elaborate?

1 Like

The fleet sync configuration are divided per environment and don't have the same configuration on each environment, so that's the issue

1 Like

What @rg305 is meaning is that Let's Encrypt counts the number of certificates issued in any given 7-day period that have the exact same subject alternative names (SANs) listed. These are known as duplicate certificates. Only 5 duplicate certificates are allowed for any specific list of SANs in any given 7-day period. It does not matter if those duplicate certificates are issued from the same account/machine or different accounts/machines.

Ok, so it means that I could change the certificate only on 5 machines every week, did he meant to spread down the renewal on the machines every week to a different 5 machines each time? if so it sounds like lots of maintenance to do

1 Like

You've correctly got the idea (and the complication).

The renewals are a completely separate issue from the initial cert issuance.
If all servers will do their own thing and will essentially be duplicating certs, then only five can be newly issued per week.
But once issued, they can renew (without limits) at any point in time.
So that even when all of your servers do have so called duplicate certs, they can all renew on the same day (if you wanted them to).

1 Like

I thought I already covered this:

Maybe I'm misunderstanding your plan here, @rg305?

1 Like

Ok so that's new information from what I initially knew.
Basically based on what you're saying the only issue is the initial cert issuance which is limited to 5 issues per week,
but lets say I'd issue every week to 5 different machines, eventually I'd be able to renew the certificate in all of them with no problem?

1 Like

That is not the only option, but that is the best option. You don't necessarily need to use a "pull" model from centralized storage, you can use a "push" model where the computer that procures the certificate installs it on all the client machines.

I have at least the following systems active:

  1. one cert is procured on an office laptop with DNS-01 challenges, then pushed onto nodes via a script
  2. one cert is procured by a centralized server, nodes use an API to pull active certificates at intervals
  3. one cert is procured by a centralized server, nodes use rsync via crontab to clone the central store

I have deployed a few other variations on these themes in the past, and I probably have a few more active ones too.

I would not pursue the multiple certificates strategy. It is possible, but you're going to overengineer what could be a simple solution, and are very likely to run into ratelimit issues that will ultimately hurt you if you need to triage something in an emergency.

3 Likes