we are a very large university (ethz.ch) with all the de-centralized system-administration that comes with such an environment. At the moment we are constantly hitting the issue rate-limit (50+ up to 180+ new requests per week). After looking at the rate-limit adjustment documentation I still have some questions.
The rate-limit adjustments are linked to an account ID. That ID has an account-secret (key). This means we can't publish that info it inside our organization (10k employees and 40k students, published info is not a secret anymore). What happens if a certbot is asking for an ethz.ch certificate without the account ID that has the increased rate-limit approval? Will the request be denied? Will the request go on with the default rate-limits?
What strategy would you guys recommend for such an organization? How can we increase the rate-limit without being bound to an un-shareable secret?
Perhaps I might have a much simpler solution for you. Have you considered applying for ethz.ch to be added to the Public Suffix List? This would solve many if not all of your woes with the rate limits.
Let's Encrypt uses it for rate limiting applications to their CA. If you just need an exception from their rate limits, please do not request a change to the list, but instead use their form, linked from their documentation. This is a faster way to achieve what you want.
I would very much like to use the "official" LE way to increase the limit. However the questions in my original post are still open and just testing to see how the system behaves is not practical. It takes a day to reach the limit on a test-domain, weeks (according to the LE-webpage) to be approved then again a day to test again.
I was hoping to meet here people that know how the system behaves in my described cases. Any help is still very sincerely appreciated.
Btw I don't think you are limited to one account to request certificates under your domain.
I am using multiple accounts myself for multiple hosts under a single domain. They do NOT need a shared account.
Each department or even each individual that wants to request certificates under your domain can do so using their own account(s), as long as they can prove ownership of the hostname(s) in question, using a suitable ACME challenge.
You will still hit the rate limit, which is domain-based and not account-based, but for this you can get in touch with LE for large domains like yours, via the LE Rate Limit Adjustment Request Form.
(the typical hosting provider case is the other way around: one account, many domains, but your case: one domain, many accounts, is documented there as well.)
Add the domain to the PSL. This has it's own advantages and drawbacks.
Use a ratelimited account and proxy all requests through the main IT department. This could be automated with an online tool, or handle it through email requests. With the DNS-01 challenge (even easier when combined with an acme-dns server) a centralized account can procure all certificates for your domain.
Any account can be used to obtain certs from your domain, so a random student/department can quickly overtake your domain. I think there is a way to mitigate this with CAA and delegation, but I've never needed to do that so have not read much on it.
hmm...
Option #3 (i.e. not previously mentioned)
Have you considered Containerize your requests to subdomains.
This would give each subfolder (department within your organization) its' own account and key.
[although AFAIK restricting which LE account is authorized for which domain isn't a thing yet]
But, yes, this could still leave room for abuse/overuse; To the point that one department could consume the limit before another department could even get one cert.
It would, however, at least show you exactly which department did so.
From there, if needed, you could request rate limit increases on those departmental accounts.
[without having to share a single account]
So, the point is: Ask for rate limit increases on more than one account (which can be individually associated to specific subdomains, or not).
I would like to throw out the possibility of using wildcard certificates, especially at the second subdomain level. While I am not sure of the security implications in this situation, distribution of wildcard certificates (per a system like @jvanasco was discussing) would solve many of the logistical issues with the rate limits. PFS would prevent traffic from being compromised in case of key leaks.
As an example of a drawback: When our university (vu.nl) tried to do something with single sign-on they couldn't get cookies to share between subdomains, because some browsers assumed that a domain that short had to be a public suffix like co.uk. All SSO had to be routed through a subdomain to get it to work.
Thank you very much for your input. If we could extend the rate-limit with an account but not be bound to requesting with that account, that would be super!
In the meantime I asked the opinion of my web-developer colleagues, and going towards what @kjb wrote, not being able to have cookies on ethz.ch would break some applications.
We will request the rate-limit increase on one account. If we have troubles down the road we apply for rate-increases for subdomains with separate accounts.
