# Can ARI-conforming clients be granted exemptions to relevant rate limits?
**URL:** https://community.letsencrypt.org/t/can-ari-conforming-clients-be-granted-exemptions-to-relevant-rate-limits/195600
**Category:** Client dev
**Created:** [March 31, 2023, 6:22pm UTC](https://community.letsencrypt.org/t/can-ari-conforming-clients-be-granted-exemptions-to-relevant-rate-limits/195600 "2023-03-31T18:22:09Z")
**Posts on this page:** 20
**Page:** 2
### Author:  [@Osiris](https://community.letsencrypt.org/u/Osiris)
#### Post date: [March 31, 2023, 8:44pm UTC](https://community.letsencrypt.org/t/can-ari-conforming-clients-be-granted-exemptions-to-relevant-rate-limits/195600/21 "2023-03-31T20:44:11Z")
> [@mholt](#):
>
> Since the window doesn't actually matter after all
The window is paramount if ARI is used to smooth out loads on the ACME servers side?
---
### Author:  [@mholt](https://community.letsencrypt.org/u/mholt)
#### Post date: [March 31, 2023, 8:49pm UTC](https://community.letsencrypt.org/t/can-ari-conforming-clients-be-granted-exemptions-to-relevant-rate-limits/195600/22 "2023-03-31T20:49:43Z")
That's what rate limits are for, I guess. Since ARI isn't actually enforced.
> [@Nummer378](#):
>
> ARI is a _suggestion_.
> [@Nummer378](#):
>
> You can still utilize ARI to be notified of impeding revocations and perhaps for load avoidance
So that's my plan -- if the window changes, assume a revocation or some other sort of problem (or expected problem), so might as well renew right away to get ahead of it, or to have a longer period of time in which to ensure service.
I'm happy to alter my plan if the CA would like to ensure we can get the cert in that window (i.e. RL exemption). I'm just going based on what has been said in this thread.
---
### Author:  [@Osiris](https://community.letsencrypt.org/u/Osiris)
#### Post date: [March 31, 2023, 8:58pm UTC](https://community.letsencrypt.org/t/can-ari-conforming-clients-be-granted-exemptions-to-relevant-rate-limits/195600/23 "2023-03-31T20:58:19Z")
> [@mholt](#):
>
> if the window changes, assume a revocation or some other sort of problem (or expected problem), so might as well renew right away to get ahead of it, or to have a longer period of time in which to ensure service.
If the window changes, it might be just as simple as a shift in load on the ACME server. As far as I know, it doesn't mean impeding doom...
---
### Author:  [@Nummer378](https://community.letsencrypt.org/u/Nummer378)
#### Post date: [March 31, 2023, 9:19pm UTC](https://community.letsencrypt.org/t/can-ari-conforming-clients-be-granted-exemptions-to-relevant-rate-limits/195600/24 "2023-03-31T21:19:28Z")
> [@Osiris](#):
>
> If the window changes, it might be just as simple as a shift in load on the ACME server. As far as I know, it doesn't mean impeding doom...
Yes, this is true. But the window also won't change dramatically - a CA saying via ARI "renew now" when you still have lots of lifetime left is a sure indicator that _something_ is wrong, whatever that may be.
If I were in @mholt's situation I would probably consider something like this:
- If `current time` is before the `suggested renewal window start`: Renewal has low priority, but could be considered, if the client deems it necessary for whatever reason (i.e. spreading out renewals)
- If `current time` is inside the `suggested renewal window`: Renewal has medium priority. It should be done, if there is capacity for it.
- If `current time` is after the `suggested renewal window`: Renewal has highest priority. Renewal should be done at the next available opportunity.
This is not exactly conforming to ARI-specification, but it might work better in rate-limited scenarios described by @mholt. If this is still triggering rate limits/spread becomes excessive, a rate limit override\* is probably preferable.
\*A multi-CA fallback is of course another viable option
---
### Author:  [@mholt](https://community.letsencrypt.org/u/mholt)
#### Post date: [March 31, 2023, 9:34pm UTC](https://community.letsencrypt.org/t/can-ari-conforming-clients-be-granted-exemptions-to-relevant-rate-limits/195600/25 "2023-03-31T21:34:55Z")
Sure. But if they're expecting load, I want to get in before the load or risk not getting service and having to retry for a longer period of time. But there's only so long before the cert expires.
Might as well try sooner to ensure service.
---
### Author:  [@mholt](https://community.letsencrypt.org/u/mholt)
#### Post date: [March 31, 2023, 9:36pm UTC](https://community.letsencrypt.org/t/can-ari-conforming-clients-be-granted-exemptions-to-relevant-rate-limits/195600/26 "2023-03-31T21:36:36Z")
That's a better approach. But the problem with that algorithm is you don't know the future. So you want to avoid getting yourself into a situation where you can't renew in time.
And yes, we already do multi CA fallback ๐
---
### Author:  [@Osiris](https://community.letsencrypt.org/u/Osiris)
#### Post date: [March 31, 2023, 10:03pm UTC](https://community.letsencrypt.org/t/can-ari-conforming-clients-be-granted-exemptions-to-relevant-rate-limits/195600/27 "2023-03-31T22:03:04Z")
> [@Nummer378](#):
>
> - If `current time` is before the `suggested renewal window start`: Renewal has low priority, but could be considered, if the client deems it necessary for whatever reason (i.e. spreading out renewals)
False. The draft RFC is relatively clear about this scenario:
> Conforming clients MUST attempt renewal at a time of their choosing based on the suggested renewal window.
If I were a RFC writer, I would make it even less ambiguous, but I assume they mean that you MUST choose a time **WITHIN** the window. I'm not sure why they use the words "based on".
Subjectively, with the current wording, one could also reason that if the client always starts renewing 10 days before the suggested time window, it has chosen a time based on the suggested window and thus conforms to the "MUST" of the RFC. However, I'm preeeeeetty sure that's not what was envisioned.
> [@Nummer378](#):
>
> This is not exactly conforming to ARI-specification
I believe it's even forbidden by the ARI draft as specified above.
> [@mholt](#):
>
> I want to get in before the load or risk not getting service and having to retry for a longer period of time. But there's only so long before the cert expires.
>
> Might as well try sooner to ensure service.
If you renew early, you might increase load on the systems. The way I see it, your cert has gotten a "slot". And other certificates _also_ have "slots". Some after, some **before** your slot. So if you renew EARLY, you're messing up the entire "slotting" system of the ACMEs server ARI system.
---
### Author:  [@mholt](https://community.letsencrypt.org/u/mholt)
#### Post date: [March 31, 2023, 10:33pm UTC](https://community.letsencrypt.org/t/can-ari-conforming-clients-be-granted-exemptions-to-relevant-rate-limits/195600/28 "2023-03-31T22:33:58Z")
> [@Osiris](#):
>
> False. The draft RFC is relatively clear about this scenario:
You can't say that's "false" like that. You even say:
> [@Osiris](#):
>
> I would make it even less ambiguous, but I assume they mean
I think it's fair to say that we can make decisions that are outside the window.
> [@Osiris](#):
>
> The way I see it, your cert has gotten a "slot".
Only if we don't get blocked by rate limits and load, seeing as the window is optional anyway.
---
### Author:  [@mholt](https://community.letsencrypt.org/u/mholt)
#### Post date: [March 31, 2023, 10:37pm UTC](https://community.letsencrypt.org/t/can-ari-conforming-clients-be-granted-exemptions-to-relevant-rate-limits/195600/29 "2023-03-31T22:37:41Z")
> [@Osiris](#):
>
> I believe it's even forbidden by the ARI draft as specified above.
Where? The draft itself says the algorithm is a recommendation not a requirement.
(Sorry for spurious edits, I'm having trouble on mobile)
---
### Author:  [@Osiris](https://community.letsencrypt.org/u/Osiris)
#### Post date: [March 31, 2023, 10:49pm UTC](https://community.letsencrypt.org/t/can-ari-conforming-clients-be-granted-exemptions-to-relevant-rate-limits/195600/30 "2023-03-31T22:49:16Z")
> [@mholt](#):
>
> You can't say that's "false" like that. You even say:
Fair enough, I meant: "In my opinion that's false."
> [@mholt](#):
>
> seeing as the window is optional anyway
What's the purpose of the "MUST" in the (draft) RFC if it's optional? Right, it wouldn't say "MUST". I'm quite sure "optional" is not what the RFC editors meant.
> [@mholt](#):
>
> Where? The draft itself says the algorithm is a recommendation not a requirement.
Their suggested example algorithm is all **within** the window, not outside. So staying within the window is mandatory IMO. That's based on the usage of the "MUST" terminology before the example.
---
### Author:  [@orangepizza](https://community.letsencrypt.org/u/orangepizza)
#### Post date: [March 31, 2023, 11:20pm UTC](https://community.letsencrypt.org/t/can-ari-conforming-clients-be-granted-exemptions-to-relevant-rate-limits/195600/31 "2023-03-31T23:20:20Z")
looks like he doesn't willing to following draft as-is (thinks windows is too small to fit all the certificates he started), but still want to watch ARI to look out impending revocation event
---
### Author:  [@mholt](https://community.letsencrypt.org/u/mholt)
#### Post date: [March 31, 2023, 11:28pm UTC](https://community.letsencrypt.org/t/can-ari-conforming-clients-be-granted-exemptions-to-relevant-rate-limits/195600/32 "2023-03-31T23:28:01Z")
> [@Osiris](#):
>
> What's the purpose of the "MUST" in the (draft) RFC if it's optional? Right, it wouldn't say "MUST". I'm quite sure "optional" is not what the RFC editors meant.
Exact wording:
> Conforming clients MUST attempt renewal at a time of their choosing
> based on the suggested renewal window.
So, fair enough. I will choose a time based on the suggested window as it states I "MUST" do. Not necessarily within, since I cannot be guaranteed to get the cert within that window. But the spec does not say within.
> [@orangepizza](#):
>
> (thinks windows is too small to fit all the certificates he started), but still want to watch ARI to look out impending revocation event
Exactly. As-is, ARI seems to be a great early renewal canary, so I will use it as such; but the precise window will be untenable for large deployments unless the CA is willing to guarantee I can get the cert _within_ that window.
* * *
I'll try to spell this out as crystal-clear as I can. Try to keep up:
- Large deployments may have more certificates to renew per day than LE rate limits will allow (I mean any/all rate limits combined, but I will yet _again_ emphasize the "New Orders" rate limit since I mentioned it on my very first post but some readers clearly did not see that). And yes we have real, actual production experience with this. None of this "theoretical" stuff mentioned above by those who admit they do not have experience with it.
- Current behavior is to spread out renewals over more time so that we can renew all of them before they expire. This means starting some earlier than 30 days out.
- Implementing ARI implies a narrower window in which to renew certificates, ironically in an effort to spread out the load more. ๐คทโโ๏ธ This is problematic for a few reasons:
- The ARI spec recommends an algorithm that renews within the window, but CA policies such as rate limits can prevent certs from being renewed in that window. Fortunately, the spec's language states that renewal scheduling decisions "MUST" at least be "based on" the "suggested" window. Since we cannot be assured a new cert within that window, we can at least use ARI as an early warning signal for upcoming or predicted problems: revocation, availability, maintenance, etc. Now, let's try to figure out a best course of action based on possible window changes for a hypothetical certificate we just obtained:
Having large-scale production experience with lots of certs and LE rate limits, the thing that makes the most sense for me to do with ARI is to use it as an early warning signal. If the window changes, there is clearly some doubt as to system availability or certificate validity. Thus start trying to renew the cert right away so the graceful backoff has as much chance to procure a certificate before expiration (or revocation โ ) This way if we hit rate limits, it will backoff and try again later, without running out of time because we started early enough.
Now... if this part I said:
> Since we cannot be assured a new cert within that window
could be changed -- i.e. if we COULD in fact be assured a cert within that window -- then everything gets better. If that window basically meant, "You can for sure have the cert anytime within this window. (But just once.)" then I would absolutely have no problem renewing all certificates precisely within that window.
This means the client must be assured that no rate limits would block them from getting a cert within the window.
If we cannot be assured that, we have to do our best to get a cert on our own, which means starting renewals earlier.
Hence, if we want ARI to be useful, I am requesting a rate limit exemption for clients adhering to the ARI window.
---
### Author:  [@Osiris](https://community.letsencrypt.org/u/Osiris)
#### Post date: [March 31, 2023, 11:37pm UTC](https://community.letsencrypt.org/t/can-ari-conforming-clients-be-granted-exemptions-to-relevant-rate-limits/195600/33 "2023-03-31T23:37:36Z")
> [@mholt](#):
>
> Not necessarily within, since I cannot be guaranteed to get the cert within that window. But the spec does not say within.
As I've already stated earlier, the draft doesn't literally state that indeed, but I believe we must assume that's what was intended. Hopefully a new revision of the draft can make this more clear. (@aarongable)
> [@mholt](#):
>
> but the precise window will be untenable for large deployments unless the CA is willing to guarantee I can get the cert _within_ that window.
Even if the window is _ **DAYS** _ long? What the...???
---
### Author:  [@jsha](https://community.letsencrypt.org/u/jsha)
#### Post date: [April 1, 2023, 12:33am UTC](https://community.letsencrypt.org/t/can-ari-conforming-clients-be-granted-exemptions-to-relevant-rate-limits/195600/34 "2023-04-01T00:33:35Z")
Thanks for bringing this up, @mholt! I think the question of how to spread load during a mass revocation event is key to ARI. I'm not sure what the exact answer should be here, but to this point:
> [@mholt](#):
>
> - Ignore ARI and start renewing certificates as early as needed to be able to spread them out enough.
> - Get a guarantee from the CA that the first successful cert for a name within the ARI window will be allowed regardless of relevant rate limits (new orders for example).
We were actually just discussing how the Renewal Exemption should probably apply to the New Orders rate limit, regardless of whether a mass revocation is underway. It currently only applies to the Certificates per Registered Domain limit. But the same rationale applies to the New Orders limit: in general, if you already have a certificate, we want to prioritize allowing that certificate to be renewed.
For other rate limits, it has been the case in the past and we expect it to be the case in the future, that when we expect mass reissuance, we will manually raise various rate limits to make sure issuance can happen on time.
---
### Author:  [@mholt](https://community.letsencrypt.org/u/mholt)
#### Post date: [April 1, 2023, 12:46am UTC](https://community.letsencrypt.org/t/can-ari-conforming-clients-be-granted-exemptions-to-relevant-rate-limits/195600/35 "2023-04-01T00:46:50Z")
Hi Jacob, thanks for the reply! (I just finished adding very detailed thoughts to my most recent post above, and for some reason the forum didn't show your post until after I submitted my edits.)
> [@jsha](#):
>
> We were actually just discussing how the Renewal Exemption should probably apply to the New Orders rate limit, regardless of whether a mass revocation is underway. It currently only applies to the Certificates per Registered Domain limit. But the same rationale applies to the New Orders limit: in general, if you already have a certificate, we want to prioritize allowing that certificate to be renewed.
That would be wonderful, and on behalf of many many site owners I thank you for considering this ๐
I was getting a little frustrated by others saying that a problem doesn't even exist -- from people who admit to not having production experience. :-/
Because yeah, like I just said in the edit to my post above (I didn't see any other replies while I was editing... maybe I should have just made it a new post), if I can absolutely trust that we can get the cert in that window, I will strive to honor that window in the implementation.
It would be good for the spec to spell this out explicitly IMO -- for other CAs to follow -- as this simply will not work if clients can't be guaranteed a new certificate within the ARI window.
PS. You won't need to completely lift the rate limits, but just enough for 1 successful issuance of that cert during the window is all that should be needed. So you can continue to count additional ones against rate limits.
---
### Author:  [@WouterTinus](https://community.letsencrypt.org/u/WouterTinus)
#### Post date: [April 2, 2023, 12:39pm UTC](https://community.letsencrypt.org/t/can-ari-conforming-clients-be-granted-exemptions-to-relevant-rate-limits/195600/36 "2023-04-02T12:39:05Z")
Just to chime in here, for my client win-acme (of which I'm not only the developer, put also a heavy user) - I also intend to follow the logic that renewals can happen _earlier_ due to ARI suggestions, but I won't make them happen _later_. This is because my client has implemented several guardrails and a user-configurable random window, to spread load, but more importantly to prevent it from hitting rate limits, that I don't want to break for the sake of following the spec to the letter.
I would consider adding a "strict" ARI mode if there could be some guarantees like the ones being discussed here.
---
### Author:  [@jvanasco](https://community.letsencrypt.org/u/jvanasco)
#### Post date: [April 4, 2023, 4:28pm UTC](https://community.letsencrypt.org/t/can-ari-conforming-clients-be-granted-exemptions-to-relevant-rate-limits/195600/37 "2023-04-04T16:28:34Z")
> [@mholt](#):
>
> Hence, if we want ARI to be useful, I am requesting a rate limit exemption for clients adhering to the ARI window.
First, I 100% get everything that @mholt has posted above and agree with him. I honestly don't understand the pushback and criticism from some people above. When you are dealing with managing certificates at scale, you are already floating dangerously close to hitting rate limits - the result of the ARI query can easily create issues.
My main concern as I look into ARI support is the timing and load of the ARI query itself. Going back to the original comment and the situation with Certify the Web โ concerns over rate limits and load are a bit crippling to large installations, as that can require tens of thousands of checks per day.
The concern I have for @mholt's exception request, is that client identification does not seem like a smart path as it could easily be forged/spoofed. It would make more sense to me for this to be account based, and perhaps there is some way to automate that - perhaps by an endpoint/service that allows a user to register their usage of a specific client to their account and request an exception. Basically, this would be a way to automate the current rate limit request form - using a client to send all the required information in a structured format to ISRG, who then gets it in Boulder and can then programmatically or manually review the request with full access to the account's history right there. That's my 2ยข on how I'd handle this.
---
### Author:  [@WouterTinus](https://community.letsencrypt.org/u/WouterTinus)
#### Post date: [April 4, 2023, 6:02pm UTC](https://community.letsencrypt.org/t/can-ari-conforming-clients-be-granted-exemptions-to-relevant-rate-limits/195600/38 "2023-04-04T18:02:23Z")
Agreed that any excemption shouldn't be granted for a specific client, but I also don't think that the solution needs to be quite so complicated.
Rather I feel the logic should be that:
- If a new order is created
- And the account creating the order has an existing certificate with the same domains
- And that certificate is currently within its ARI suggested renewal window
- And this is the first order that matches those conditions
- Then that specific order should not count towards the rate limit.
Alternatively (if this would entail too many database lookups, worsening load issues), the ARI GET-request might return some server signed, once-usable token (JWT?) which could be submitted along with the order to allow it to bypass the limit.
---
### Author:  [@mholt](https://community.letsencrypt.org/u/mholt)
#### Post date: [April 4, 2023, 6:06pm UTC](https://community.letsencrypt.org/t/can-ari-conforming-clients-be-granted-exemptions-to-relevant-rate-limits/195600/39 "2023-04-04T18:06:45Z")
Hey Jonathan, thanks for chiming in.
> [@jvanasco](#):
>
> The concern I have for @mholt's exception request, is that client identification does not seem like a smart path as it could easily be forged/spoofed. It would make more sense to me for this to be account based, and perhaps there is some way to automate that - perhaps by an endpoint/service that allows a user to register their usage of a specific client to their account and request an exception. Basically, this would be a way to automate the current rate limit request form - using a client to send all the required information in a structured format to ISRG, who then gets it in Boulder and can then programmatically or manually review the request with full access to the account's history right there. That's my 2ยข on how I'd handle this.
I'm not totally sure how client ID can be forged/spoofed, since ACME requests are authenticated with account private keys. (Maybe I'm misunderstanding though.)
To clarify, what I'm suggesting requires no extra endpoint or infrastructure. Before enforcing a rate limit during an ACME certificate issuance, the CA server does the following checks:
- Is the certificate being obtained within the ARI window we're advertising?
- Is the authenticated account on this request the same as the account for the certificate this is replacing (most recent with same set of SANs)?
- Is this the first time the certificate is being renewed since advertising the ARI window?
If the answer to all those questions is **yes** , rate limits should be effectively ignored, and the client should be allowed to get a certificate right away.
---
### Author:  [@jvanasco](https://community.letsencrypt.org/u/jvanasco)
#### Post date: [April 4, 2023, 6:32pm UTC](https://community.letsencrypt.org/t/can-ari-conforming-clients-be-granted-exemptions-to-relevant-rate-limits/195600/40 "2023-04-04T18:32:23Z")
> [@mholt](#):
>
> I'm not totally sure how client ID can be forged/spoofed, since ACME requests are authenticated with account private keys. (Maybe I'm misunderstanding though.)
This is the disconnect:
- You are talking about Account/Subscriber ID.
- I am talking about an identified Client Application, such as Certify The Web, Caddy, or CertMagic.
Having worked on large APIs with multiple clients that each support larger numbers of subscribers, after reading your post I immediately started to think of how ISRG could best identify all users of version X of Caddy and extend the increased rate limits to them. That would have the benefits of both supporting conforming clients, and also encourage other client developers to become conforming clients.
This can't rely on user-agent headers or similar, because that can easily be spoofed by lesser clients. A potential workaround is to allow subscribers to post to an endpoint notifying ISRG that a subscriber (AccountKey) is using a conforming client and requests a ratelimit exemption. If posted to Boulder, ISRG could potentially review the request programmatically - taking into account the number of domains/certificates on the account, past ARI activity, and other relevant metrics โ which could remove the manual review/decision process for these rate-limit exemption requests. This could also be designed in an extensible way that allows for any account-based rate limits exemptions to be submitted via API for programmatic or a streamlined manual review.
[Previous page](https://community.letsencrypt.org/t/can-ari-conforming-clients-be-granted-exemptions-to-relevant-rate-limits/195600.md?page=1)
[Next page](https://community.letsencrypt.org/t/can-ari-conforming-clients-be-granted-exemptions-to-relevant-rate-limits/195600.md?page=3)