New Orders per Account rate limit reasoning

[mholt]

The last rate limit is described as being:

For users of the ACME v2 API you can create a maximum of 300 New Orders per account per 3 hours.

There are ~2,160 hours in a 90-day period.

90-day certs should generally be renewed after about 60 days, or ~1,440 hours.

In a 1,440 hour span, there are 480 non-overlapping 3-hour-long intervals.

That means that a single ACME account's effective capacity is 144,000 active certificates.

That assumes that a certificate is needed exactly once every 36 seconds (0.6 minutes). Any variance in that pattern will deplete the burst size and cause a backlog.

I'm improving the internal rate limiting in CertMagic and was wondering if there are specific reasons for choosing 300 per 3 hours instead of, say, 1 every 36 seconds or 2 every 72 seconds, etc. We have users with tens of thousands of certificates and the feedback cycle is quicker when the window smaller. Sure, right now we can get a batch of 300 certificates pretty quickly, but then we have to wait 3 hours before we can process any more. It's more comforting to see a couple certs per minute than it is to see 300 every 3 hours.

(FWIW, I'm changing CertMagic's internal throttle from 300 every 3 hours to 2 every 72 seconds. But, ultimately, LE's servers are the authority, and we only implement throttling internally to be nice.)

(Apologies if this has been asked before. Searching wasn't yielding anything helpful this time.)

Edit: Bonus question! What qualifies as a "new order"? If a validation fails, does that create a new order? Or is a new order only for validated names where a certificate is issued? Similarly, if the client is rate-limited, is that attempt to process an order counted as a "new order"? I guess I'm wondering: should I only count against my internal rate limiter if the validation succeeded and a cert was issued?

[mnordhoff]

@mholt

What qualifies as a “new order”? If a validation fails, does that create a new order? Or is a new order only for validated names where a certificate is issued?

I believe just POSTing to /acme/new-order (edit: with a successful result) is what counts as a new order. And you have to do that before trying to validate, so it counts whether validation subsequently succeeds or fails (or whether your computer crashes and you don't proceed at all).

@mholt

Similarly, if the client is rate-limited, is that attempt to process an order counted as a “new order”?

I believe only successfully created orders count. The code just does an SQL query of the orders table. It would be counterproductive to put so much stuff in the database when creating a new order has been blocked by a rate limit.

@mholt

I guess I’m wondering: should I only count against my internal rate limiter if the validation succeeded and a cert was issued?

I believe you need to count successful responses from /acme/new-order.

(It's probably possible for Boulder to create an order but return an error to you if the web server crashes at just the wrong moment, or something, but that kind of thing should almost never happen...)

[mholt]

Thanks for the info.

Any idea if a rate limit with a smaller window would be desirable?

[mholt]

Well, regardless, I’ve decided after working with these rate limits on larger deployments that it’s not the client’s job to enforce the server’s rate limits: it’s practically impossible to do correctly, as our internal state doesn’t match the CA’s authoritative state, and the only way to know is to try. So I’ve adjusted our internal rate limit philosophy to “don’t firehose the CA’s servers when there are millions of domains to get certificates for” and I think that, along with its exponential backoff, will have to suffice.