Cert failed to renew - authorizations expired?

Hi, I currently auto-renew all my LE certs every 30 days (I figured monthly was good enough).
They were all originally authorized by DNS challenge, afterwards the TXT records were removed to keep the zone clean.
Originally I had maybe 80% succeed and 20% fail during the renewal process, I would just manually have the failed ones get authorized/signed and move on.
Now for the first time i have seen a 100% failure to renew now as authorization expired across all the certs in all the domains.

Do I need to renew the certs more often? Such as every 7 days instead of every 30?
If I auto-renew every week wouldn’t that increase signing load on the CA unnecessarily? 30 days to renew seemed reasonable given the 90 day life of the cert.
Is there a limit to how many certs for a TLD can be renewed per time-frame?

Thanks.

Hi @metaclassing,

The lifetime of a valid authorization has been reduced from 60 to 30 days, which might be producing the behavior that you’re seeing. (If you renew very slightly after 30 days, the authorization may already be expired.)

But it’s also worth remembering that re-using an authorization doesn’t extend its lifetime, so you’ll have to have some way to re-validate your control of the domain names every 30 days, one way or another.

Hi @schoen
I guess im confused by the logic around rate limiting, my understanding was that renewals were exempt from the new-cert per-domain-per-week rate limit.
But it seems the only way to re-authorize is to go through the full signing process again, which counts against the new-cert-per-domain-per-week rate limit.
Is there something im missing here? How can I re-authorize and renew rather than fresh authorize and sign new?

Hi @metaclassing,

I’m not sure if there are any mainstream clients that will let you obtain authorizations without going on to use those authorizations to request a certificate, but this is a logically possible thing to do.

Most of the rate limits are calculated based on certificate issuance, which is the most expensive thing for the CA because it uses the hardware security module (HSM) and also then requires the HSM to start signing OCSP updates confirming that the certificate is unrevoked—regardless of whether that certificate is ever used or superseded.

For rate limit purposes, “renewals” are certificates that apply to the same domain name, regardless of whether the same or different authorizations at the ACME protocol level (or even the same ACME account) were used to obtain those certificates. So, you won’t do better or worse with respect to that limit by a particular strategy about how you structure accounts or authorization objects.

Thanks for that insight @schoen

I just read my client’s code and realized that it is calling the same uri ‘/acme/new-cert’ for both new cert AND renewal, the only difference being it wasn’t re-attempting authorization for renewals as it assumed they would still be authorized as an existing identical certificate was already issued. It sounds like the two things (new/renew rate limit and authorization logic) are de-coupled.

I may have to go read more about Boulder rate limit logic for that new-cert path to try and understand the scenarios that tick up the rate limiter for a given domain. If there is an existing valid cert with identical subjects and NEW authorizations, will it ding the per-domain rate limit or skip as a “renewal”

It would be very handy to have a way to figure out what a given domains rate-limit-counter is currently at without having to get rejected. Is there such an API I can call to find that information about an account or domain?

Thanks!

(update based on some reading)
I think my understanding of the rate limiter can now be described much more accurately:
The rate-limit error is skipped IF there is a record in the Boulder database for the SHA256 hash of lower-case comma-delimited (deduplicated AND sorted*) list of subjects for the new-cert request.
Example: SHA256(domain1.com,www.domain1.com,mail.domain1.com) = ec1797ac7c93290825ae72b35d65c5f9b865851d42849c1bebd9195c99ce52dc
IF ec1797ac7c93290825ae72b35d65c5f9b865851d42849c1bebd9195c99ce52dc is in the DB, then the checkCertificatesPerNameLimit function returns nil.
Additionally ra.certsForDomainStats.Inc(“FQDNSetBypass”, 1) is called, I will have to read more to try and understand what that is used for…

Hi @metaclassing,

Everything is meant to be described at

https://letsencrypt.org/docs/rate-limits/

If there’s something there that you don’t understand or that you find ambiguous, let’s get it fixed!

Thanks for the link, now that I read how the exemption code actually works some of the statements in the documentation are more clear. For example: “To get the maximum possible number of certificates, you must perform all new issuances before renewals during a given time window.” Why? Well, its because there is functionally no difference in the client between issuing a cert for subject set a,b,c the first time vs “renew” a month later. They both call the same API endpoint with the same information in Boulder and the only way the CA knows if its a “renewal” or not is the SHA256 alphabetized comma delimited subject hash value is in the database. To do this work on a large scale it would take some planning to ensure your auto-renewal process doesn’t eat your cert limit for a given time-frame blocking new issuances.

One outstanding question I have is with regard to multi-domain certificates. If I register a cert for domain1.com,domain2,net,domain3.org does the per-domain limit tick up for EACH of these 3? So one cert would ding each domain in it one time? Or does the limit only tick up for say the first domain?

To me it would make more sense to simply have two different API endpoints with different logic paths for new-cert and “renew-cert”. That way order of operation could be negated with regard to issue then renew logic and each path having different checks put in place. Returning rate limit metadata via additional response information (header?) or even a dedicated API endpoint for the client to consume would be very beneficial to client developers and users that run into the rate limit.

It dings each domain once. “domain1.com, www.domain1.com, domain2.net” would tick domain1.com once and domain2.net once, for example.

Edit: If you change your certificates frequently, it may make sense to use one separate certificate for each domain. You’d have more certificates but it’s actually easier on the rate limits.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.