There is time we hit the rate limit and we block for all week.
This is not something a company can work with, companies need the option to pay extra for enterprise solutions.
Hi @noamway,
Let's Encrypt is a not-for-profit organization and apart from any other considerations would probably spend more time and money creating a payment infrastructure for exceptional cases than it would receive that way.
The rate limits are documented at
so they don't have to come as a surprise (although I would still advocate an earlier proposal that there should be shorter limits that apply earlier, in addition to the current limits, so that people become clearly aware of them faster).
Currently there are several other certificate authorities that offer certificates using the ACME protocol.
It also seems that Digicert and Sectigo have ACME services available for paying certificate customers.
All of these ACME services should be compatible with the same client software that Let's Encrypt works with because ACME is meant to be an interoperable technology, not tied to any one certificate provider. So, if you need to get a certificate right away or want to pay someone for a different issuance strategy, there should be several options from other CAs that can work for you.
I don't mean to dismiss your suggestion; I just want to emphasize that the switch from "never billing any user for anything" to "sometimes billing some users" is a huge jump in overhead for an organization, as entities that collect payments for services have to deal with disputes, chargebacks, fraud, PCI compliance, engineering for payment flow, etc. Obviously most online services that primarily have a fee-for-service model will bear those costs in order to be able to receive revenue, but for an organization that doesn't primarily have a fee-for-service model, that would be a huge shift. Finally, Let's Encrypt would need to figure out how to incorporate payments for services into its tax-exempt nonprofit status, which can also be legally complicated.
So, I think Let's Encrypt is very likely to continue to leave paid services to other certificate authorities, which, thanks to the ACME protocol, ought to be fairly interoperable with software environments that already use the Let's Encrypt API to get certificates.
Thanks @schoen
In the last Let's Encrypt revoke you could see how the "internet" becomes dependent on Let's Encrypt solution.
Millions of websites go down because people didn't understand what they need to do, and when they try to do something they block for a week because of that, and their website goes down.
I think Let's Encrypt becomes so big and important for the internet that they also need to start thinking about that situation. If millions of websites go down because of the rate limit, I think there is a problem that needs to be solved.
And for the payment solution part, there is so many "plug-and-play" solution these days that I don't think it will change the roadmap of the project:
- Simple form.
- Enter your accountID.
- Enter the package of rate limit you need.
- Pay with Stripe or Paddle (or any other solution that takes care of payment and invoice and tax).
- Option to cancel or change package - from the payment provider.
- In a webhook, Let's Encrypt update the rate for the accountID.
If there was a solution for that, I don't care to pay $1000 each month. It's 12K for a year that can improve the open-source project.
Which rate limit exactly are you talking about?
During the recent incident, when people were hitting the max. order per 3 hours limit (300 by default), Let's Encrypt temporarily increased that specific rate limit to 1000 max. orders per 3 hours. That should be enough to renew a lot of certificates! That's 8000 certificates per day!
When other rate limits are hit, it's usually due to misconfigured ACME clients or something else in the same category. You can't blame Let's Encrypt for that. Sometimes it's just a sysop with too little knowledge of the ACME system, who doesn't know what he/she/them's doing. You also can't blame Let's Encrypt for that I think.
Sorry, my English is not so good so sorry if it's like I blame Let's Encrypt
I have a project with 100K domains so I hit the limit very fast, and I'm using Caddy to take care of everything so I don't deal with that.
Anyway, I'm not blaming Let's Encrypt, I'm saying that the internet needs to be LIVE 24/7 and all the SSL projects make things very difficult to manage because of that rate limit. Not all developers have the same knowledge in the SSL field so things need to be much easier so the internet will not be broken.
The reality, there is not a lot of good commercial solution in the ACME field because of the Let's Encrypt free solution (no one wants to compete with free alternative), so I think Let's Encrypt need to understand it and offer a better solution so the internet will not go down.
Increase limit from 300 to 1000 is just a plaster when you don't know how different customer is affected.
I hope I explain my situation better now (and again sorry for the English)
I actually agree that it's not as hard to enable paid enhancements as it would first appear, many one-person companies scale to thousands of paying customers using basic integrations with Stripe etc.
Peace of mind that rate limits are not going to apply to your account would actually be quite nice for some larger organisations. While I understand it's important to provide free certificates, steadfastly refusing to accept money for any services is self-limiting.
For example, I know users would pay a subscription for any of:
- Rate limits being lifted or removed
- Longer lived certs
- A dashboard of all certificates managed by the account and their status.
- Extended support
I think it's great that LE exists via donations/sponsorship alone but being non-profit does not preclude an organisation from earning money, developing more services and paying more staff.
Long term, of all the current ACME CAs I'd expect ZeroSSL will eventually go this route because their parent organisation is very much a for-profit.
This can already be done for free with the rate limit exception form, LE's goal is not to prevent large organizations from using their certificates, but to make sure their resources are being used in a responsible way. Additionally unique domain registrations are not subject to the 50 certificates per registered domain limit.
This would be contrary to all the work LE and the broader security community has done to reduce the lifespan of certificates and encourage automation. Getting certificates down to 1 year from 10 years has already been a huge battle. I personally think the maximum life per the baseline requirements should be 180 days
I thought about mentioning this form, but I don't think it's useful in the case of mass revocation and consequent renewals. It takes quite some time for the form to be processed (weeks) and I don't know if LE is willing to agree with rate limit exceptions just for future incidents.
To add on to what others have already said along these lines, many companies do pay extra for enterprise solutions. They are just using commercial CAs, which offer them support and SLAs and a person you can call when you have problems, rather than using Let's Encrypt which works really hard but at its core is a non-profit with a really small staff and relying on random people on the Internet here to handle the vast majority of support. If "post on a forum" and "maintain a system myself within rate limits" isn't good enough for your business, then there's nothing wrong with paying someone for those extra services, and as much as I appreciate Let's Encrypt I think even they would agree. Sometimes in life, one gets what one pays for.
On the contrary, ACME is becoming more and more popular, and @schoen above linked to several CAs that provide it. I think there is still plenty of room for competition. It's just that they're competing based on service and those "enterprise features" (like having an easy dashboard of all your certs and support), not really competing on, like, "who does a better job validating domain names" or something like that.
I think in this case it would be good for LE to have a written policy regarding what will be done in the event of mass revocations. Such as what time frame and what sort of rate limit exceptions could be expected.
You can simply ask for a higher rate limit, there is no need to pay:
This is pointed out in the rate limit documentation.
This is probably getting to be a different topic, but there probably is some good thinking here. I'm sure Let's Encrypt has put a lot of thought into how to handle mass-revocations, as they had a blog post last year about the infrastructure improvements they had done to ensure that their servers could handle the capacity of re-issuing all their certs quickly. And they're working on extensions to ACME to allow for clients to more easily automatically detect that their certificate was revoked for a policy issue and to automatically renew early. But in the meantime, and even once those extensions are in place, it might be good to have clear direction about exactly what actions and notifications will happen, and some possible improvements to them (for instance, it'd be nice if one could opt-out of renewal notice email without also opting-out for revocation notice emails), and being clear on what actions will take place (such as temporarily higher rate limits) for what period of time might help everybody plan their incident response.
Further to @petercooperjr's suggestion, maybe some users would like something in between renewal reminder e-mails and subscribing to API Announcements. Like "notices if a typical small integration is likely to experience disruptions" or something.
Edit: for a broad notion of "typical", because using TLS-ALPN-01 validations at all isn't typical, but it isn't uncommon.
You can subscribe to https://letsencrypt.status.io/
Although I'm not sure if you can filter based on the type of status message. Boulder updates aren't very interesting for example.
Also, status.io offers different type of plans with different amount of max. subscribers..
Yeah, I think a big source of confusion with the recent incident has been that people don't necessarily know which validation method they use, especially with stuff like caddy that just does it all for you and clients built into systems (like, routers/firewalls/appliance type stuff) where it can just get its own cert and the user doesn't feel like they're "administrating a web server".
Of course, the next incident may have nothing to do with validation method, and be based on something else that the subscribers wouldn't necessarily know about. I'm eager for those ACME when-should-you-renew extensions to get implemented and used by every client so just people don't need to worry about it at all.
I just hope this incident makes some clients check not only expiration but revocation status as well, when deciding if a certificate needs renewing.
Certbot had planned that since the beginning, but only added it after a prior revocation incident.
Unfortunately, implementing OCSP checks is kind of a lot of work
on the scale of an ACME client, which is why the ACME-internal revocation status check feature would be really nice and probably get a lot wider deployment.
The other advantage to the ACME-internal status check feature, of course, is that it could tell a client to prematurely-renew during that brief window between when the CA says they're going to need to revoke your cert, and when they actually revoke it.
Yeah, that would definitely be another advantage, since OCSP doesn't have a "valid ... but not for long!" status.
As far as I know, that is indeed the intention of the Automated Certificate Management Environment (ACME) Renewal Information (ARI) Extension.