I’ve recently implemented and integrated the certbot into our Kubernetes cluster management via dockerization of the official certbot Docker image. Due to the ephemeral nature of the docker containers, I am facing a architecture decision of either
building this process of always request a new LE certificate or
registering it once first and then renewal before expiration (how soon before expiration to renew is another topic).
Having said that, I wonder if LetsEncrypt project has a certificate issuance policy guideline regarding this practice. Obviously, option 1 is going to put more load on the LE ACME server when the community gets bigger. But I have yet to see any best practice guideline to specifically forbid option 1. And it’s much much easier to implement option 1 and don’t have to worry about the certificate transitional states.
So my question is, if there is indeed a guideline about which approach to take, can someone enlighten me please? Also, will there be a strict ban on option 1 in the future if none exist now? Furthermore, how soon should I renew existing certificates before expiration (certbot doc mentions about 1 month before expiration at the time of exectution)?
You have to consider the rate limits. The ones that are relevant are the 5 certs per week for the exact set of names or 20 per week for a single domain (public suffix + 1 label).
Certainly! Rate limit is a good factor to consider when always requesting new certificates at each run. But rate limits aside (assuming all requests are well within the rate limits), is there any other guidelines prevent / forbid from users taking option 1?
The rate limits are what the CA server will enforce, nothing else. The rest is up to you, though from Let’s Encrypt side, I’m sure a solution that results in fewer issuance operations is preferable.
If I understand option 1 correctly, you plan is to not store the certificate persistently and just request them whenever e.g. a container (re)starts. From an availability point of view, this will limit you to 5 container spawns/restarts per week before you lose the ability to serve TLS traffic. That’d be something I’d want to think about, things always break in unexpected ways. Then again, I’m not exactly familiar with Kubernetes.
Most configuration management software has the ability to persistently store long-term credentials/keys/etc., I’m sure that’s an option with Kubernetes. There are also a number of dedicated clients for Kubernetes that might be of use here, such as kube-lego or kube-cert-manager.
As far as the option 1 goes, my approach is that the certificates + private keys are stored into Kubernetes secrets for persistent storage, however, everthing else (account meta data, CSR, LE logs, renewal configs) are NOT persistently stored. For instance, as seen below, just fullchain.pem and privkey.pem files in /etc/letsencrypt/live/www.example.com/ folder are persistently stored, no other files in the /etc/letsencrypt will be persisted. Then, there will be a process and triggers this certificate request process at a rate less than what LE’s current limit are.
And yes, there are many other ways to persist the whole LE data set in the entire folder. but for a large node cluster, distributing a R/W-able data set is definitely more work and complex than simply cherry-picking the data that’s absolutely necessary to persist, but not impossible.
My initial question was merely to inquire about the service boundaries/guidelines so that I can balance our system architecture’s simplicity vs. not triggering LE’s abuse complain.
Lastly, I am assuming other than the rate limit, there isn’t really any guideline in terms of when to renewal correct? I noticed when all data are preserved in place, certbot will automatically make a decision on either do nothing (exipiration is greater than 1 months away) or renewal (expiration is one month away or less).
Thanks for the clarification. I’d say a new registration every time you renew is not a big deal at all and wouldn’t get you anywhere close to any behaviour that would be considered abusive. Registrations are relatively cheap (especially compared to issuance operations, which are limited by the HSM signing capacity) and with the limit being 500 per 3 hours per IP … well, let’s say one per 60-90 days won’t bother anyone.
I don’t see any other huge downsides to not persistently storing the account key in this scenario, the only scenario where you might need the account key would be revocation, but you can do that using the certificate private key as well.
Renewal one month prior to the certificate expiring seems like a good option (with plenty of time to get things fixed in case something goes wrong anywhere), but that’s really just a soft recommendation, if you have a good reason to do something else, I don’t see why that would be a problem. The only other best practice here would be to pick a random time for renewal in order to avoid everyone’s cronjob triggering at midnight UTC or something like that, which would essentially DDoS Let’s Encrypt.
