Best practices for 20,000+ domain setup? (High density environment)


#1

Hello,
I would like to hear/share some ideas on using let’s encrypt within high-density environment.

I would like to leverage let’s encrypt to protect about 20,000 individual domains on my Nginx servers.

New domain gets added every day. Therefore, I don’t think keep re-generating my SNI certificate would be a good idea.

I am planning to request new individual cert for every new domain but I have some concerns in following areas:

  • Managing 20,000+ cert and renewal?: I am thinking to heavily use certbot here
  • Syncing certs between multiple web-servers: I have 10 nginx webserver and I am planning to use NFS here (or AWS EFS)
  • Creating nginx configuration for each domain: Looks like we cannot make use of nginx variables to dynamically define our certification settings. Seems like I need to create/generate 20,000 nginx different configuration. (this is going to be a management headache as well.) I am thinking to create a nginx template and then generate new ones based on that template.

What are your thoughts? recommendations?
Thanks!


#2

That’s the standard answer, yes. As an alternative, OpenResty has a Lua SSL module that supports dynamic certificate management, but i’ve never used it personally.


#3

Hi neturalizer

are the 20,000 domains sub domains of a tld?

Wildcard certificates (which lets encrypt don’t do) may be a better option here (and they do cost). The advantage of wilcard certs is that you spend less time validating and there is an option to have longer than 90 days.


#5

Hello @ahaw021
No, they are all unique domain. Therefore, I cannot make use of wildcard certs :frowning:


#6

hi neutralizer

the integration guide is a great resource https://letsencrypt.org/docs/integration-guide/

have a look at this as i think it may also help you out:


https://getcarina.com/docs/tutorials/nginx-with-lets-encrypt/

The second one is specific to Carina (a rackspace service) but some of the concepts are the same

for 20,000 domains you may want to have a dedicated (small) server to manage the certs
I would also think about staging the certs over time. For example if you issue all certs on a daily basis over month you would need to manage 667 certs per day every 3 months. If you spread this over 2 months it goes down to 333 and 3 months to 111.
Figure out if you are going to use one account key for all certs or different account keys depending on what the domain is
Figure out how you are going to manage failures
Figure out how you are going to manage validation (are you going to use DNS or HTTP challenges). I don’t know your setup but if you don’t have access to DNS then use the method in link 1 (re-write any challenge requests to your “cert management” server)
If you are able to group your domains by lets say types of customers or countries or some other way you may be able to reduce the number of certs you need to manage by using SAN certificates. Create a host for a group of lets say 20 domains and have one certificate.
You will definitely need a database and a web interface to manage all this :smiley:


#7

also i believe there is a limit of 100 validations per certificate


#8

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.