SAN cert limitations on number of hosts

[casterln]

I have a huge SAN (293 hosts) currently issued from sectigo. I'd like to use acme and Let's Encrypt on apache with mod_md. I'm hitting a 100 DNS name limit. Our SAN sites all have CNAMEs pointing to our main domain. Is there a way to break things up to get under the 100 host limit? Thanks. _Gary

My domain is: nature.berkeley.edu

I ran this command:
with apache mod_md added MDomain nature.berkeley.edu

It produced this output:
Invalid argument: acme problem urn:ietf:params:acme:error:malformed: Error creating new order :: Order cannot contain more than 100 DNS names

My web server is (include version):
apache 2.4.34

The operating system my web server runs on is (include version):
RHEL 7.9
My hosting provider, if applicable, is:

I can login to a root shell on my machine (yes or no, or I don't know): yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):
not applicable

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):

[Osiris]

Looking at the mod_md documentation, I'm not sure if the module is capable of setting a limit on the number of SAN.

I don't have any experience with mod_md, but I can imagine the module issues a single cert per virtualhost. Do you have 293 hosts in a single virtualhost by any chance? Maybe you can split it up into multiple virtualhost sections?

[casterln]

Each SAN host has it's own VirtualHost configuration. A Servername set to nature.berkeley.edu and ServerAlias set to the SAN host (which has the CNAME pointing to nature.berkeley.edu). It appears that all these are presented to Let's Encrypt as one big request. I may have to keep doing things the way I have been. Would nginx have more capability in this regard?

[Osiris]

I'm not sure if nginx even has a module like mod_md. Chances are you'd need a separate ACME client for nginx. And then it wouldn't make much sense to switch to nginx, as you could also use that ACME client with Apache.

[MikeMcQ]

Did you specify all the hostnames in the same MDomain statement?

Because you get one cert for each MDomain statement as I understand it.

You could have one MDomain statement for each HTTPS VirtualHost which names a hostname in its ServerName or ServerAlias (mod_md automatically groups all names in the same VirtualHost so you only need to list one in the MDomain). Or, group them in other ways but this might be easiest.

See here for an explanation of how the names are grouped. I am guessing you listed all your names in one managed group.

GitHub - icing/mod_md: Let's Encrypt (ACME) in Apache httpd

And, see here for the general description of name handling

GitHub - icing/mod_md: Let's Encrypt (ACME) in Apache httpd

[casterln]

I did not explicitly list all the SAN hosts. Just
MDomain nature.berkeley.edu

Then mod_md looked out into all vhost configs, found the ServerAlias lines and crafted a certificate request that is way over the 100 host limit.

I wonder if certbot might be able to make smaller requests of Let's Encrypt.

[MikeMcQ]

So, just so I understand, in one VirtualHost you have a ServerAlias list of more than 99 names?

Can you just paste the MDomain statement and its associated VirtualHost here? Please add 3 backticks before and after so we don't lose some apache tags due to forum formatting.

Like:
```
MDomain ...
VirtualHost for port 443
```

@casterln

Then mod_md looked out into all vhost configs

That sounds like a bug in mod_md if that's what it did. Might be worth also asking on that github I linked. Or, do you have the same name as the ServerName in each VirtualHost and rely just on ServerAlias for SNI selection?

[MikeMcQ]

These two MDomain statements created two separate certificates. Each cert's SAN list had just two names in it - the ServerName and its ServerAlias.

I looked at the public CT logs and see your existing certs. Unless all those names are in one VirtualHost I don't know why something similar would not work. I think we are using terminology in different ways so thought a specific example would help frame the discussion.

# Use expanded MDomain format to specify Let's Encrypt Staging system
<MDomain testu2.example.com>
  MDCertificateAuthority https://acme-staging-v02.api.letsencrypt.org/directory
</MDomain>
<VirtualHost *:443>
     ServerName testu2.example.com
     ServerAlias extra-testu2.example.com
     CustomLog ${APACHE_LOG_DIR}/MD443u2access.log combined
     DocumentRoot /var/www/html
     SSLEngine on
     # other SSL options here (but not cert info)
</VirtualHost>

<MDomain testu3.example.com>
  MDCertificateAuthority https://acme-staging-v02.api.letsencrypt.org/directory
</MDomain>
<VirtualHost *:443>
     ServerName testu3.example.com
     ServerAlias extra-testu3.example.com
     CustomLog ${APACHE_LOG_DIR}/MD443u3access.log combined
     DocumentRoot /var/www/html
     SSLEngine on
     # other SSL options here (but not cert info)
</VirtualHost>

Note: mod_md dynamically adds the cert definitions to Apache so they are not defined explicitly in the VirtualHost. Other options like SSLCipherSuite can/should still be set (globally or in the VirtualHost itself).

mod_md has flexible options for combining and managing certs. This is just one simple example. It is meant as an example only - not a working template.

[casterln]

Yes. Every VirtualHost has the same ServerName, but each has its own ServerAlias.

[rg305]

@casterln

Every VirtualHost has the same ServerName

Say what?

[MikeMcQ]

@casterln

Every VirtualHost has the same ServerName, but each has its own ServerAlias.

That's unusual. If using name-based VirtualHosts that ServerName won't ever be used to resolve requests. You could just add a unique qualifier to the name so it is not duplicated.

Or, look at using the mod_md manual option of the MDomain statement. The auto option is the default and what is causing the conflict with your duplicate names.

Still, I think making sure all your ServerNames are unique is a much cleaner way forward.

[rg305]

@MikeMcQ

That's unusual.

To say the least!
I'd dare say that doing that has never been a recommended practice.

@casterln, where did you learn to do that?

[casterln]

I learned it by the seat of my pants, nowhere else! As you can all see, this is a case of an old dog learning new tricks. Thank you for bearing with me.

I am encouraged now though. I've set the ServerName to what I used have as the ServerAlias and have gotten good Let's Encrypt certs now for two VirtualHosts. Now I'll gradually work through the rest.

[scsiraidguru]

Did you consider using wildcard certificates? Create one for each domain.

[jvanasco]

It sounds like @MikeMcQ found a potential directive that can be used to segment the certificates for you. I would try a small test with that and, if it doesn't work, I would reach out to the developers of the mod_md.

That being said, while I don't know what each of your various domains are configured to do under Apache, I think this difficulty may be over-complicated due to the architecture of your system, and different infrastructures may work better.

Apache typically isn't the best server to handle high throughput or hundreds of domains. With setups like yours, it is not uncommon to have HTTPS terminated on a gateway server or a websever that excels at high connections (like nginx). It is also not uncommon to see a setup where domains are partitioned into multiple Apache (or Nginx) configurations.

For example:

• port 80: "Vanilla" Apache or nginx. Accept all the domains, proxypass into 3 different ip ranges, each range served by a different Apache master/config. Potentially terminate SSL
  ** port 8080 : assign 99 domain names
  ** port 8081 : assign 99 domain names
  ** port 8082 : assign 99 domain names

When it comes to obtaining certificates and ssl, you then have several options two of which include:

• The 8080, 8081, 8082 servers each have mod_md. because they each have 99 domains, you will inherently meet the limits so you can let mod_md run as normal on each of the 99 domain name sets.

• The port 80 server terminates the SSL, and uses proxypass of the acme-challenge directory to Certbot running in standalone mode on a higher port to handle ACME..

To manage all this, you could write some simple scripts to partition the domains and manage them. Ideally each host has been defined in it's own include file already, so you're often managing symlinks.

In terms of using other acme-clients, my suggestion for a setup like this would be to do something like this:

1- terminate SSL on nginx or a gateway on port 80, then proxypass back to apache. domains could easily be enrolled/unenrolled via a script to generate and manage the config files. nginx and gateways are fairly lightweight and great with concurrency.

2- manually assign certificates to virtualhosts, then run certbot in standalone mode via a proxypass.

Certbot does not do well managing hundreds of server blocks on any package. I believe nginx has more issues.

I will also stress that running many domains on any single certificate is not a good idea anymore. At one point it was recommended for many reasons, but in practice has caused many issues as any single failed domain will block reissue. Because many clients do not properly cleanup pending authorizations on failed orders, these can stack up and ratetlimit an account.

Personally, I would run nginx on port 80 and proxypass back to multiple apaches. I would have each domain run it's own certificate, and i'd have certbot do the work in standalone mode on a higher port. I'd use fabfile.org to write a script that can be invoked to enroll domains and remove them from the system. i.e. enrolling would assign the domain to an apache config file, generate a stub include file for that domain's host, and manage the proxypass from nginx onto the right apache port, and invoking Certbot to get the certificate.

[MikeMcQ]

@jvanasco

It sounds like @MikeMcQ found a potential directive that can be used to segment the certificates for you. I would try a small test with that

Actually, I was just demonstrating that the default MDomain statement should have worked for them. From that we discovered they used an unusual (and frankly incorrect) method of repeating the identical ServerName in every VirtualHost. Once they set each ServerName properly it worked fine. That is, the end result is each VirtualHost has its own (small) certificate for just the couple/few names listed in its ServerName and ServerAlias statements.

Agree though there are other architectures that could work too. Even something like caddy in front of Apache.

[webprofusion]

Instead of stuffing your cert with domains, just use one cert per domain (or domain + subdomains), you will need to update your workflow and you'll need a web server config per domain (which I think you have already), but it's much simpler to scale as you add domains.