I have 3 Nginx servers as reverse proxy for thousands of domains(only some domains have SSL). The 3 servers are identical in server specs and configurations.
The certbot install and renew SSL without issue when there’s less than 4000 sites. After I added 8000 more sites to one server, certbot won’t be able to install new ssl or renew existing ssl. Everyday when certbot renew runs, it runs for 6 to 8 mins and using 100% CPU. At end, it fails with 404 error.
I’ve managed to reduce the total sites from 12,000 to 8000, the renew still fail but there’s a little chance one site renewal might success.
I am wondering anyone else experiencing same issue and may share the solution. Thank you for any input!
Update: There’s only 18 domains on this server has SSL.
My domain is: exampledomain.com
I ran this command: certbot --nginx certonly -d exampledomain.com
It produced this output:
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator nginx, Installer nginx
Starting new HTTPS connection (1): acme-v02.api.letsencrypt.org
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for exampledomain.com
Waiting for verification…
Cleaning up challenges
Failed authorization procedure. exampledomain.com (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from https://exampledomain.com/.well-known/acme-challenge/RRZ13ty2JMw7iDzi22d-V47D0Nyl_vhqA7jFB1ryi7M [64.xx.xx.12]: “\r\n404 Not Found\r\n\r\n
404 Not Found
\r\nnginx\r\n”
IMPORTANT NOTES:
-
The following errors were reported by the server:
Domain: exampledomain.com
Type: unauthorized
Detail: Invalid response from
https://exampledomain.com/.well-known/acme-challenge/RRZ13ty2JMw7iDzi22d-V47D0Nyl_vhqA7jFB1ryi7M
[64.xx.xx.12]: “\r\n404 Not
Found\r\n\r\n404 Not
\r\n
Found
nginx\r\n”To fix these errors, please make sure that your domain name was
entered correctly and the DNS A/AAAA record(s) for that domain
contain(s) the right IP address.
My web server is (include version): nginx version: nginx/1.15.9
The operating system my web server runs on is (include version): CentOS Linux release 7.6.1810 (Core)
I can login to a root shell on my machine (yes or no, or I don’t know): yes
I’m using a control panel to manage my site (no, or provide the name and version of the control panel): no control panel, this is a reverse proxy server.
The version of my client is (e.g. output of certbot --version
or certbot-auto --version
if you’re using Certbot): certbot 0.30.2 (just upgraded to certbot 0.34.2, same issue)