Thank you very much for your reply. I do have the correct domain. I edited it for this forum since I am new to security culture and not certain what you should and should not share. In retrospect, I suppose the domain is not bad, so I will post it in case you can spot another problem.
server {
listen 80;
listen [::]:80;
server_name hippocampusanalytics.com www.hippocampusanalytics.com;
location ~ /.well-known/acme-challenge {
allow all;
root /var/www/dsp-ui;
}
location / {
# First attempt to serve request as file, then
# as directory, then fall back to displaying a 404.
try_files $uri $uri/ /index.html;
}
}
server {
listen 443 ssl http2;
listen [::]:443 ssl http2;
server_name hippocampusanalytics.com www.hippocampusanalytics.com;
server_tokens off;
ssl_certificate /etc/letsencrypt/live/www.hippocampusanalytics.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/www.hippocampusanalytics.com/privkey.pem;
ssl_buffer_size 8k;
ssl_dhparam /etc/ssl/certs/dhparam-2048.pem;
ssl_protocols TLSv1.2 TLSv1.1 TLSv1;
ssl_prefer_server_ciphers on;
ssl_ciphers ECDH+AESGCM:ECDH+AES256:ECDH+AES128:DH+3DES:!ADH:!AECDH:!MD5;
ssl_ecdh_curve secp384r1;
ssl_session_tickets off;
ssl_stapling on;
ssl_stapling_verify on;
resolver 8.8.8.8;
location / {
try_files $uri $uri/ /index.html;
}
location @nodejs {
proxy_pass http://nodejs:8080;
add_header X-Frame-Options "SAMEORIGIN" always;
add_header X-XSS-Protection "1; mode=block" always;
add_header X-Content-Type-Options "nosniff" always;
add_header Referrer-Policy "no-referrer-when-downgrade" always;
add_header Content-Security-Policy "default-src * data: 'unsafe-eval' 'unsafe-inline'" always;
# add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" always;
# enable strict transport security only if you understand the implications
}
root /var/www/dsp-ui;
index index.html;
}
Is there a way of checking if I have requested a certificate too many times. I am using the --staging setting for my docker-compose file, but I have tried many different potential solutions.
Also, here is my docker-compose in case you can spot the problem there.
Having helped thousands of people with similar problems, I can say for my part that what you've posted is very common and shouldn't contain anything particularly useful to an attacker. It might identify weak protocols and/or such, but by the time we're done it should serve as more of a deterrent than an aid to an attacker.
That said...
You generated two certificates today: one covering hippocampusanalytics.com and one covering www.hippocampusanalytics.com. These will have different certificate names in certbot. What you really want is one certificate covering bothhippocampusanalytics.com and www.hippocampusanalytics.com. That certificate should be named "hippocampusanalytics.com".
If anything fails at any point in the following process, please stop there and let us know.
If you've made it this far, things should be working as expected. You can now remove any unneeded certificates by...
finding the name of any unneeded certificate:
sudo certbot certificates
deleting the certificate using its name:
sudo certbot delete --cert-name name
Keep in mind that a certificate's name is just a label and thus deleting a certificate that happens to have a particular name (e.g. www.hippocampusanalytics.com) does not remove coverage by another certificate of any domain names. Multiple certificates can independently cover the same domain name (though this is usually a sign of misunderstanding of what names should be covered by a particular certificate).
@griffin Thank you for your detailed instructions and encouragement
I made it successfully up to the dry-run test and revoked the certificate for www.hippocampusanalytics.com so you can reuse it (if that is even a concern).
The dry-run test suggests that the domain is invalid
Challenge failed for domain hippocampusanalytics.com
http-01 challenge for hippocampusanalytics.com
Cleaning up challenges
Failed to renew certificate hippocampusanalytics.com with error: Some challenges have failed.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
All simulated renewals failed. The following certificates could not be renewed:
/etc/letsencrypt/live/hippocampusanalytics.com/fullchain.pem (failure)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1 renew failure(s), 0 parse failure(s)
IMPORTANT NOTES:
- The following errors were reported by the server:
Domain: hippocampusanalytics.com
Type: unauthorized
Detail: Invalid response from
http://hippocampusanalytics.com/.well-known/acme-challenge/S5eBcRWKVfaL5dk8xVDBXX9_DJNY-cYKsYIqWbTUAYE
[159.65.162.220]: "<html>\r\n<head><title>404 Not
Found</title></head>\r\n<body>\r\n<center><h1>404 Not
Found</h1></center>\r\n<hr><center>nginx/1.19.10</c"
To fix these errors, please make sure that your domain name was
entered correctly and the DNS A/AAAA record(s) for that domain
contain(s) the right IP address.
docker-compose shows that the server is up, but a curl test fails.
The A record on my ISP's cPanel contains the inet address and the AAAA record contains the inet6 address.
in the docker-container logs it appears that certbot wants me to make a choice, but I do not know how I would answer the question:
Plugins selected: Authenticator webroot, Installer None
certbot | Cert not yet due for renewal
certbot |
certbot | You have an existing certificate that has exactly the same domains or certificate name you requested and isn't close to expiry.
certbot | (ref: /etc/letsencrypt/renewal/hippocampusanalytics.com.conf)
certbot |
certbot | What would you like to do?
certbot | - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
certbot | 1: Keep the existing certificate for now
certbot | 2: Renew & replace the certificate (may be subject to CA rate limits)
certbot | - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
certbot | An unexpected error occurred:
certbot | EOFError
certbot | Please see the logfiles in /var/log/letsencrypt for more details.
certbot | Select the appropriate number [1-2] then [enter] (press 'c' to cancel): Saving debug log to /var/log/letsencrypt/letsencrypt.log
certbot | Plugins selected: Authenticator webroot, Installer None
certbot | Cert not yet due for renewal
certbot |
certbot | You have an existing certificate that has exactly the same domains or certificate name you requested and isn't close to expiry.
certbot | (ref: /etc/letsencrypt/renewal/hippocampusanalytics.com.conf)
certbot |
certbot | What would you like to do?
certbot | - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
certbot | 1: Keep the existing certificate for now
certbot | 2: Renew & replace the certificate (may be subject to CA rate limits)
certbot | - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
certbot | An unexpected error occurred:
certbot | EOFError
certbot | Please see the logfiles in /var/log/letsencrypt for more details.
certbot | Select the appropriate number [1-2] then [enter] (press 'c' to cancel):
Finally, there were errors in the webserver but I updated the ssl protocols and updated nginx per your last instructions and they went away!
I think you might have a saved certbot configuration inside your docker instance. I noticed from your certificate history that you did acquire a correct certificate (covering hippocampusanalytics.com and www.hippocampusanalytics.com), but then you acquired another incorrect certificate afterwards (only covering hippocampusanalytics.com). You should first delete all of the unnecessary certificates from your certbot image (using certbot delete as I instructed above) then update the certificate named hippocampusanalytics.com to cover both using the nginx authenticator instead of the webroot authenticator. When using docker instances, you need to be sure that you are saving the correct certificate in your persistent image and not acquiring a new certificate with each run because you will almost definitely hit the rate limits in short order! The persistent certificate should then be renewed as needed using certbot renew.
I believe the command here is involved with the issue:
The --keep tells certbot not to acquire a new certificate if a correct one exists that is not near expiration. It prevents you from hitting the rate limits. Its (very dangerous) opposite is --force-renewal, which tells certbot to acquire a new certificate even if a correct one exists that is not near expiration.
The problem with both the original command and the one that I just gave you is that unlike the run command, the certonly command will not reload nginx when a certificate is acquired/renewed. In non-docker situations, you could just add --deploy-hook "nginx -s reload" to have certbot send a reload signal to the nginx process. In docker though, my understanding (note: I'm not a docker expert) is that you don't have command line access to the nginx process, so you need to use docker through certbot to signal the nginx container to reload via --deploy-hook "docker kill -s HUP webserver".
When I bring the server up, the certificates are served correctly and clients will get a response, but someone (not certain if it is nginx or letsencrypt) runs a test calling for a static response. This is failing. I have included a snippet of the error below. The file .well-known and its contents are not present in the root directory (/var/www/html). I am not certain when that was established but I have been seeing this problem all along. This time, I think the error is real (the server can't find the file) Do I need to reinitialize something to fill that directory?
That is most definitely a Let's Encrypt server attempting to validate an http-01 challenge. Keep in mind that the certbot nginx authenticator adds an exception to your nginx configuration to serve the challenge file. Since you already have a similar exception, there might be a conflict. Additionally, your webroot directory (root) for port 80 is only defined inside of your current exception, which might cause a failure to serve the challenge file if certbot's exception interferes. To avoid this issue entirely, we can switch back to the webroot authenticator, but include the correct webroot this time in the certbot command.