Certbot auto-renew has stopped working

devdrake0 · September 21, 2019, 9:17am

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. https://crt.sh/?q=example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: codetips.co.uk

I ran this command: sudo certbot renew --dry-run

It produced this output:

Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/codetips.co.uk.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cert is due for renewal, auto-renewing...
Plugins selected: Authenticator nginx, Installer nginx
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for codetips.co.uk
http-01 challenge for www.codetips.co.uk
Waiting for verification...
Cleaning up challenges
Attempting to renew cert (codetips.co.uk) from /etc/letsencrypt/renewal/codetips.co.uk.conf produced an unexpected error: Failed authorization procedure. codetips.co.uk (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from https://www.codetips.co.uk/.well-known/acme-challenge/leKBkXwzErXJHH_gjOQ26Z6yiS-01Olih60g9OEzmXU [2606:4700:20::681a:3de]: "<html>\n<head><title>404 Not Found</title></head>\n<body bgcolor=\"white\">\n<center><h1>404 Not Found</h1></center>\n<hr><center>ngin", www.codetips.co.uk (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from https://www.codetips.co.uk/.well-known/acme-challenge/2CkTpbKk4zn-MSez3Mh2X5R0CAy_TBbmrlIoepbBwJs [2606:4700:20::681a:3de]: "<html>\n<head><title>404 Not Found</title></head>\n<body bgcolor=\"white\">\n<center><h1>404 Not Found</h1></center>\n<hr><center>ngin". Skipping.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/comments.codetips.co.uk.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cert is due for renewal, auto-renewing...
Plugins selected: Authenticator nginx, Installer nginx
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for comments.codetips.co.uk
http-01 challenge for www.comments.codetips.co.uk
Waiting for verification...
Cleaning up challenges
Attempting to renew cert (comments.codetips.co.uk) from /etc/letsencrypt/renewal/comments.codetips.co.uk.conf produced an unexpected error: Failed authorization procedure. www.comments.codetips.co.uk (http-01): urn:ietf:params:acme:error:tls :: The server experienced a TLS error during domain verification :: Fetching https://www.comments.codetips.co.uk/.well-known/acme-challenge/ANVENyZB2a4eL2jy7EvhyJXnIyqJzdfhS1Z5TT9YRv4: remote error: tls: handshake failure, comments.codetips.co.uk (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from https://comments.codetips.co.uk/.well-known/acme-challenge/hm55k6i_h_kNZUUolrpuVbvIReL1hpeGzL7jjTlrgXY [2606:4700:20::681a:2de]: "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 3.2 Final//EN\">\n<title>404 Not Found</title>\n<h1>Not Found</h1>\n<p>The requested URL was". Skipping.
All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/codetips.co.uk/fullchain.pem (failure)
  /etc/letsencrypt/live/comments.codetips.co.uk/fullchain.pem (failure)

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
** DRY RUN: simulating 'certbot renew' close to cert expiry
**          (The test certificates below have not been saved.)

All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/codetips.co.uk/fullchain.pem (failure)
  /etc/letsencrypt/live/comments.codetips.co.uk/fullchain.pem (failure)
** DRY RUN: simulating 'certbot renew' close to cert expiry
**          (The test certificates above have not been saved.)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
2 renew failure(s), 0 parse failure(s)

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: codetips.co.uk
   Type:   unauthorized
   Detail: Invalid response from
   https://www.codetips.co.uk/.well-known/acme-challenge/leKBkXwzErXJHH_gjOQ26Z6yiS-01Olih60g9OEzmXU
   [2606:4700:20::681a:3de]: "<html>\n<head><title>404 Not
   Found</title></head>\n<body bgcolor=\"white\">\n<center><h1>404 Not
   Found</h1></center>\n<hr><center>ngin"

   Domain: www.codetips.co.uk
   Type:   unauthorized
   Detail: Invalid response from
   https://www.codetips.co.uk/.well-known/acme-challenge/2CkTpbKk4zn-MSez3Mh2X5R0CAy_TBbmrlIoepbBwJs
   [2606:4700:20::681a:3de]: "<html>\n<head><title>404 Not
   Found</title></head>\n<body bgcolor=\"white\">\n<center><h1>404 Not
   Found</h1></center>\n<hr><center>ngin"

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address.
 - The following errors were reported by the server:

   Domain: comments.codetips.co.uk
   Type:   unauthorized
   Detail: Invalid response from
   https://comments.codetips.co.uk/.well-known/acme-challenge/hm55k6i_h_kNZUUolrpuVbvIReL1hpeGzL7jjTlrgXY
   [2606:4700:20::681a:2de]: "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML
   3.2 Final//EN\">\n<title>404 Not Found</title>\n<h1>Not
   Found</h1>\n<p>The requested URL was"

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address.
 - The following errors were reported by the server:

   Domain: www.comments.codetips.co.uk
   Type:   tls
   Detail: Fetching
   https://www.comments.codetips.co.uk/.well-known/acme-challenge/ANVENyZB2a4eL2jy7EvhyJXnIyqJzdfhS1Z5TT9YRv4:
   remote error: tls: handshake failure

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address. Additionally, please check that
   you have an up-to-date TLS configuration that allows the server to
   communicate with the Certbot client.

My web server is (include version): nginx version: nginx/1.14.0 (Ubuntu)

The operating system my web server runs on is (include version): Ubuntu 18.04

My hosting provider, if applicable, is: N/A

I can login to a root shell on my machine (yes or no, or I don’t know): Yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel): No

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you’re using Certbot): certbot 0.31.0

I’ve been running Certbot fine for months but, for some reason, the auto-renew has stopped working. I’ve read through quite a few articles on this site already, but I haven’t been able to find the fix.

I originally used this Digital Ocean article to set it up, and I tried running the following command as recommended in another article:

sudo sh -c "sed -i.bak -e 's/^\(pref_challs.*\)tls-sni-01\(.*\)/\1http-01\2/g' /etc/letsencrypt/renewal/*; rm -f /etc/letsencrypt/renewal/*.bak"

I’ve not changed my nginx config, and it still has the block for .well-known

        server {
                server_name www.codetips.co.uk;
                root /var/www/ghost/system/nginx-root;

                location / {
                        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                        proxy_set_header X-Forwarded-Proto $scheme;
                        proxy_set_header X-Real-IP $remote_addr;
                        proxy_set_header Host $http_host;
                        proxy_pass http://127.0.0.1:2368;
                }

                location ~ /.well-known {
                        allow all;
                 }

                client_max_body_size 50m;


                listen 443 ssl; # managed by Certbot
                ssl_certificate /etc/letsencrypt/live/codetips.co.uk/fullchain.pem; # managed by Certbot
                ssl_certificate_key /etc/letsencrypt/live/codetips.co.uk/privkey.pem; # managed by Certbot
                include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
                ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

        }

I’m at a bit of a loss for what to do next so, hopefully, someone can help.

Thanks!

_az · September 21, 2019, 9:26am

Did you happen to set “Full” or “Full (Strict)” in your Cloudflare SSL settings, at some point? If so, I think you might be encountering this Certbot bug: https://github.com/certbot/certbot/issues/7275

The best workaround I can think of is just to switch over to webroot authentication if you’re using Cloudflare with your domain.

So maybe add something like this in both your port 80 and port 443 virtualhosts:

location /.well-known/acme-challenge/ {
    root /var/www/letsencrypt;
}

then

mkdir -p /var/www/letsencrypt
service nginx reload
certbot renew --cert-name www.codetips.co.uk --webroot -w /var/www/letsencrypt --dry-run

devdrake0 · September 21, 2019, 10:48am

Thanks for your reply - you’re right I have enabled “Full” on Cloudflare.

Even with that location block I’m still getting a 404, so I’m assuming there is something wrong with my Nginx config somewhere.

devdrake0 · September 21, 2019, 10:56am

The issue is actually the root command. If I change the location block to the following I do get a response when using CURL.

                location ~ /.well-known/acme-challenge/ {
                        return 200 'gangnam style!';
                        add_header Content-Type text/plain;
                        # root /var/www/letsencrypt;
                }

That directory definitely exists

devdrake0 · September 21, 2019, 11:59am

OK I finally got this working.

I had to set my NGINX config to:

location ~ /.well-known/acme-challenge/ {
                        root /var/www/letsencrypt/;
}

Had to make the following directory:

sudo mkdir -p /var/www/letsencrypt/.well-known/acme-challenge/

Changed the permissions:

sudo chown -R www-data:www-data /var/www/letsencrypt/

Created an index.html file under the acme-challenge directory.

Then the dry-run was successful

sudo certbot renew --cert-name codetips.co.uk --webroot -w /var/www/letsencrypt --dry-run

devdrake0 · September 21, 2019, 12:00pm

Is there a way to update the auto renews, to use this new command ?

JuergenAuer · September 21, 2019, 1:05pm

Hi @devdrake0

use the command one time without --dry-run.

That updates your config file.

devdrake0 · September 21, 2019, 1:19pm

Ah ok excellent, thanks. I did this earlier, is there a way to confirm it has updated?

I'm assuming if I now do a normal certbot renew --dry-run it will use the new config and that will confirm it?

JuergenAuer · September 21, 2019, 1:24pm

Check your config file:

/etc/letsencrypt/renewal

There you should see the new values. A new --dry-run isn't required.

https://certbot.eff.org/docs/using.html#configuration-file

mnordhoff · September 21, 2019, 2:52pm

If you’re using the equivalent of “certbot certonly --webroot” now, Certbot will no longer automatically reload Nginx after certificates are renewed.

You can use the webroot authenticator with the nginx installer with “-a webroot -i nginx”, which will continue to automatically reload Nginx, or you can add a deploy hook to do something like “systemctl reload nginx”, either with the --deploy-hook command line option, or by putting a script in /etc/letsencrypt/renewal-hooks/deploy/.

devdrake0 · September 21, 2019, 2:56pm

Thanks @mnordhoff - I just ran another dry-run, with the following command. Am I right in assuming this will update the config and it will reload nginx for me going forward?

sudo certbot renew --cert-name codetips.co.uk -a webroot -i nginx -w /var/www/letsencrypt --dry-run

system · October 21, 2019, 3:10pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.