Getting 404 when renewing a cert

Server
1

Domain: hidden domain
Type: unauthorized
Detail: Invalid response from
http://hidden/.well-known/acme-challenge/ieIbudn_UZG901I8d9FfxFmRX91zRiS3FHbPoV4N1qc:
"

404 Not Found

404 Not Found

"

I have already successfully added certificates to 10 websites but when I run certbot renew they all fail with the same issue.
The server is running nginx.
All of the domains are 301’d to https://
When the renew happens certbot tries to check the certificate token in /.well-known/… using http but fails with a 404 when the redirect happens.
If I remove the redirects the renew works ok.

I am stuck however on how to automate this. I am adding new domains at the rate of 2 per month so the problem is going to get worse.

I cant be the first to have this problem. How do people work this?

Thanks

2

Hi @fagansystems,

Other people’s https://example.com/.well-known/acme-challenge/ directories are typically served from the same location (document root) as http://example.com/.well-known/acme-challenge/, so the redirect doesn’t usually affect their ability to satisfy the challenge. Do you know of some reason that your site is using a different document root or different path configuration for the HTTP and HTTPS versions of the site?

3

The document root is the same location for all the variants http/https, www/naked domain.

4

Something must be going wrong, then, since you're getting 404 errors...

Are Certbot and Nginx set to use the same document root? Is one incorrect?

Creating the certificates the first time works, but renewing fails? How has the Nginx configuration changed?

Could you provide more information? Certbot's /var/log/letsencrypt/letsencrypt.log? Nginx's error.log? The Certbot and Nginx configuration?

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is:

I ran this command:

It produced this output:

My web server is (include version):

The operating system my web server runs on is (include version):

My hosting provider, if applicable, is:

I can login to a root shell on my machine (yes or no, or I don't know):

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):

5

When the certificate is first requested there is no redirect. All works ok. I found this after accidentally adding the redirect before I had requested the certificate.

Yes the web root and the nginx root are the same, the files are in the correct part of the tree, placing a file in the folder and accessing that file with a browser works

This is one of the domains with this issue
https://folders.cordelia-malthere.com/.well-known/test.txt
the file is in /var/www/vhosts/folders.cordelia-malthere.com/httpdocs/.well-known/

My domain is: folders.cordelia-malthere.com

I ran this command: /usr/bin/certbot renew

It produced this output:

My web server is (include version):
nginx version: nginx/1.10.3 (Ubuntu)
built with OpenSSL 1.0.2g 1 Mar 2016
TLS SNI support enabled

The operating system my web server runs on is (include version):
Ubuntu 16.04 LTS

My hosting provider, if applicable, is:
N/A this is hosted on a VPS on a linode

I can login to a root shell on my machine (yes or no, or I don’t know): yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel): No

I am new to this forum and unclear how I attach files to this reply, can you advise please?

6

How about /.well-known/acme-challenge?

I think this is a permission that automatically gets enabled for new users once they reach a certain level of activity on the forum. People who've needed to upload things before that have usually used an external site like Pastebin.

7

acme-challenge gets created and then removed again
If I add that folder I can retrieve files from there as well

I have added a file into there instead

This is an extract from the last letsencrypt log
I notice that certbot is trying http rather then https
When this happens the 301 happens and the renew fails

2017-10-01 23:11:12,473:DEBUG:certbot.reporter:Reporting to user: The following errors were reported by the server:

Domain: folders.cordelia-malthere.com
Type: unauthorized
Detail: Invalid response from http://folders.cordelia-malthere.com/.well-known/acme-challenge/tAklTuFst0WGzgeVsOwLo-bjqHMGJ3s4kgzw5mC5OL0: "

404 Not Found

404 Not Found

"

To fix these errors, please make sure that your domain name was entered correctly and the DNS A/AAAA record(s) for that domain contain(s) the right IP address.
2017-10-01 23:11:12,473:INFO:certbot.auth_handler:Cleaning up challenges
2017-10-01 23:11:12,473:DEBUG:certbot.plugins.webroot:Removing /var/www/vhosts/folders.cordelia-malthere.co.uk/httpdocs/.well-known/acme-challenge/tAklTuFst0WGzgeVsOwLo-bjqHMGJ3s4kgzw5mC5OL0
2017-10-01 23:11:12,474:DEBUG:certbot.plugins.webroot:All challenges cleaned up, removing /var/www/vhosts/folders.cordelia-malthere.co.uk/httpdocs/.well-known/acme-challenge
2017-10-01 23:11:12,474:WARNING:certbot.renewal:Attempting to renew cert (folders.cordelia-malthere.com) from /etc/letsencrypt/renewal/folders.cordelia-malthere.com.conf produced an unexpected error: Failed authorization procedure. folders.cordelia-malthere.com (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://folders.cordelia-malthere.com/.well-known/acme-challenge/tAklTuFst0WGzgeVsOwLo-bjqHMGJ3s4kgzw5mC5OL0: "

404 Not Found

404 Not Found

". Skipping. 2017-10-01 23:11:12,475:DEBUG:certbot.renewal:Traceback was: Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/certbot/renewal.py", line 421, in handle_renewal_request main.renew_cert(lineage_config, plugins, renewal_candidate) File "/usr/lib/python2.7/dist-packages/certbot/main.py", line 650, in renew_cert _get_and_save_cert(le_client, config, lineage=lineage) File "/usr/lib/python2.7/dist-packages/certbot/main.py", line 77, in _get_and_save_cert renewal.renew_cert(config, domains, le_client, lineage) File "/usr/lib/python2.7/dist-packages/certbot/renewal.py", line 297, in renew_cert new_certr, new_chain, new_key, _ = le_client.obtain_certificate(domains) File "/usr/lib/python2.7/dist-packages/certbot/client.py", line 318, in obtain_certificate self.config.allow_subset_of_names) File "/usr/lib/python2.7/dist-packages/certbot/auth_handler.py", line 81, in get_authorizations self._respond(resp, best_effort) File "/usr/lib/python2.7/dist-packages/certbot/auth_handler.py", line 138, in _respond self._poll_challenges(chall_update, best_effort) File "/usr/lib/python2.7/dist-packages/certbot/auth_handler.py", line 202, in _poll_challenges raise errors.FailedChallenges(all_failed_achalls) FailedChallenges: Failed authorization procedure. folders.cordelia-malthere.com (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://folders.cordelia-malthere.com/.well-known/acme-challenge/tAklTuFst0WGzgeVsOwLo-bjqHMGJ3s4kgzw5mC5OL0: " 404 Not Found

404 Not Found

"

and this is a snippet from the nginx config
server {
listen 80;
server_name folders.cordelia-malthere.com;
return 301 https://folders.cordelia-malthere.com$request_uri;
}

server {

listen 80; ## listen for ipv4; this line is default and implied

    listen 443 ssl http2;

    server_name folders.cordelia-malthere.com;

    root /var/www/vhosts/folders.cordelia-malthere.com/httpdocs;

    access_log  /var/log/nginx/folders.cordelia-malthere_access.log;
    error_log  /var/log/nginx/folders.cordelia-malthere_error.log;

    include snippets/ssl-folders-cordelia-malthere.conf;
    include snippets/ssl-params.conf;

I will find a way to send you the files tomorrow.

Thanks For the help

8

Also, what’s in your renewal configuration file in /etc/letsencrypt/renewal?

9

renew_before_expiry = 30 days

version = 0.14.2
archive_dir = /etc/letsencrypt/archive/folders.cordelia-malthere.com
cert = /etc/letsencrypt/live/folders.cordelia-malthere.com/cert.pem
privkey = /etc/letsencrypt/live/folders.cordelia-malthere.com/privkey.pem
chain = /etc/letsencrypt/live/folders.cordelia-malthere.com/chain.pem
fullchain = /etc/letsencrypt/live/folders.cordelia-malthere.com/fullchain.pem

Options used in the renewal process

[renewalparams]
authenticator = webroot
installer = None
account = e0bdb0c07bb102119c9a3a8d9c57a3d5
webroot_path = /var/www/vhosts/folders.cordelia-malthere.co.uk/httpdocs,
[[webroot_map]]
folders.cordelia-malthere.com = /var/www/vhosts/folders.cordelia-malthere.co.uk/httpdocs

10

Nginx is configured to use /var/www/vhosts/folders.cordelia-malthere.com/httpdocs and Certbot is configured to use /var/www/vhosts/folders.cordelia-malthere.co.uk/httpdocs. Notice one is “.com” and one is “.co.uk”.

Nginx’s error.log can probably confirm that that’s the issue.

You can run “certbot certonly” with the same options used to originally create the certificate, with that one path modified, or you can edit the /etc/letsencrypt/renewal/ config file in a text editor to adjust it (while being careful not to change anything else).

1 Like
11

Thanks for the help, I can confirm that resolved the issue. I will have to check the others now to try to figure out why they aren’t working.

12

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.