Double-check from another box that you can actually access the resource (for example with wget -S http://server/) - just to make sure that you are able to connect, and the redirect is what you expect it to be.
Yes, inside the /.well-known/acme-challenge/ directory I have a index.html which can be accessed in the browser:
http://example.com/.well-known/acme-challenge/
still getting this error
Failed authorization procedure. www.example.com (http-01): urn:acme:error:connection :: The server could not connect to the client to verify the domain :: Could not connect to http://www.example.com/.well-known/acme-challenge/91-iGi8G53Z5jRfE1M6s4V6gf3PgF3BZxb88H7rzeKs, example.com (http-01): urn:acme:error:connection :: The server could not connect to the client to verify the domain :: Could not connect to http://example.com/.well-known/acme-challenge/mV_VU31p0XnJpcx_KWTyHL5ZipYwSJOgfw7GPMW1PBw
IMPORTANT NOTES:
- The following errors were reported by the server:
Domain: www.example.com
Type: connection
Detail: Could not connect to
http://www.example.com/.well-known/acme-challenge/91-iGi8G53Z5jRfE1M6s4V6gf3PgF3BZxb88H7rzeKs
Domain: example.com
Type: connection
Detail: Could not connect to
http://example.com/.well-known/acme-challenge/mV_VU31p0XnJpcx_KWTyHL5ZipYwSJOgfw7GPMW1PBw
To fix these errors, please make sure that your domain name was
entered correctly and the DNS A record(s) for that domain
contain(s) the right IP address. Additionally, please check that
your computer has a publicly routable IP address and that no
firewalls are preventing the server from communicating with the
client. If you're using the webroot plugin, you should also verify
that you are serving files from the webroot path you provided.
The cli command was invoked with /opt/letsencrypt/certbot-auto certonly -c /etc/letsencrypt/configs/renew-webroot-example.ini --pre-hook "service nginx stop" --post-hook "service nginx start"
And here is the config for that domain:
cat /etc/letsencrypt/configs/renew-webroot-example.ini
email = no-reply@example.com
domains = example.com, www.example.com
text = True
authenticator = webroot
webroot-path = /usr/share/nginx/html
I´m out of ideas.
I had the same problem with a nginx proxy instance. What I did was avoided 301 redirecting the /.well-known directory by having a separate location block
location /.well-known {
alias /var/www/letsencrypt/.well-known; # have this as the webroot
}
location / {
return 301 https://$server_name$request_uri;
}
5 Likes
Thank you for your reply. Yes, as you can see I have a separate location block:
location ^~ /.well-known/acme-challenge/ {
# Set correct content type. According to this:
# https://community.letsencrypt.org/t/using-the-webroot-domain-verification-method/1445/29
# Current specification requires "text/plain" or no content header at all.
# It seems that "text/plain" is a safe option.
default_type "text/plain";
# This directory must be the same as in /etc/letsencrypt/cli.ini
# as "webroot-path" parameter. Also don't forget to set "authenticator" parameter
# there to "webroot".
# Do NOT use alias, use root! Target directory is located here:
# /var/www/common/letsencrypt/.well-known/acme-challenge/
root /usr/share/nginx/html;
}
Accessing .well-known/acme-challenge/ from a browser works so this URL doesn´t get redirected to https.
During the certificate renewal I watched the contents of the directory
watch -n1 ls
and i could see the files where created:
kwpmIklXr1K_g1yZdnI8ZBNzoEPGUle1XoFlbKi3zw4
test # my test file
vhB_-Xa7C1M6efMpvtR3sFrN_7kDZCgKIb7mGnXspAU
and after throwing the error this files (except my test file) are deleted.
You’re stopping nginx before the renewal process starts and the validation is performed. That is not necessary with the webroot plugin, and it is in fact what’s preventing the validation server from connecting to your site - there’s simply no web server listening on port 80 during that time.
The renew example using that --pre-hook (which I assume you got from the certbot documentation) is using the standalone plugin, which would spawn a web server for you, but that’s not the case with webroot.
Basically, remove the --pre-hook and replace the --post-hook command with something like service nginx reload and you should be good to go.
4 Likes
You’re stopping nginx before the renewal process starts and the
validation is performed. That is not necessary with the webroot plugin,
and it is in fact what’s preventing the validation server from
connecting to your site - there’s simply no web server listening on port
80 during that time.
The renew example using that --pre-hook
(which I assume you got from the certbot documentation) is using the
standalone plugin, which would spawn a web server for you, but that’s
not the case with webroot.
Basically, remove the --pre-hook and replace the --post-hook command with something like service nginx reload and you should be good to go.
OMG you´re so right. Thank you so much it works now.
3 Likes
If it makes you feel any better, I was writing a rather long reply about debugging connectivity problems from the validation server when I finally noticed that --pre-hook and made the connection. Sometimes the most obvious thing is the most unnoticed. 
3 Likes
Damn right. I can´t tell you how much time I´ve spent getting this *** to work 
2 Likes
Happens to the best of us
2 Likes
Brilliant, thanks … I was missing the location / { before the return 301 https://blahblah$request_uri; } … now have webroot authentication working again.
1 Like
I’ve tried many variations on the suggestions but I’m still stuck. I’m doing HTTP to HTTPS redirect via nginx and as well I’m redirect from non-www to www. This is using the certbot from the certbot PPA for ubuntu, ubuntu 16.04.
Relevant parts of (the latest version of) server blocks:
server {
server_name mydomain.com;
listen 80;
listen [::]:80;
location /.well-known/acme-challenge/ {}
location / { return 301 https://www.mydomain.com$request_uri; }}
server {
server_name www.mydomain.com
gzip on;
gzip_proxied any;
gzip_types text/plain text/xml text/css application/javascript;
gzip_vary on;
gzip_disable “MSIE [1-6].(?!.*SV1)”;
# SSL configuration
listen 443 ssl http2;
listen [::]:443 ssl http2;
include snippets/ssl-www.mydomain.com.conf;
include snippets/ssl-params.conf;
access_log /var/log/nginx/mydomain.com.access.log;
error_log /var/log/nginx/mydomain.com.error.log;
root /var/www/www.mydomain.com;
index index.php;
location ~* \.(?:js|css|png|jpe?g|gif|ico)$ {
expires max;
break;
}
location / {
try_files $uri $uri/ /index.php?q=$request_uri;
}
# If directly accessing a PHP file in the public dir other than index.php
location ~* \.php$ {
fastcgi_pass unix:/run/php/php7.0-fpm.sock;
fastcgi_index index.php;
include /etc/nginx/fastcgi_params;
fastcgi_param SCRIPT_FILENAME $document_root/$fastcgi_script_name;
}
}
As others have done I did a watch on the “ls” and verified the files get created and removed after the ‘certbot renew --dry-run’ fails.
And here’s my failing dry-run:
sudo certbot renew --dry-run
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Processing /etc/letsencrypt/renewal/www.mydomain.com.conf
Cert is due for renewal, auto-renewing…
Starting new HTTPS connection (1): acme-staging.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for mydomain.com
http-01 challenge for www.mydomain.com
Waiting for verification…
Cleaning up challenges
Attempting to renew cert from /etc/letsencrypt/renewal/www.mydomain.com.conf produced an unexpected error: Failed authorization procedure. www.mydomain.com (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://www.mydomain.com/.well-known/acme-challenge/7c6_KkOErGi-Nk4rObk5uri5Nj2NB35D9nVLfsBiU-Q: "
404 Not Found
". Skipping. ** DRY RUN: simulating 'certbot renew' close to cert expiry ** (The test certificates below have not been saved.)
All renewal attempts failed. The following certs could not be renewed:
/etc/letsencrypt/live/www.mydomain.com/fullchain.pem (failure)
** DRY RUN: simulating ‘certbot renew’ close to cert expiry
** (The test certificates above have not been saved.)
1 renew failure(s), 0 parse failure(s)
IMPORTANT NOTES:
-
The following errors were reported by the server:
Domain: www.mydomain.com
404 Not Found
Type: unauthorized
Detail: Invalid response from
http://www.mydomain.com/.well-known/acme-challenge/7c6_KkOErGi-Nk4rObk5uri5Nj2NB35D9nVLfsBiU-Q:
"404 Not Found
"To fix these errors, please make sure that your domain name was
entered correctly and the DNS A record(s) for that domain
contain(s) the right IP address.
@chrisco23, it doesn’t look like your configuration file defines any web root at all (via root or alias directives, I guess), unless you didn’t include that in your configuration excerpt. In that case, the web server would not know where to look for the files that Certbot creates.
Sorry, had abbreviated the config and having trouble with the formatting a lost the line that set “# …etc” … I have edited my post to include the rest of the nginx config.
Note that I’ve run this site for years and the lets encrypt for… forget now but maybe a year. I renewed it once before with some difficulty but I used a different method then, involving certbotrenew.sh.
Thanks!
If you create a file /var/www/www.mydomain.com/.well-known/acme-challenge/test.txt, can you then see it in a browser at http://www.myomain.com/.well-known/acme-challenge/test.txt?
Nope 
Seems Chrome is still wanting to redirect to https, and failing. What’s worse is that Chrome won’t even let you through with the “Advanced” button (as I’m pretty sure it used to) so the site is completely inaccessible right now.
I guess that means there’s really something I’m missing about the nginx config then right?
now I’m seeing if I try the same test without the “www” (but with http rather than https) I get 404.
I think so. It’s possible that the {} block did not actually prevent the subsequent block from applying; I don’t know enough about Nginx configuration to know whether more than one location block can apply to the same request or not.
The Let’s Encrypt CA is actually willing to follow a 301 redirect in this case (which is often useful), but that might not be relevant to your debugging here.
In order to pass the authentication challenge, Certbot needs to be able to create text files in the webroot location specified in /etc/letsencrypt/renewal/www.mydomain.com.conf, within the subdirectory .well-known/acme-challenge and then have those files be visible to the public via HTTP under http://www.mydomain.com/.well-known/acme-challenge/.
In desperation, I temporarily changed the top server block to respond to www.mydomain.com instead of mydomain.com, then did the renew, then changed it back again.
Obviously this is not a permanent solution but just enough to get the site back online.