LE fails to verify domain when site has been set up only to listen to port 443 and all requests for http are redirected to https

Help
1

I’m trying to automate Let’s Encrypt on Nginx using webroot with the following in the server block:

location '/.well-known/acme-challenge' {
            root /var/www/example.com/html;
            try_files $uri /$1;
    }

This works well the first time around when Nginx only listens to port 80. But when renewing the certificate after it has been set up only to listen to port 443 and all requests for http are redirected to https, it fails:

 - The following errors were reported by the server:

   Domain: www.example.com
   Type:   urn:acme:error:connection
   Detail: Could not connect to http://www.example.com/.well-known
   /acme-challenge/fpzLz-y3_ei42E7b1tDnszZj5NUoXXChh3uVqQbVicc

   Domain: example.com
   Type:   urn:acme:error:connection
   Detail: Could not connect to http://example.com/.well-known/acme-
   challenge/RW-4jLj_vMILKb8Eahh3oHs8tMY3fYjrezlv7baAhjE

How can I set this up so it also will try https?

2

How are all http requests redirected to https if the server doesn’t listen on :80?

3

Like this:

server {
    	listen 80;
    	server_name example.com;
    	return 301 https://www.example.com$request_uri;
}
server {
        listen 80;
        server_name www.example.com;
        return 301 https://www.example.com$request_uri;
}
4

So the server should listen on 80. Issue

curl -I http://www.example.com/.well-known/acme-challenge/fpzLz-y3_ei42E7b1tDnszZj5NUoXXChh3uVqQbVicc

and see what happens

5

Of course you must not redirect HTTP to HTTPS for the path /.well-known/acme-challenge/* if you’re using the http-01 verification. Make sure it’s exempt.

6

I’m redirecting everything just like he does and it works fine. LE follows redirects.

7

How can we update a site that redirects all requests to https://www.example.com?

We’re using

sudo /opt/letsencrypt/letsencrypt-auto certonly --config /opt/letsencrypt/webroot.ini

to do it manually, with the following in /opt/letsencrypt/webroot.ini

rsa-key-size = 4096
email =me@example.com
domains = example.com, www.example.com
authenticator = webroot
webroot-path = /var/www/example.com/html

Thank you so much for your help!

8

Let’s Encrypt will follow redirects from HTTP to HTTPS. You just have to make sure that you’re redirecting to a URL serving the challenge token. The issue is not with your client invocation, but with your redirect or rather what you’re serving at the target URL.

1 Like
9

Hey, but as you see in my posts above, Let’s Encrypt didn’t follow the redirects.

10

Did you actually try it, or is this your assumption based on your server configuration?

@blfr provided you with a test command to check where the redirect is leading and what content it’s serving, could you provide that output?

11

Hey @pfg – here’s the entire configuration we’re using for our sites:

server {
        listen 80;
        server_name example.com;
        return 301 https://www.example.com$request_uri;
}

server {
        listen 80;
        server_name www.example.com;
        return 301 https://www.example.com$request_uri;
}

server {
        listen 443;
        server_name example.com;
        return 301 https://www.example.com$request_uri;
}

server {
	listen 443 ssl;
	listen [::]:443 ssl;

	server_name www.example.com;

	root /var/www/example.com/html;

	index index.php index.html index.htm;

	location / {
		try_files $uri $uri/ =404;
	}

	location ~ \.php$ {
		include snippets/fastcgi-php.conf;
		fastcgi_pass unix:/var/run/php5-fpm.sock;
	}

	location '/.well-known/acme-challenge' {
		root /var/www/example.com/html;
		try_files $uri /$1;
	}

	ssl on;
	ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
	ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
	ssl_stapling on;
	ssl_stapling_verify on;
	ssl_prefer_server_ciphers on;
	ssl_session_cache shared:SSL:10m;
	ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
	ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA;
	add_header Strict-Transport-Security "max-age=63072000; includeSubdomains; preload";
	ssl_dhparam /etc/ssl/private/dhparams.pem;
}
12

Please provide the output of @blfr’s curl command. If the challenge file has since been deleted, create a new one to test this (e.g. http://www.example.com/.well-known/acme-challenge/test with random content).

14

We tried it again with a test domain – moonloupe.com, and got this:

# sudo curl -I http://www.moonloupe.com/.well-known/acme-challenge/fpzLz-y3_ei42E7b1tDnszZj5NUoXXChh3uVqQbVicc
HTTP/1.1 301 Moved Permanently
Server: nginx/1.6.2
Date: Thu, 04 Feb 2016 07:43:46 GMT
Content-Type: text/html
Content-Length: 184
Connection: keep-alive
Location: https://www.moonloupe.com/.well-known/acme-challenge/fpzLz-y3_ei42E7b1tDnszZj5NUoXXChh3uVqQbVicc

But running the following right afterwards still failed:

# /opt/letsencrypt/letsencrypt-auto certonly --config /opt/letsencrypt/webroot.moonloupe.com.ini

The contents of /opt/letsencrypt/webroot.moonloupe.com.ini:

rsa-key-size = 4096
email = me@moonloupe.com
domains = moonloupe.com, www.moonloupe.com
authenticator = webroot
webroot-path = /var/www/moonloupe.com/html

The output:

Failed authorization procedure. moonloupe.com (http-01): urn:acme:error:connection :: The server could not connect to the client to verify the domain :: Could not connect to http://moonloupe.com/.well-known/acme-challenge/KY8MgM4ZMuxtBAaDOzHaq8BtKOIGv9wnhigftY12Jnk, www.moonloupe.com (http-01): urn:acme:error:connection :: The server could not connect to the client to verify the domain :: Could not connect to http://www.moonloupe.com/.well-known/acme-challenge/FyAkeXfOYkuBeTHVpeuLrfGI2Q0um0harnl2PR_OOGY

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: moonloupe.com
   Type:   urn:acme:error:connection
   Detail: Could not connect to http://moonloupe.com/.well-known/acme-
   challenge/KY8MgM4ZMuxtBAaDOzHaq8BtKOIGv9wnhigftY12Jnk

   Domain: www.moonloupe.com
   Type:   urn:acme:error:connection
   Detail: Could not connect to http://www.moonloupe.com/.well-known
   /acme-challenge/FyAkeXfOYkuBeTHVpeuLrfGI2Q0um0harnl2PR_OOGY
15

I’m not sure if this is due to your test server being down or something like that, but connecting to https://www.moonloupe.com/.well-known/acme-challenge/fpzLz-y3_ei42E7b1tDnszZj5NUoXXChh3uVqQbVicc doesn’t work for me.

If that’s just a temporary problem, I would suggest running the curl command again and then browsing to the URL mentioned in the last line (Location: ...). This should serve the challenge token.

16

You have a redirect, but it's not going to a valid location ( as @pfg says as well)

curl -I --location http://www.moonloupe.com/.well-known/acme-challenge/fpzLz-y3_ei42E7b1tDnszZj5NUoXXChh3uVqQbVicc

shows the two steps it tries, the redirect, and then the problem afterwards;

HTTP/1.1 301 Moved Permanently
Server: nginx/1.6.2
Date: Thu, 04 Feb 2016 08:32:26 GMT
Content-Type: text/html
Content-Length: 184
Connection: keep-alive
Location: https://www.moonloupe.com/.well-known/acme-challenge/fpzLz-y3_ei42E7b1tDnszZj5NUoXXChh3uVqQbVicc

curl: (35) Unknown SSL protocol error in connection to www.moonloupe.com:443

17

It doesn’t work. Your server does not respond properly on :443:

$ curl --insecure https://www.moonloupe.com/.well-known/acme-challenge/fpzLz-y3_ei42E7b1tDnszZj5NUoXXChh3uVqQbVicc
curl: (35) gnutls_handshake() failed: The TLS connection was non-properly terminated.
18

This is the funny thing. We first set moonloupe.com up to listen to port 80 only and were able to set up a certificate for it. We then changed the configuration to the one I shared above. All was good and we had the green pad lock. Then, when we ran the script again, LE failed, and – it took the site down. That is, there is an LE certificate in place (the one we generated the first time), but for some reason after we attempted to replace it, it’s no longer available to the site, and so the site is unavailable too. We actually had LE set up also on our main site – https://www.fashion.net, but when the automated update failed, we promptly reverted it back to COMODO. So right now, moonloupe.com actually is live, it’s just that LE prevents it from opening up.

19

I just had a look to see if the certificates had been corrupted, but they all look good. So the fact that a failed update makes the current certificates unavailable to the site seems to indicate that the path changed or the symlinks are broken. It must be a bug; a failed update surely shouldn’t bring down a site – it was a very scary experience. We set up a cron job for seven test sites and when the automated updates kicked in, they went offline in sequence. When that first happened, we had no idea what was going on. But after running the updates manually, we got the error messages that I posted when opening this thread.

20

Nginx generally refuses to start if certificates aren’t where they should be. There should at the very least be some kind of message in your error log indicating as much. The client also tries to avoid modifying any files unless renewal was successful.

Generally speaking, you might want to utilize --renew-by-default for renewal.

21

Thank you! Well, that confirms that Let's Encrypt breaks the path if it fails to update; really scary and it means we can't use Let's Encrypt on a production site.

We added that flag, but as you see here, it still failed to connect to https:

# /opt/letsencrypt/letsencrypt-auto certonly --renew-by-default --config /opt/letsencrypt/webroot.moonloupe.com.ini
Updating letsencrypt and virtual environment dependencies......
Requesting root privileges to run with virtualenv: /root/.local/share/letsencrypt/bin/letsencrypt certonly --renew-by-default --config /opt/letsencrypt/webroot.moonloupe.com.ini
Failed authorization procedure. www.moonloupe.com (http-01): urn:acme:error:connection :: The server could not connect to the client to verify the domain :: Could not connect to http://www.moonloupe.com/.well-known/acme-challenge/uy0J8fnDurVcv1WfVPJ_G8d-KK0KBbg1y-cCOtgF-fQ, moonloupe.com (http-01): urn:acme:error:connection :: The server could not connect to the client to verify the domain :: Could not connect to http://moonloupe.com/.well-known/acme-challenge/AWnz2PIftUe-qMW90JXAaoDMEbEEyCnoWp8dMT8TA_I

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: www.moonloupe.com
   Type:   urn:acme:error:connection
   Detail: Could not connect to http://www.moonloupe.com/.well-known
   /acme-challenge/uy0J8fnDurVcv1WfVPJ_G8d-KK0KBbg1y-cCOtgF-fQ

   Domain: moonloupe.com
   Type:   urn:acme:error:connection
   Detail: Could not connect to http://moonloupe.com/.well-known/acme-
   challenge/AWnz2PIftUe-qMW90JXAaoDMEbEEyCnoWp8dMT8TA_I