[Webroot] Only performs http-01 challenge which doesn't follow HTTP redirects to HTTPS site

As I’m unable to get an automated client running for Apache, as the Let’s Encrypt client is depending on some custom scripts from Debian (-based) distributions (and Gentoo doesn’t include those), I’m trying to run the client with the webroot plugin. And as I’m not needing the whole Apache-part, I thought to myself: I won’t be running the client as root. So far so good.

Except, I’m running this domain HTTPS-only: https://wiki.apache.org/httpd/RedirectSSL

Just a simple Redirect permanent and your site is HTTPS only. So far so good too! Yippy!

Except when it comes to Let’s Encrypt. The Webroot plugin insists on trying the domain validation via http-01:

The command I ran is:

letsencrypt certonly --verbose --config-dir ~/letsencrypt/ --logs-dir ~/letsencrypt/logs/ --webroot -w /var/www/vhosts/www.${domain}/htdocs/ -d ${domain} -d www.${domain}

The result:

Root (sudo) is required to run most of letsencrypt functionality.    
Starting new HTTPS connection (1): acme-v01.api.letsencrypt.org      
Generating key (4096 bits):                                          
/home/osiris/letsencrypt/keys/0005_key-letsencrypt.pem               
Creating CSR: /home/osiris/letsencrypt/csr/0005_csr-letsencrypt.pem  
Starting new HTTPS connection (1): acme-v01.api.letsencrypt.org      
Starting new HTTPS connection (1): acme-v01.api.letsencrypt.org      
Starting new HTTPS connection (1): acme-v01.api.letsencrypt.org      
Performing the following challenges:                                 
http-01 challenge for ${domain}
http-01 challenge for www.${domain}
Waiting for verification...                                          
Starting new HTTPS connection (1): acme-v01.api.letsencrypt.org      
Starting new HTTPS connection (1): acme-v01.api.letsencrypt.org      
Starting new HTTPS connection (1): acme-v01.api.letsencrypt.org      
Starting new HTTPS connection (1): acme-v01.api.letsencrypt.org
Reporting to user: The following 'urn:acme:error:connection' errors  
were reported by the server:                                         
Domains: ${domain}, www.${domain}
Error: The server could not connect to the client for DV

Unfortunately, everything with --tls-sni-01 seems to fail or not do anything… --standalone-supported-challenges tls-sni-01 doesn’t influence the Webroot plugin obviously.

My server does give a HTTP 301 redirect, but the ACME server doesn’t do anything with that aparently… Not according to my Apache logs nor according to tcpdump… No TLS connection what so ever.

Is this something by design or something that could be implemented in the future? Or am I missing a command line option? :worried:

Oh, by the way, webroot.py did have something about TLS in it’s history, but aparently it’s stripped away in the current master webroot.py… No TLS what so ever when I search for it :stuck_out_tongue:

We’re having the same issue. We force all HTTP connections to be redirected to HTTPS for certain domains.

The option would be turning off HTTPS redirect but that wouldn’t be secure, would it :wink:

I'm trying to exclude the path /.well-known/acme-challenge/ from the redirect, but I'm using the Redirect directive and I'm having difficulties with that.. An empty <Location "/.well-known/acme-challenge/"></Location> doesn't "cancel" out the Redirect set somewhere else I'm afraid..

Ofcourse, it would be rather simpel with mod_rewrite.. But I'm hoping it's possible without such a heavy mechanism for something rather.. simple.. :stuck_out_tongue:

RedirectMatch supports PCRE-style regex syntax, so you could do something like

RedirectMatch 301 ^(?!/\.well-known/acme-challenge/).* https://mydomain.example$0
2 Likes

I checked my configuration, as I have some sub domain using redirect permanent, and for me I did not encountered this problem.

So for me the letsencrypt challenger does follow redirection.

Did you check that your 443 port is reachable from the outside.

As @jmorahan already mentioned, if you configure your webserver to not only redirect HTTP-URLs to the https://yourdomain.com/ but to the exact path the user or in this case lets encrypt is requesting, it will succeed. We have several domains forcing the use of TLS that way and it worked like a charm.
Here would be (one) example of how to do that on nginx:
return 301 https://$server_name$request_uri;

2 Likes

Well, that seems to work indeed :smiley:

Never quite understood regexes, always a pain in the ()#$()#* when I want to implement one.. For example, I had no idea what the (?!foo) syntax was.. Apparently this is something called negative lookahead. I recon a fancy PCRE regex substitution for a 'logical not' or something :stuck_out_tongue:

Ah well, as long as it works.. Let's see what the Let's Encrypt client thinks :smile:

Hmm, removed the “Solution”-mark, because I’m interested in a discussion about whether or not the client should support TLS anyway and perhaps the mentioned ‘solution’ is just a workaround :wink:

The acme server when performing http-01 challenge do follow http redirects to https.

So I think that you may have a broken redirect, a redirect to another host, or maybe some other configuration which hide the challenge folder in your apache config files.

Check all logs of apache usually in /var/log/apache2

Ofcourse I've checked all kinds of logs, even tcpdump: no TLS incoming connections were made what so ever. And the redirect works, I've checked that also ofcourse, manually and in the browser..

Didn't see your previous post, sorry: yes, my server is accessible from the outside world on port 443, both IPv4 and IPv6 :wink:

I'm going to accept the aformentioned solution as a solution anyway, as the SimpleHTTPS challenge for the Webroot plugin was removed for a reason, as I found out from this post: Preventing Letsencrypt 3rd party clients going the Android way? - #29 by My1
Seems to be a vulnerability with default TLS vhosts and SNI: that way a misconfigured host apparently could 'steal' the domain validation or something..

This is pretty strange, I just did a validation with RedirectPermanent activated against the live server and here are my log:

letsencrypt.example.com:80 66.133.109.36 - - [04/Dec/2015:22:32:14 +0000] "GET /.well-known/acme-challenge/VioViw79I_0eLxhK1WGPBxK-XCPw-OWSjGA_sC6mo2g HTTP/1.1" 301 675 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
letsencrypt.example.com:443 66.133.109.36 - - [04/Dec/2015:22:32:15 +0000] "GET /.well-known/acme-challenge/VioViw79I_0eLxhK1WGPBxK-XCPw-OWSjGA_sC6mo2g HTTP/1.1" 200 2505 "http://letsencrypt.example.com/.well-known/acme-challenge/VioViw79I_0eLxhK1WGPBxK-XCPw-OWSjGA_sC6mo2g" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"

Precision: the web server is configured to serve self-signed certificate, with name mismatch.

If you see no tls connection with tcpdump, this make me think of some firewall interfering.

With what letsencrypt command line did you manage that? :open_mouth:

Because I’m very, very sure my server is accessible by TLS… Otherwise SSLLabs wouldn’t give it an A+ I recon :wink:

Well the usual letsencrypt command:

path_to_letsencrypt -d letsencrypt.example.com
I performed a letsencrypt-auto -v, to check if my version was up to date.

with this config file:

# 

rsa-key-size = 4096

# Now live so default is this server.
#server = https://acme-v01.api.letsencrypt.org/directory

email = admin@example.com
agree-tos = true

# Do not use ugly curse interface
text = true

# Webroot challenge
authenticator = webroot
webroot-path = /var/www/letsencrypt

Apart some weird blacklist on 443 of the acme server I’m out of idea.
There might be strange behaviour with ipv6 and ipv4, but I highly doubt that. My server is ipv4 only.

Le Uses Firefox? LOL!

With the risk feeding the troll: https://en.wikipedia.org/wiki/User_agent#Format_for_human-operated_web_browsers

since when is the LE browser human operated? LE is CLEARLY automatic.

also here it says Mozilla was PREVIOUSLY used to indicate compatibility with it, rather than just stating “I am compatible with HTML5” or whatever, what would be more usefulthe browser makers are at fault on their own using Mozilla in the agent just to get around the issues they caused themselves is plain cheating, and still it is previous so each browser should be honest, and mozilla be only denoted in actual browsers of Mozilla.

Sure sure… But even if your point is sound, practically every bot out there uses this scheme… And it’s quite off topic… So… :confused:

As I’m getting the same error I’d be interested what apache configuration entry triggers that kind of error. I understand that the workaround works. It does for me too. But I have no indication that my configuration could be somehow misconfigured. I’m also using 301 redirects.

Here I created a certificate without having a redirect enabled, that works flawlessly: https://www.ssllabs.com/ssltest/analyze.html?d=interndemo.sfg-singen.de

However, when I enable the redirect I get the error again.

In the apache error log I can only find the entry:
[Wed Dec 16 12:41:08.152193 2015] [core:info] [pid 10075] [client 66.133.109.36:59143] AH00128: File does not exist: /var/www/letsencrypt/DK1Kwm_ZlZPDXNg7AnB2bWu5Iyh9nwQVGzkNRHMn9k0

But when I check the directory with watch -n .1 ls I can see the file is created and is removed as soon as LE throws the error.

I created a test file which is accessible:
https://interndemo.sfg-singen.de/.well-known/acme-challenge/test

http://interndemo.sfg-singen.de/.well-known/acme-challenge/test

However, with http it gets redirected to http://interndemo.sfg-singen.de.well-known/acme-challenge/test

I have no idea why :frowning:

both http and https go to the correct https version for me. Do you need to clear your local cache when checking ? or have you already corrected the issue ?

This is not a solution and only a workaround. If it doesn’t work, your redirect doesn’t work properly.