Certbot - TLS-SNI Apache Challenge Doesn't Pass - Alternatives

Hello,
I try to issue SSL certificate uses certbot on server_1 to move it on server_2.
On server_2 there is open SSL certificate for local domain.
When i try to issue certificate on server_2 i get error “Certbot has problem setting up the virtual environment. We were not be able to guess the right solution from your pip output.”.
So, i decided to issue certificate on server_1 and move it to server_2.
Please, tell me what i do wrong.

My domain is:
mydomain.com

I ran this command:
./certbot-auto --apache certonly -d dominos-wifi.bsh.ru

It produced this output:
Failed authorization procedure. mydomain.com (tls-sni-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Incorrect validation certificate for tls-sni-01 challenge. Requested a3cafa5f780d237c71b21ae0de4cb909.7382aae0eccbdd887dd6b3362d1cd827.acme.invalid from MY_IP:443. Received 1 certificate(s), first certificate had names “mydomain0.local”

IMPORTANT NOTES:

  • The following errors were reported by the server:

    Domain: mydomain.com
    Type: unauthorized
    Detail: Incorrect validation certificate for tls-sni-01 challenge.
    Requested
    a3cafa5f780d237c71b21ae0de4cb909.7382aae0eccbdd887dd6b3362d1cd827.acme.invalid
    from MY_IP:443. Received 1 certificate(s), first certificate
    had names “mydomain0.local”

    To fix these errors, please make sure that your domain name was
    entered correctly and the DNS A record(s) for that domain
    contain(s) the right IP address.

My web server is (include version):
Apache/2.2.16 (Debian)
The operating system my web server runs on is (include version):
Debian
I can login to a root shell on my machine (yes or no, or I don’t know):
yes
I’m using a control panel to manage my site (no, or provide the name and version of the control panel):
no

Hi @sergei,

certbot --apache cannot support this use case. It can only be run directly on the server that the domain name points to. The error you saw is a typical error when running certbot --apache on a server other than the server the domain name is pointed to.

A different verification method can work in this case. If server_1 and server_2 both answer HTTP on port 80, you can make it send an HTTP 301 redirect from http://server_2/.well_known/acme_challenge/ to http://server_1/.well-known/acme_challenge/. Then you can pass the HTTP-01 challenge on server_1 on behalf of server_2 using certbot --webroot, specifying the directory from which server_1's web content is served with -w.

Alternatively, you can use the DNS-01 challenge if you can modify your DNS zone via an API. The best current support for that method is provided in the acme.sh client, which has a large library of DNS provider APIs.

Certbot can also do this but currently supports many fewer DNS provider APIs via scripts, compared to acme.sh.

however there are several PRs in certbot for DNS providers based on the lexicon library

not sure what their ETA is

Andrei

1 Like

When i run on the server_1:
./certbot-auto certonly --webroot -w /var/www/html/

I give that error:
/root/.local/share/letsencrypt/lib/python2.6/site-packages/cryptography/init.py:26: DeprecationWarning: Python 2.6 is no longer supported by the Python core team, please upgrade your Python. A future version of cryptography will drop support for Python 2.6
DeprecationWarning
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Please enter in your domain name(s) (comma and/or space separated) (Enter ‘c’ to cancel):mydomain.com
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for mydomain.com
Using the webroot path /var/www/html for all unmatched domains.
Waiting for verification…
Cleaning up challenges
Unable to clean up challenge directory /var/www/html/.well-known/acme-challenge
Failed authorization procedure. mydomain.com (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://mydomain.com/.well-known/acme-challenge/eR5r9YQV52syOE6VVh3vWUqilq_0SIwCqQ6_oaWvuVc: "

HTML code of 404 page from server_2.

IMPORTANT NOTES:

  • The following errors were reported by the server:

Domain: mydomain.com
Type: unauthorized
Detail: Invalid response from
http://mydomain.com/.well-known/acme-challenge/eR5r9YQV52syOE6VVh3vWUqilq_0SIwCqQ6_oaWvuVc:

HTML code of 404 page from server_2.

To fix these errors, please make sure that your domain name was
entered correctly and the DNS A record(s) for that domain
contain(s) the right IP address.

Hi @sergei,

Did you first follow the suggestion to make server_2 send a 301 redirect message for this path? It sounds like that part isn’t working or isn’t in place yet.

Hello @schoen

On both my servers run apache2.
I set redirect from server_2 to server_1.

Server_2 apache conf:

<VirtualHost *:80>
	Redirect 301 /.well-known/acme-challenge/ http://(server_1 IP)/.well-known/acme-challenge/
...
</VirtualHost>

When i open http://(server_2 IP)/.well-known/acme-challenge/example.txt in my browser it’s redirect to http://(server_1 IP)/.well-known/acme-challenge/example.txt,
and i see content of file from server_1.

Hi @sergei,

Let's Encrypt won't follow the redirection if it is an ip, it should be a resolvable domain name.

Cheers,
sahsanu

Hello @sahsanu,
Sorry, i made a mistake.
server_2 available from internet by domain name mydomain.com.

I mean, if your domain for server2 is: server2.example.com and your ip for server1 is: 1.2.3.4, when Let's Encrypt tries to reach your domain http://server2.example.com/.well-known/acme-challenge/whatever and it finds a 301 redirect to http://1.2.3.4/.well-known/acme-challenge/whatever it won't follow that redirection because Let's Encrypt doesn't follow ips, it follows resolvable domains.

You must change your redirection, instead of using the ip of server1 you should use a resolvable domain for server1 pointing to the ip 1.2.3.4

Redirect 301 /.well-known/acme-challenge/ http://server1.domain.tld/.well-known/acme-challenge/

Cheers,
sahsanu

Hi @sahsanu, do you have some reference for this restriction? I don’t remember encountering that limitation before.

Hi @schoen,

I didn’t find any official reference but the other day, @mnordhoff pointed me out this behaviour Certbot error while obtaining certificate using webroot plugin

I did a test and I got the same conclusion… just to be sure I’ve made another test a few minutes ago.

Two servers, server 1 serves domain redirect.27a.net (163.172.166.61) and it has a redirect rule pointing /.well-known/acme-challenge/ requests to another server (server 2) using an ip (198.98.49.246) as redirection.

$ curl -ikL http://redirect.27a.net/.well-known/acme-challenge/test
HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Tue, 30 May 2017 21:49:49 GMT
Content-Type: text/html
Content-Length: 178
Connection: keep-alive
Location: http://198.98.49.246/.well-known/acme-challenge/test

HTTP/1.1 200 OK
Server: nginx/1.12.0
Date: Tue, 30 May 2017 21:49:49 GMT
Content-Type: application/octet-stream
Content-Length: 26
Last-Modified: Tue, 30 May 2017 21:40:58 GMT
Connection: keep-alive
ETag: "592de6ea-1a"
Accept-Ranges: bytes

server 2 - challenge test

And now, I tried to get a certificate (using --dry-run) from server 2.

./certbot-auto certonly --webroot -w /usr/share/nginx/html/ -d redirect.27a.net --register-unsafely-without-email --dry-run
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for redirect.27a.net
Using the webroot path /usr/share/nginx/html for all unmatched domains.
Waiting for verification...
Cleaning up challenges
Unable to clean up challenge directory /usr/share/nginx/html/.well-known/acme-challenge
Failed authorization procedure. redirect.27a.net (http-01): urn:acme:error:connection :: The server could not connect to the client to verify the domain :: Could not connect to 198.98.49.246

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: redirect.27a.net
   Type:   connection
   Detail: Could not connect to 198.98.49.246

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A record(s) for that domain
   contain(s) the right IP address. Additionally, please check that
   your computer has a publicly routable IP address and that no
   firewalls are preventing the server from communicating with the
   client. If you're using the webroot plugin, you should also verify
   that you are serving files from the webroot path you provided.
 - Your account credentials have been saved in your Certbot
   configuration directory at /etc/letsencrypt. You should make a
   secure backup of this folder now. This configuration directory will
   also contain certificates and private keys obtained by Certbot so
   making regular backups of this folder is ideal.

Here is the letsencrypt.log https://gist.github.com/sahsanu/f0993ac953dd1beee5e780f8e250bcd4

So, we can’t get our certificate.

Now, instead of using the ip 198.98.49.246 for the redirect, I’ve used a domain pointing to the same ip 198.98.49.246.dnsip.xyz

$ curl -ikL http://redirect.27a.net/.well-known/acme-challenge/test
HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Tue, 30 May 2017 21:53:08 GMT
Content-Type: text/html
Content-Length: 178
Connection: keep-alive
Location: http://198.98.49.246.dnsip.xyz/.well-known/acme-challenge/test

HTTP/1.1 200 OK
Server: nginx/1.12.0
Date: Tue, 30 May 2017 21:53:09 GMT
Content-Type: application/octet-stream
Content-Length: 26
Last-Modified: Tue, 30 May 2017 21:40:58 GMT
Connection: keep-alive
ETag: "592de6ea-1a"
Accept-Ranges: bytes

server 2 - challenge test

So lets try to get our cert again from server 2.

./certbot-auto certonly --webroot -w /usr/share/nginx/html/ -d redirect.27a.net --register-unsafely-without-email --dry-run
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for redirect.27a.net
Using the webroot path /usr/share/nginx/html for all unmatched domains.
Waiting for verification...
Cleaning up challenges
Unable to clean up challenge directory /usr/share/nginx/html/.well-known/acme-challenge

IMPORTANT NOTES:
 - The dry run was successful.

Here is the letsencrypt.log https://gist.github.com/sahsanu/773de6c00958107b1fdb7e31c5985a2c

So, this time it validated correctly and I could get my cert ;).

If you check the logs, when using the ip redirection we get this:

"validationRecord": [
        {
          "url": "http://198.98.49.246/.well-known/acme-challenge/4Zl8HupWU_1bE6Hg3LsG2thRo3aH1umKRHd_b-ZgDUE",
          "hostname": "198.98.49.246",
          "port": "80",
          "addressesResolved": [],
          "addressUsed": "",
          "addressesTried": []
        },
        {
          "url": "http://redirect.27a.net/.well-known/acme-challenge/4Zl8HupWU_1bE6Hg3LsG2thRo3aH1umKRHd_b-ZgDUE",
          "hostname": "redirect.27a.net",
          "port": "80",
          "addressesResolved": [
            "163.172.166.61"
          ],
          "addressUsed": "163.172.166.61",
          "addressesTried": []
        }

So it can’t resolve the ip "addressesResolved": [], "addressUsed": "", and seems Let’s Encrypt needs to resolve all redirections.

Using the domain in the redirection, we get this:

"validationRecord": [
        {
          "url": "http://198.98.49.246.dnsip.xyz/.well-known/acme-challenge/5dwvKomtuF-VLXWjXYviOvIbhDSM-I5TiqCxYfoyi1Q",
          "hostname": "198.98.49.246.dnsip.xyz",
          "port": "80",
          "addressesResolved": [
            "198.98.49.246"
          ],
          "addressUsed": "198.98.49.246",
          "addressesTried": []
        },
        {
          "url": "http://redirect.27a.net/.well-known/acme-challenge/5dwvKomtuF-VLXWjXYviOvIbhDSM-I5TiqCxYfoyi1Q",
          "hostname": "redirect.27a.net",
          "port": "80",
          "addressesResolved": [
            "163.172.166.61"
          ],
          "addressUsed": "163.172.166.61",
          "addressesTried": []
        }

So in this case it can resolve the redirection and will use that info to follow it "addressesResolved": ["198.98.49.246"], addressUsed": "198.98.49.246",

I don’t know the reason for this behaviour but I suppose there is some good reason ;).

Cheers,
sahsanu

@jsha, could you confirm whether this is intended behavior for Boulder? (forbidding the target of an HTTP redirection to be a URL containing an IP address when validating with HTTP-01)

It’s expected, but not strictly intentional. It’s a side effect of the fact that Boulder looks up each hostname in a redirect chain so it can record the IP addresses looked up and which one was used.

1 Like

Thanks. I won’t press to have it changed because this is the first time I’ve ever seen it affect someone, so I don’t think it represents a big obstacle for most users’ ability to get certificates.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.