HTTP verification failure for a single domain

[onlime]

This issue only occurs on cert issuance for eglijona.ch since 2018-01-11 07:01AM - it works for every other domain. First suspected this to be related to the ACME TLS-SNI-01 verification lockdown, but we were never using TLS-SNI-01, always using HTTP verification instead.

domain: eglijona.ch

I ran this command: certbot certonly -t -n --webroot -w /var/www/html/ -d eglijona.ch -d www.eglijona.ch

It produced this output:

$ certbot certonly -t -n --webroot -w /var/www/html/ -d eglijona.ch -d www.eglijona.ch
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for eglijona.ch
http-01 challenge for www.eglijona.ch
Using the webroot path /var/www/html for all unmatched domains.
Waiting for verification...
Cleaning up challenges
Failed authorization procedure. eglijona.ch (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: The key authorization file from the server did not match this challenge [3vTyU919pHT_g445_bVy1Kd9ECp7CCo5jQmyu29UwPY.0yNq4ojWJuCW7vur81eSwkdH-XOdbv0sMv_exGcLoeM] != []

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: eglijona.ch
   Type:   unauthorized
   Detail: The key authorization file from the server did not match
   this challenge
   [3vTyU919pHT_g445_bVy1Kd9ECp7CCo5jQmyu29UwPY.0yNq4ojWJuCW7vur81eSwkdH-XOdbv0sMv_exGcLoeM]
   != []

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A record(s) for that domain
   contain(s) the right IP address.

during HTTP verification, the following challenges are created and correctly pruned afterwards (seems a bit weird that there were 2 challenges):

$ tree /var/www/html/.well-known/
/var/www/html/.well-known/
└── acme-challenge
    ├── 3vTyU919pHT_g445_bVy1Kd9ECp7CCo5jQmyu29UwPY
    └── 8VX3ofkYzi8dasuUTqnsYr4VA2LzzqG6cLN6h6YGmJQ

the domains correctly resolve to the webserver:

$ dig +noall +answer eglijona.ch @8.8.8.8
eglijona.ch.		299	IN	A	46.231.201.209

$ dig +noall +answer www.eglijona.ch @8.8.8.8
www.eglijona.ch.	295	IN	CNAME	web.onlime.ch.
web.onlime.ch.		1795	IN	A	46.231.201.209

certbot delete --cert-name eglijona.ch doesn’t help. In /etc/letsencrypt/ there are no certs/keys for eglijona.ch, also checked whole content with recursive grep. Not even /etc/letsencrypt/renewal/eglijona.ch.conf exists.

@josh helped me out here (thx!) but issue could not yet be resolved.

My web server is: Apache/2.4.25 (Debian)

Certbot version: 0.10.2

My hosting provider: Onlime Webhosting - https://www.onlime.ch

The operating system my web server runs on is: Debian Stretch 9.3

I can login to a root shell on my machine: yes

I’m using a control panel to manage my site: no

[Osiris]

This might be due to IPv6. Your ‘base’ domain has an AAAA record configured:

;; ANSWER SECTION:
eglijona.ch.		254	IN	AAAA	2a00:d70:0:b:2002:0:d91a:3244

Your CNAME for www.eglijona.ch doesn’t.

When you try to access a /.well-known/acme-challenge/ resource through IPv6, the server (always) responds with an empty HTTP/1.1 200 OK response:

osiris@desktop ~ $ curl -Lv eglijona.ch/.well-known/acme-challenge/werwewqeqwewqfsd
*   Trying 2a00:d70:0:b:2002:0:d91a:3244...
* Connected to eglijona.ch (2a00:d70:0:b:2002:0:d91a:3244) port 80 (#0)
> GET /.well-known/acme-challenge/werwewqeqwewqfsd HTTP/1.1
> Host: eglijona.ch
> User-Agent: curl/7.49.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Date: Sat, 13 Jan 2018 12:11:06 GMT
< Server: Apache/2.4
< Upgrade: h2,h2c
< Connection: Upgrade
< Transfer-Encoding: chunked
< Content-Type: text/html; charset=UTF-8
< 
* Connection #0 to host eglijona.ch left intact
osiris@desktop ~ $

Through IPv4 however you get a file not found (as we would expect):

osiris@desktop ~ $ curl -4Lv eglijona.ch/.well-known/acme-challenge/werwewqeqwewqfsd
*   Trying 46.231.201.209...
* Connected to eglijona.ch (46.231.201.209) port 80 (#0)
> GET /.well-known/acme-challenge/werwewqeqwewqfsd HTTP/1.1
> Host: eglijona.ch
> User-Agent: curl/7.49.0
> Accept: */*
> 
< HTTP/1.1 404 Not Found
< Date: Sat, 13 Jan 2018 12:11:13 GMT
< Server: Apache/2.4
< Transfer-Encoding: chunked
< Content-Type: text/html; charset=UTF-8
< 
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="utf-8"/>
    <meta http-equiv="X-UA-Compatible" content="IE=edge"/>
    <meta name="viewport" content="width=device-width, initial-scale=1"/>
    <meta name="robots" content="noindex"/>
    <title>404 Not Found</title>
(...)

Strangely enough, the above problem doesn’t affect surfing to eglijona.ch through IPv6… Only for eglijona.ch/.well-known/acme-challenge/.

[onlime]

Great, @Osiris! That did the trick. We removed the IPv6 AAAA record and HTTP verification run through fine.
Thanks a lot!