Challenge fails due to broken IPv6 connectivity & enabled 301 redirect

[npurt]

My domain is: letsencrypt.8443.ml

I ran this command: issue a certificate via Plesk’s extension

It produced this output:
Domain validation failed for letsencrypt.8443.ml: Invalid response from https://acme-v02.api.letsencrypt.org/acme/authz-v3/1292421841.
Details:
Type: urn:ietf:params:acme:error:connection
Status: 400
Detail: Fetching https://letsencrypt.8443.ml/.well-known/acme-challenge/B0jCehyH7blNhKwZXZ6cehf8E1ChG--jFrZztqUl1Hw: Timeout during connect (likely firewall problem)
My web server is (include version): Server version: Apache/2.4.18 (Ubuntu)

The operating system my web server runs on is (include version): Ubuntu 16.04

My hosting provider, if applicable, is: n/a

I can login to a root shell on my machine (yes or no, or I don’t know): yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel): Plesk 17.8.11

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you’re using Certbot): Let’s Encrypt extension 2.8.4-547

Could you please explain the following:

I have AAAA DNS record for the domain, and IPv6 doesn’t work in both cases, but:

1. There’s 301 redirect configured (http -> https), and the challenge fails. No certificate was issued with a quite unclear error. https://acme-v02.api.letsencrypt.org/acme/authz-v3/1292421841
2. There’re no redirects, and the challenge passes successfully. The certificate is issued, even though the token still could not be obtained via IPv6. https://acme-v02.api.letsencrypt.org/acme/authz-v3/1292440928

Why does any redirect affect the challenge so much? Could the error be a little bit more clear and point somehow to IPv6 issues? Because the token is fetched without any errors via a browser, and it confuses.

[rg305]

It seems the IPv6 address is causing the break.
[LE prefers IPv6 over IPv4]

wget -4 http://letsencrypt.8443.ml/ <works>
wget -4 https://letsencrypt.8443.ml/ <works>
wget -6 http://letsencrypt.8443.ml/ <fails>
wget -6 https://letsencrypt.8443.ml/ <fails>

I don’t see a redirect now, so I can’t troubleshoot that.

[JuergenAuer]

Hi @npurt

@npurt

My domain is: letsencrypt.8443.ml

checking your domain you have created a new certificate - https://check-your-website.server-daten.de/?q=letsencrypt.8443.ml

Issuer	not before	not after	Domain names	LE-Duplicate	next LE
Let's Encrypt Authority X3	2019-11-18	2020-02-16	letsencrypt.8443.ml
1 entries	duplicate nr. 1

And you use it.

CN=letsencrypt.8443.ml
	18.11.2019
	16.02.2020
expires in 90 days	letsencrypt.8443.ml - 1 entry

Ipv6 is defined and doesn't work, that will block http validation. But now you have 90 days to fix that.

[cpu]

Hi @npurt, welcome to the community forum

@npurt

Could you please explain the following:

I have AAAA DNS record for the domain, and IPv6 doesn’t work in both cases, but:

1. There’s 301 redirect configured (http -> https), and the challenge fails . No certificate was issued with a quite unclear error. https://acme-v02.api.letsencrypt.org/acme/authz-v3/1292421841
2. There’re no redirects, and the challenge passes successfully. The certificate is issued, even though the token still could not be obtained via IPv6. https://acme-v02.api.letsencrypt.org/acme/authz-v3/1292440928

Why does any redirect affect the challenge so much? Could the error be a little bit more clear and point somehow to IPv6 issues? Because the token is fetched without any errors via a browser, and it confuses.

Interesting timing. I just this morning closed an issue on the Boulder repo with an explanation for this same problem. Unfortunately this is a limitation with Boulder's implementation of happy eyeballs/fast fallback that we've accepted as a wont-fix for the time being. The short of it is that IPv6 to IPv4 fallback doesn't happen for redirected HTTP-01 validation requests.

The linked issue has three possible approaches to fixing the problem but I'll duplicate it here for convenience. I think there are three workarounds available here and the third might be interesting for you to consider:

1. Removing the AAAA record from your authoritative DNS servers.
2. Making the expected HTTP-01 key authorization available via both IPv4 and IPv6
3. Not redirecting the HTTP-01 challenge request from :80 to :443
   Option 3 is less obvious than the other two but might be easier for you to consider implementing. Our IPv6 to IPv4 fallback will work reliably on the initial connection. The failure to fallback from IPv6 to IPv4 that happens on :443 is a result of it being during the redirect processing stage.

RFC 8555 mandates HTTP-01 challenges arrive on :80 so we can't make the initial connection to :443 but you could drop the redirect from your config and I I believe we will end up reading the key-authorization from IPv4:443.

[npurt]

Hi @cpu , thank you for the detailed reply.
Unfortunately, disabling all the possible redirects (http>https, non-www>www, etc.) is not an option for customers So, at least some clear error message would be great. Will think about it…

[rg305]

@npurt

Unfortunately, disabling all the possible redirects (http>https, non-www>www, etc.) is not an option for customers

You don't have to disable ALL redirects.
Only the ones going to the /.well-known/acme-challenge/ folder.
The implementation of the redirect exclusion it simple and straightforward in both Apache and NGINX.

[npurt]

Oh, I see. Thank you for the clarification I’ll check it.

[cpu]

Heya @npurt,

@npurt

Unfortunately, disabling all the possible redirects (http>https, non-www>www, etc.) is not an option for customers

@rg305's reply captures what I was thinking here. It would be a matter of only disabling the redirect for the specific HTTP-01 validation requests (e.g. by checking the request path).

@npurt

at least some clear error message would be great. Will think about it…

I'll give this some thought too. For the same reasons it is difficult to fallback to IPv4 during a redirect with Go's standard library it may be difficult to make an error message that calls out the problem directly.