HTTP-01 not re-issuing; verification not following redirects

My domain is: fantasysurvivorgame.com
I ran this command: /root/.acme.sh/acme.sh --cron --home "/root/.acme.sh"
It produced this output: Invalid status, fantasysurvivorgame.com:Verify error detail:50.116.31.29: Fetching http://www.fantasysurvivorgame.com/.well-known/acme-challenge/pMa501a85DERlW149xt1aTqcODkbSIihZkbBaf_u9ro: Timeout during connect (likely firewall problem)
My web server is (include version): Apache/2.4.56
The operating system my web server runs on is (include version): Debian 11.8
My hosting provider, if applicable, is: Linode
I can login to a root shell on my machine (yes or no, or I don't know): yes
I'm using a control panel to manage my site (no, or provide the name and version of the control panel): no
The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): acme.sh 3.0.7

Been using Let's Encrypt for years, but I must have screwed something up since I'm getting errors on the re-issue/renewal now.
I can see the Let's Encrypt validation servers hit .well-known/acme-challenge/pM.... and my server responds with a 301 redirect to the www subdomain, but I don't see it try after that.

54.184.136.46 - - [23/Oct/2023:19:37:29 +0000] "GET /.well-known/acme-challenge/pMa501a85DERlW149xt1aTqcODkbSIihZkbBaf_u9ro HTTP/1.1" 301 707 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)" 18.116.87.230 - - [23/Oct/2023:19:37:29 +0000] "GET /.well-known/acme-challenge/pMa501a85DERlW149xt1aTqcODkbSIihZkbBaf_u9ro HTTP/1.1" 301 707 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)" 23.178.112.107 - - [23/Oct/2023:19:37:29 +0000] "GET /.well-known/acme-challenge/pMa501a85DERlW149xt1aTqcODkbSIihZkbBaf_u9ro HTTP/1.1" 301 707 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"

According to this doc https://letsencrypt.org/docs/challenge-types/ it should follow the redirect (nothing on non-standard ports). So I'm a little stumped as to why it's not connecting.

The JSON response from the acme-v02.api.letsencrypt.org server includes:

    "type": "http-01",
    "status": "invalid",
    "error": {
        "type": "urn:ietf:params:acme:error:connection",
        "detail": "50.116.31.29: Fetching http://www.fantasysurvivorgame.com/.well-known/acme-challenge/pMa501a85DERlW149xt1aTqcODkbSIihZkbBaf_u9ro: Timeout during connect (likely firewall problem)",
        "status": 400
    }

I've dropped a test file at http://fantasysurvivorgame.com/.well-known/acme-challenge/test
Should be available on both IPv4 and IPv6, so I'm confused as to the error "Timeout during connect (likely firewall problem)"
You'll see the redirect to www, is that not supported any longer?

The IPv6 connections are not working. Double-check the public IP you have in the DNS AAAA record. You can do this to see what your server has

curl -4 https://ifconfig.io
curl -6 https://ifconfig.io

We actually saw one like this recently and the format of the error messages make it confusing.

You seem somewhat skilled so I'll describe ...

  1. Let's Encrypt server makes an IPv6 HTTP challenge to http://(domain)/.well-known...
  2. The IPv6 connect fails and Let's Encrypt server retries on IPv4
  3. IPv4 request successfully reaches your server http://(domain)/.well-known...
  4. Your server redirects to the www domain (perfectly fine)
  5. Let's Encrypt server follows the redirect and tries it on IPv6
  6. But, IPv6 fails (predictably, again)
  7. Challenge fails

Why does the challenge fail?
Because the IPv4 fallback only happens with the first request. After any redirect only IPv6 will be tried if the AAAA record exists.

Why does the IPv4 address show in the error message?
I don't know. Call it a quirk.

Sample request from my own server:

curl -I6 -m6 http://fantasysurvivorgame.com/.well-known/acme-challenge/Test404
curl: (28) Failed to connect to fantasysurvivorgame.com port 80 after 3008 ms:
Connection timed out

curl -I4 -m6 http://fantasysurvivorgame.com/.well-known/acme-challenge/Test404
HTTP/1.1 301 Moved Permanently
Date: Mon, 23 Oct 2023 21:08:08 GMT
Server: Apache/2.4.56 (Debian)
Location: http://www.fantasysurvivorgame.com/.well-known/acme-challenge/Test404
4 Likes

Hello @MidSpeck, welcome to the Let's Encrypt community. :slightly_smiling_face:

Your IPv6 address is being filtered on Ports 80 & 443.
https://letsdebug.net/www.fantasysurvivorgame.com/1648607

AAAANotWorking
Error
www.fantasysurvivorgame.com has an AAAA (IPv6) record (2600:3c00::f03c:93ff:fee9:b863) but a test request to this address over port 80 did not succeed. Your web server must have at least one working IPv4 or IPv6 address. You should either ensure that validation requests to this domain succeed over IPv6, or remove its AAAA record.
A timeout was experienced while communicating with www.fantasysurvivorgame.com/2600:3c00::f03c:93ff:fee9:b863: Get "http://www.fantasysurvivorgame.com/.well-known/acme-challenge/letsdebug-test": context deadline exceeded

Trace:
@0ms: Making a request to http://www.fantasysurvivorgame.com/.well-known/acme-challenge/letsdebug-test (using initial IP 2600:3c00::f03c:93ff:fee9:b863)
@0ms: Dialing 2600:3c00::f03c:93ff:fee9:b863
@10000ms: Experienced error: context deadline exceeded 
>nmap -4 -Pn -p80,443 fantasysurvivorgame.com
Starting Nmap 7.94 ( https://nmap.org ) at 2023-10-23 21:06 UTC
Nmap scan report for fantasysurvivorgame.com (50.116.31.29)
Host is up (0.054s latency).
Other addresses for fantasysurvivorgame.com (not scanned): 2600:3c00::f03c:93ff:fee9:b863
rDNS record for 50.116.31.29: mail.rootmanager.com

PORT    STATE SERVICE
80/tcp  open  http
443/tcp open  https

Nmap done: 1 IP address (1 host up) scanned in 0.76 seconds
>nmap -6 -Pn -p80,443 fantasysurvivorgame.com
Starting Nmap 7.94 ( https://nmap.org ) at 2023-10-23 21:06 UTC
Nmap scan report for fantasysurvivorgame.com (2600:3c00::f03c:93ff:fee9:b863)
Host is up.
Other addresses for fantasysurvivorgame.com (not scanned): 50.116.31.29
rDNS record for 2600:3c00::f03c:93ff:fee9:b863: mail.rootmanager.com

PORT    STATE    SERVICE
80/tcp  filtered http
443/tcp filtered https

Nmap done: 1 IP address (1 host up) scanned in 4.15 seconds
2 Likes

Thank you both!
Indeed my IPv6 address did change since the last renewal and I see I failed to get it 100% updated. I'll get this fixed on my end.
Thank you for the detailed explanations and the quick reply!

4 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.