We're attempting to create a new certificate for a customer domain hosted on the Discourse infrastructure and receiving the dreaded error:
urn:ietf:params:acme:error:connection Fetching https://api.discourse.org/api/ssl_challenges?hostname=(redacted)&filename=/.well-known/acme-challenge/LpIZOn6UjHjtir408rTfu2VLyxIOZXuxsk0ycndwV0Y: Timeout during connect (likely firewall problem)
However, we can confirm that three successful retrievals of the challenge secret were made by
Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org) from the system hosting the challenge response:
|Apr 5, 2021 @ 17:31:50.094||2a05:d014:3ad:701:d969:e08f:1bb9:62bd||hostname=(redacted)&filename=/.well-known/acme-challenge/LpIZOn6UjHjtir408rTfu2VLyxIOZXuxsk0ycndwV0Y||200|
|Apr 5, 2021 @ 17:31:49.897||2600:1f16:269:da01:367:cea2:153a:d5c8||hostname=(redacted)&filename=/.well-known/acme-challenge/LpIZOn6UjHjtir408rTfu2VLyxIOZXuxsk0ycndwV0Y||200|
|Apr 5, 2021 @ 17:31:49.588||2600:1f14:804:fd02:1be3:bfea:ffcc:a21f||hostname=(redacted)&filename=/.well-known/acme-challenge/LpIZOn6UjHjtir408rTfu2VLyxIOZXuxsk0ycndwV0Y||200|
Since Let's Encrypt performs multiple perspective validation, I'm inferring that 1/4 of the challenges failed.
Looking at the logs for the actual webserver hosting the domain but not the LE challenge response, it seems that it was this last one coming from 220.127.116.11:
|Apr 5, 2021 @ 17:32:02.456||18.104.22.168||/.well-known/acme-challenge/LpIZOn6UjHjtir408rTfu2VLyxIOZXuxsk0ycndwV0Y||301|
|Apr 5, 2021 @ 17:31:49.585||2a05:d014:3ad:701:d969:e08f:1bb9:62bd||/.well-known/acme-challenge/LpIZOn6UjHjtir408rTfu2VLyxIOZXuxsk0ycndwV0Y||301|
|Apr 5, 2021 @ 17:31:49.430||2600:1f16:269:da01:367:cea2:153a:d5c8||/.well-known/acme-challenge/LpIZOn6UjHjtir408rTfu2VLyxIOZXuxsk0ycndwV0Y||301|
|Apr 5, 2021 @ 17:31:49.354||2600:1f14:804:fd02:1be3:bfea:ffcc:a21f||/.well-known/acme-challenge/LpIZOn6UjHjtir408rTfu2VLyxIOZXuxsk0ycndwV0Y||301|
(these logs are for the actual hostname and redirect to the system from which the first set of logs came from)
It seems that the validation from 22.214.171.124 (outbound1.letsencrypt.org.) is not following the redirect. We see zero traffic from that IP to the server in question hosting the challenge response (or, it's trying on IPv6 and not making it there, possibly due to the HE/Cogent problem, but not then attempting IPv4) and http/https traffic is explicitly allowed.
IPv4 connectivity is up between the server and 126.96.36.199 (as I can ping it), so I know that's not the problem.
Is there any visibility from LE's side in to what may be going wrong?