HTTP-01 validation timeout from some Let's Encrypt validation perspectives

Help
1

My domain is: id.airpay.kz

Public IPv4: 95.56.228.222

I am using NGINX Proxy Manager with Certbot.

The existing certificate renewal intermittently fails with:

Certbot failed to authenticate some domains (authenticator: webroot).
The Certificate Authority reported these problems:

Identifier: id.airpay.kz
Type: connection
Detail: 95.56.228.222: Fetching http://id.airpay.kz/.well-known/acme-challenge/yua28UrDHD21EHMrBh_2D5rQ7__GIFq3veAGVfaM-e8: Timeout after connect

However, at almost the same time, the same ACME challenge was successfully reached by a Let's Encrypt validation server:

[30/Sep/2026:12:31:10 +0500]
200 GET http id.airpay.kz "/.well-known/acme-challenge/yua28UrDHD21EHMrBh_2D5rQ7__GIFq3veAGVfaM-e8"
[Client 23.178.112.219]
"Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"

The request returned HTTP 200 and the challenge was served successfully.

There is no AAAA record for the domain because IPv6 is not provided by my ISP.

TCP port 80 is publicly reachable and NGINX Proxy Manager is listening on:

0.0.0.0:80
[::]:80

The important point is that at least one Let's Encrypt validation request reaches my server successfully, while another validation attempt reports:

Timeout after connect

If the failing validation perspective cannot reach my server at all, I obviously cannot see its packets in tcpdump or my web server logs.
Here is the setup:

Provider / Public IP / NAT / GW / Docker / NGINX Proxy Manager are operational — an external server successfully executes:

curl -4 -v
-A 'Mozilla/5.0 (compatible; Let'''s Encrypt validation server; +https://www.letsencrypt.org)'
http://id.airpay.kz/.well-known/acme-challenge/yua28UrDHD21EHMrBh_2D5rQ7__GIFq3veAGVfaM-e8

CrowdSec has been ruled out — stopping the crowdsec-firewall-bouncer did not change the situation.

IPv6 is not involved — id.airpay.kz has no AAAA record.

Let's Encrypt can indeed reach NPM — a request was previously logged:

23.178.112.219
/.well-known/acme-challenge/...
HTTP 200

However, Let's Encrypt itself reports:

95.56.228.222: Fetching http://id.airpay.kz/.well-known/acme-challenge/...
Timeout after connect

Therefore, the issue appears to stem from the difference between Let's Encrypt's validation perspectives: one validator reaches the server, while the other does not. Furthermore, we cannot see the IP of the unsuccessful validator because the TCP connection to your server fails to establish.

Could you please check the production HTTP-01 validation for id.airpay.kz / 95.56.228.222 from the Let's Encrypt validation infrastructure and determine whether there is a connectivity or routing problem from one of the validation perspectives?

I can provide full Certbot logs, NGINX access logs and packet captures for the validation attempts if required.

Thank you.

2

Yes, but, there are currently 5 validation centers. You are only seeing one in your nginx logs and that is the Primary center. None of the 4 secondary centers reach you. These originate from different parts of the world. Two of which are currently in the US as is the Primary.

This strongly points to a firewall at your location which blocks repeated "unusual" requests from different source IP.

There is something unusual though about the error message. Usually when the failures are because of Secondary center failures the error message says "Secondary". Your error does not. I haven't looked at the LE Boulder code to see if they've changed that. In any case, you should be seeing at least 4 and usually 5 nginx access log entries for a successful challenge (as one secondary is allowed to fail)

I tested connectivity using various tools and could not repeat the timeout error. This further points to a "smart firewall" or "DDoS" protection firewall at your end blocking the repeated requests.

3

Thank you for your prompt response.

Perhaps I did not provide the complete logs, which may have caused a misunderstanding of the issue. Let me clarify the situation in more technical detail.

  1. All of our websites are hosted behind the same Nginx Proxy Manager instance and are subject to the same firewall and network configuration. Therefore, it is unlikely that this is a general firewall or web server connectivity issue on our side. If our firewall or Nginx configuration were blocking Let's Encrypt validation servers, we would expect certificate renewals to consistently fail for all certificates, rather than succeed for some certificates and fail for others.

  2. The problem is intermittent and appears to depend on the particular validation attempt. The certificate for the same website can be renewed successfully on one day, while the renewal of the same certificate can fail on another day without any changes to our firewall, Nginx Proxy Manager, DNS, or web server configuration.

  3. We can also reproduce this behavior when renewing multiple certificates sequentially. For example, two certificates are requested one after another using the same Nginx Proxy Manager and the same network configuration. The first certificate is successfully validated and renewed, while the second certificate fails during validation. Please see today's logs for an example of this behavior.

  4. We have also tested HTTP access to the affected website from an external server using a User-Agent that identifies itself as a Let's Encrypt validation server. The request completes successfully. This confirms that the website is publicly reachable over HTTP and that there is no general restriction on incoming HTTP connections from external networks.

Therefore, the key point is that the problem is not a permanent connectivity or firewall issue. The same infrastructure successfully serves Let's Encrypt validation requests, while other validation attempts fail under otherwise identical conditions.

Could you please check the validation logs on the Let's Encrypt side and, in particular, identify which validation server(s) or source IP address(es) were used for the failed validation attempts? This information would help us determine whether the failures are associated with specific Let's Encrypt validation servers or network paths.

Version used
jc21/nginx-proxy-manager:latest v2.16.0
nginxproxymanager.com

If the issue is related to an error within Nginx Proxy Manager itself (there are corresponding errors visible in the logs), could you please clarify the exact nature and cause of this error? These errors appear to be related to Certbot's operation. Therefore, I am concerned that if I contact the Nginx Proxy Manager developers, they may simply refer me back to Let's Encrypt for further investigation. It would therefore be very helpful if you could clarify whether the error indicates a problem with the Let's Encrypt validation process or with the communication between Certbot and the Let's Encrypt infrastructure.

6

The error in your "fail" log is not a "timeout" like you showed earlier. It is a 403 forbidden sent by your system to the Let's Encrypt server. You should review your nginx access logs and your overall system config.

"error": {
"type": "urn:ietf:params:acme:error:unauthorized",
"detail": "95.56.228.222: Invalid response from http://vault.airpay.kz/.well-known/acme-challenge/UPTeS0O8oC6LSGI_fRGBV6IG0TfAB5GXNp7mkCZd-lg: 403",

7

Thanks for the clarification. The 403 code wasn't very informative within the large log, so it got lost in the shuffle. The issue was indeed on my end, but I wouldn't have found it without your help.