Renew claims connect timeout, but my server claims to have sent the reply

Help
1

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is:
cv.cognitivetools.com

I ran this command:
sudo certbot renew

It produced this output:

Attempting to renew cert (cv.cognitivetools.com) from /etc/letsencrypt/renewal/cv.cognitivetools.com.conf produced an unexpected error: Failed authorization procedure. cv.cognitivetools.com (http-01): urn:ietf:params:acme:error:connection :: The server could not connect to the client to verify the domain :: Fetching http://cv.cognitivetools.com/.well-known/acme-challenge/gP0uAaIEBjJ7QDCD_js8UX1CVhgaqblvcHR-Vmuo2VM: Timeout during connect (likely firewall problem). Skipping.

My web server is (include version):

SWI-Prolog HTTP server

The operating system my web server runs on is (include version):

Raspbian GNU/Linux 10 (buster)

My hosting provider, if applicable, is:

ziggo.nl

I can login to a root shell on my machine (yes or no, or I don't know):

yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):

certbot 0.31.0

While LetsEncrypt claims it cannot connect, I copied the challenge file and restored it, after which I can get it using curl from a host outside my network.

curl http://cv.cognitivetools.com/.well-known/acme-challenge/gP0uAaIEBjJ7QDCD_js8UX1CVhgaqblvcHR-Vmuo2VM

Also, when I enable debugging the web server I do get a connect and reply:

[Thread httpd@80_1] 2021-04-18 09:53:22.273  First line: GET /.well-known/acme-challenge/z9YSZmYsmtGlhkvMe-gdVGY1oxpOUIBtf5QTxjDrsWo HTTP/1.1
[Thread httpd@80_1] 2021-04-18 09:53:22.274  Header = 
Host: cv.cognitivetools.com
User-Agent: Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)
Accept: */*
Accept-Encoding: gzip
Connection: close



[Thread httpd@80_1] 2021-04-18 09:53:22.277 Field: [host('cv.cognitivetools.com'),user_agent('Mozilla/5.0 (compatible; Let\'s Encrypt validation server; +https://www.letsencrypt.org)'),accept([media(_4600/_4602,[],1.0,[])]),accept_encoding(gzip),connection(close)]
[Thread httpd@80_1] 2021-04-18 09:53:22.279 [40] get /.well-known/acme-challenge/z9YSZmYsmtGlhkvMe-gdVGY1oxpOUIBtf5QTxjDrsWo ...
[Thread httpd@80_1] 2021-04-18 09:53:22.285 [40] 200 file(application/unknown,/home/pi/src/zwave/.well-known/acme-challenge/z9YSZmYsmtGlhkvMe-gdVGY1oxpOUIBtf5QTxjDrsWo); 87 bytes

This used to work fine. The system was running Jessie when it started reporting this issue. I updated the OS and all software to their latest version, but this problem remains unchanged.

The server runs behind a Turris router that forwards port 80 and 443. Turris has a dynamic distributed firewall. Just to be sure I disabled this briefly, but that doesn't make a difference (as expected because the server does connect).

Could it be an issue that the server ignores Accept-Encoding: gzip? The replied header is

HTTP/1.1 200 OK
Date: Sun, 18 Apr 2021 09:25:46 GMT
Last-modified: Sun, 18 Apr 2021 08:42:33 GMT
Connection: Keep-Alive
Content-Length: 87
Content-Type: application/unknown

I left the challenge file on the server. Any clue is welcome ...

edit I added handling for Accept-Encoding: gzip to the server. I can verify the compression is done and the reply header is now as below, but the behavior has not changed (still claiming it cannot connect, probably due to a firewall issue).

HTTP/1.1 200 OK
Date: Sun, 18 Apr 2021 11:57:05 GMT
Last-modified: Sun, 18 Apr 2021 11:56:45 GMT
Connection: close
Content-Length: 113
Content-Type: application/unknown
Content-Encoding: gzip
2

Hi @janw

I can open your url, Letsencrypt can't.

So you have something like a regional firewall, failban, a blocking htaccess.

Find and remove that.

3

Thanks for looking into this. That was also my guess, but some things seem to contradict this:

  • I do have OpenWRT/Turris distributed firewall in my router. Disabling this makes no difference though.
  • My web server sees the connection (from 3.128.26.105, ec2-3-128-26-105.us-east-2.compute.amazonaws.com
  • My web server claims no errors, not when sending the reply, nor when closing the connection. It seems the socket is perfectly fine during the entire (small) request.

P.s. Also set Content-type to text/plain, but that to doesn't change anything :frowning:

4

Let's Encrypt checks from several points on the Internet, to ensure that you own the name as seen from everywhere. You should be seeing at least 3 or 4 connections in your web server logs if things were working right. You have some firewall blocking most of these connections, even if it's letting one of them through.

5

Ok. Thanks. That explains something. I get either one or two requests most of the times. Now the interesting question is what is blocking the others ... Definitely not the web server, so either the router or my ISP.

6

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.