Timeout receving the certificate

Help
1

My domain is: dev.tierschnack.de

I ran this command: The internal ACME client from KeyHelp

It produced this output:
[17-Jun-2024 15:10:26] INFO | Sending signed request to "https://acme-staging-v02.api.letsencrypt.org/acme/chall-v3/12798603533/Uqq7lg".
[17-Jun-2024 15:10:27] ERROR | Apache: a Let's Encrypt error occurred: Verification ended with an error.
Details: 2a03:7847:2252:180:5c65:88ff:fef2:b9a1: Fetching http://dev.tierschnack.de/.well-known/acme-challenge/_T6L5IKdx6JCJlr4Qi608dNfrhfud6Zl1r8xDpuHltY: Timeout during connect (likely firewall problem)
Type: urn:ietf:params:acme:error:connection
Full response: {"type":"http-01","url":"https://acme-staging-v02.api.letsencrypt.org/acme/chall-v3/12798603533/Uqq7lg","status":"invalid","validated":"2024-06-17T13:10:08Z","error":{"type":"urn:ietf:params:acme:error:connection","detail":"2a03:7847:2252:180:5c65:88ff:fef2:b9a1: Fetching http://dev.tierschnack.de/.well-known/acme-challenge/_T6L5IKdx6JCJlr4Qi608dNfrhfud6Zl1r8xDpuHltY: Timeout during connect (likely firewall problem)","status":400},"token":"_T6L5IKdx6JCJlr4Qi608dNfrhfud6Zl1r8xDpuHltY","validationRecord":[{"url":"http://dev.tierschnack.de/.well-known/acme-challenge/_T6L5IKdx6JCJlr4Qi608dNfrhfud6Zl1r8xDpuHltY","hostname":"dev.tierschnack.de","port":"80","addressesResolved":["2a03:7847:2252:180:5c65:88ff:fef2:b9a1"],"addressUsed":"2a03:7847:2252:180:5c65:88ff:fef2:b9a1"}]}
[17-Jun-2024 15:10:29] INFO | SNI configuration updated

I tried both live and staging

My web server is (include version):
Apache 2.4.59

The operating system my web server runs on is (include version):
Debian Bookworm 12.5

My hosting provider, if applicable, is: NetCup

I can login to a root shell on my machine (yes or no, or I don't know): yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): KeyHelp 24.1

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): Not using certbot, the internal client, but I have is running on other systems without any issues.

The request reaches my host:
2a03:7847:2252:180:5c65:88ff:fef2:b9a1 - - [17/Jun/2024:15:10:07 +0200] "GET /.well-known/acme-challenge/local-check-667035af38ef55.96836941 HTTP/1.1" 200 35 "-" "KeyHelp/24.1 (https://www.keyhelp.de) PHP/8.2.18 curl/7.88.1" 193 257
2a03:7847:2252:180:5c65:88ff:fef2:b9a1 - - [17/Jun/2024:15:10:08 +0200] "GET /.well-known/acme-challenge/_T6L5IKdx6JCJlr4Qi608dNfrhfud6Zl1r8xDpuHltY HTTP/1.1" 200 87 "-" "KeyHelp/24.1 (https://www.keyhelp.de) PHP/8.2.18 curl/7.88.1" 201 309

1 Like
2

weird: from whois tierschnack.de registered but info is half cut (says it's taken but not show any info about that, not even text that it's under domain privacy) and de nameserver doesn't think tierschnack.de is registerd (NSEC3 for that domain) I think your domain registration process is in some invalid state and you should ask domain register for that (can't see it from whois)

https://dnsviz.net/d/dev.tierschnack.de/dnssec/

1 Like
3

Those look like some sort of checking that your client is doing ahead of time; since they're from your own server IP and the user agent says "KeyHelp". You should be seeing at least 5 other requests, from other IPs, with user agents that say "Let's Encrypt".

3 Likes
4

The Let's Debug test site is good while you correct and debug your comms setup. If you make too many failed requests to Let's Encrypt you will get temporarily rate limited (blocked).

I also see some problems with your DNS configuration. But, it is not causing the "timeout" problem reported by Let's Debug from the LE Staging system. The LE Staging system is able to see your AAAA record.

Still, you should review this
https://dnsviz.net/d/dev.tierschnack.de/dnssec/

2 Likes
5

Hi all, thx for you fast replies.
Regarding the broken WHOIS, that because of the stupid GPDR, they rendered WHOIS useless.
The domain is valid, main site is on a different server, but reachable: https://tierschnack.de/ (German language only)
Ah, yes, I didn't realize, those request seems to come from my panel.
However, when I try load fetch the file from a different server it works:
❯ wget http://dev.tierschnack.de/.well-known/acme-challenge/JAx-_2opu1SQHhaDnjlvPVEYdOhkjyLRjKkLOo9xqhA
--2024-06-17 16:08:54-- http://dev.tierschnack.de/.well-known/acme-challenge/JAx-_2opu1SQHhaDnjlvPVEYdOhkjyLRjKkLOo9xqhA
Resolving dev.tierschnack.de (dev.tierschnack.de)... 2a03:7847:2252:180:5c65:88ff:fef2:b9a1
Connecting to dev.tierschnack.de (dev.tierschnack.de)|2a03:7847:2252:180:5c65:88ff:fef2:b9a1|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 87
Saving to: ‘JAx-_2opu1SQHhaDnjlvPVEYdOhkjyLRjKkLOo9xqhA’

JAx-_2opu1SQHhaDnjlvPVEYdOhkjyLRjKkLOo9xqhA 100%[=============================================================================================================>] 87 --.-KB/s in 0s

2024-06-17 16:08:54 (12.6 MB/s) - ‘JAx-_2opu1SQHhaDnjlvPVEYdOhkjyLRjKkLOo9xqhA’ saved [87/87]

1 Like
6

Was that on the same local network? Because Let's Debug and Let's Encrypt timeout trying to reach your domain over the public internet. I also cannot reach your domain from my own test server. Your domain needs to be reachable from the public internet to satisfy the HTTP Challenge from Let's Encrypt.

Can you try from the public internet? Like with a mobile phone with wifi turned off to use your carrier's network?

Or, do you have a firewall blocking certain IP or countries?

3 Likes
7

No, the test was from a vps, and, I must admit, I messed up my settings.
The data from above, vps at NetCup is the machine with the main domain. My dev system is at home, behind my FTTH with static IPv4 and IPv6.
However, I already have some sites running at home, but they get their certs via certbot.
And the target has all required ports open:
nmap -6 dev.tierschnack.de
Starting Nmap 7.93 ( https://nmap.org ) at 2024-06-17 17:26 CEST
Nmap scan report for dev.tierschnack.de (2a03:7847:2252:180:5c65:88ff:fef2:b9a1)
Host is up (0.038s latency).
Not shown: 986 filtered tcp ports (no-response)
PORT STATE SERVICE
20/tcp closed ftp-data
21/tcp open ftp
22/tcp open ssh
25/tcp closed smtp
53/tcp open domain
80/tcp open http
110/tcp open pop3
143/tcp open imap
443/tcp open https
465/tcp closed smtps
587/tcp closed submission
993/tcp open imaps
995/tcp open pop3s
30000/tcp closed ndmps

Nmap done: 1 IP address (1 host up) scanned in 5.21 seconds

8

You should check your firewall settings. Below is from my own test server in an AWS East Coast region. Let's Encrypt uses AWS for most of its auth servers too.

I needed to use -Pn to force nmap to consider your destination reachable. "Filtered" means blocked.

nmap -Pn -p21,80,443 -6 dev.tierschnack.de

Starting Nmap 7.80 ( https://nmap.org ) at 2024-06-17 15:37 UTC
Nmap scan report for dev.tierschnack.de (2a03:7847:2252:180:5c65:88ff:fef2:b9a1)
Host is up.

PORT    STATE    SERVICE
21/tcp  filtered ftp
80/tcp  filtered http
443/tcp filtered https

The Let's Debug test site I previously linked is excellent testing tool when setting up new systems.

3 Likes
9

That is strange, I did my scan from a vps at NetCup, a German hoster, and 80 is open.
What do you get with a scan for 2a03:7847:2252:180:a00:27ff:fed1:cb53?
All IP are are in an alias in a OPNsense, each are allowed Port 80 and 443 from everywhere. And the machine with the cb53 at the end currently hosts three sites secured by LE.
To be sure that is is not the ACME client from KeyHelp, I installed certbot, and got the same result. Timeout.

10

I can see an Apache server at that IP from my USA East Coast AWS region. But, only with HTTPS (not HTTP). The cert for HTTPS is for a cake3 subdomain

Check that you firewall allows access from all points of the world. Make sure you check every "layer" of access to your server from the outermost ISP to it.

Here is my nmap for this address

nmap  -p21,80,443 -6 2a03:7847:2252:180:a00:27ff:fed1:cb53
Starting Nmap 7.80 ( https://nmap.org ) at 2024-06-17 16:36 UTC
Nmap scan report for 2a03:7847:2252:180:a00:27ff:fed1:cb53
Host is up (0.10s latency).

PORT    STATE    SERVICE
21/tcp  filtered ftp
80/tcp  filtered http
443/tcp open     https

Using the domain name from that cake3 cert and Let's Debug HTTP Challenge predictably times out as port 80 is blocked. I suppose you could be using TLS-ALPN challenge if using mod_md with Apache. Are you?

2 Likes
11

I must admit, I'm not so deep into LE to know details, they make a good job, it just works.
So this is the first time I have issues, so I need to take a deeper look.
I suppose it is HTTP01 challenge what I use. Just installed certbot and that python stuff for Apache and run certbot, they did all the work (except making the configs look nice ^^)>
I found an issue on my side, I had two IPv6 addresses, as I needed to override it, and that hat the flag noprefixroute, I fixed that, only one IPv6 left:
inet6 2a03:7847:2252:180:5054:ff:fe6c:13d1/64 scope global
That is the one I really need to maintain, as there are DNS pointers I cannot change* pointing on it. But it still doesn't work. I changed the ptr for dev.tierschnack.de, but still no luck.

  • I'm not into something strange, it is just that some ptr from my work point to service.myname.company.com, and I don't want to bother the DNS admin to update the pointers just because I change my setup.
12

You may have to bother them if you can't figure out why Let's Debug and Let's Encrypt time out using that domain with that IPv6 address.

2 Likes
13

Unlike ipv4 ipv6 doesn't do NAT so DDNS running on router is invalid for server behind the router: you'd need to run DDNS on that server itself
btw dev subdomain doesn't have ipv4 and just a ipv6: is that expected?

14

I've been able to NAT IPv6

2 Likes
15

@MikeMcQ That's what I'm trying to do. Currently it seems to be a issue on the AWS side, as I can reach port 80 from outside. And someone else from the KeyHelp forum, too.

16

It's not specific to AWS, connecting to your site doesn't work from my home connection either (in the northeast United States, and yes I'm using IPv6).

3 Likes
17

Which AWS "side"? All of them? Let's Encrypt uses four different AWS regions across the world. I think my own test server is even in a different one from those.

Further, the primary LE auth center is not in AWS and that can't see you either.

The Let's Debug site runs two connection tests. One using the LE Staging system. But, the other is from its own server. It is not in AWS (it is at Hetzner) and it can't reach you with HTTP (port 80) from its server either.

You might also try the below test site. It can't see you from any of its test locations. Although, I don't know where they run their servers

Probably don't rely on geopeeker test. I am not certain they support IPv6-only sites. The ones above do.

https://www.geopeeker.com/fetch/?url=http%3A%2F%2Fdev.tierschnack.de&csrf_token=9qZuX%2FZ46uWyxH0xZHT6ag8FJw0NDZRgZUh8pDhnO%2FU%3D

3 Likes
18

Looks like you resolved some of your connection problem. I can now reach your domain using HTTPS (port 443). Just port 80 is failing.

curl -i -m8  http://dev.tierschnack.de
curl: (28) Connection timed out after 8000 milliseconds

curl -ik -m8 https://dev.tierschnack.de
HTTP/2 302
location: http://dev.tierschnack.de/
server: Apache
3 Likes
19

4 posts were split to a new topic: Timeout trying to get cert

21

OK, some other users from Germany can reach both ports, some not.
I has to adapt the IP, but that's not related to the LE issue.
A user made a test from a VPN in Amsterdam:
nmap -6 dev.tierschnack.de
Starting Nmap 7.80 ( https://nmap.org ) at 2024-06-17 20:46 CEST
Nmap scan report for dev.tierschnack.de (2a03:7847:2252:180:5054:ff:fe6c:13d1)
Host is up (0.027s latency).
Not shown: 997 filtered ports
PORT STATE SERVICE
80/tcp open http
443/tcp open https
8080/tcp open http-proxy

All good.
I still don't see a scheme behind that.