Timeout during connect (likely firewall problem) but visible in access logs?

Help
1

I have really weird behaviour I'm not understand.

  1. I do have one web server, where multiple domains are in place
  2. ALL except one domains are working fine, they share same config, but different domain name there of course
  3. I'm able to get testing curl http:///.well-known/acme-challenge/letsdebug-test without any problem, even from public internet and "curl test sites"
  4. I DO see the request in access logs of server, so it DOES connects. (of course, random challenge is not there, but "letsdebug-test" does)
access log 
172.104.24.29 router.jpleva.cz - [09/Jun/2022:01:32:36 +0200] "GET /.well-known/acme-challenge/letsdebug-test HTTP/1.1" 200 7 "-" "Mozilla/5.0 (compatible; Let's Debug emulating Let's Encrypt validation server; +https://letsdebug.net)"
3.73.52.44 router.jpleva.cz - [09/Jun/2022:01:32:37 +0200] "GET /.well-known/acme-challenge/MUfMoaUW1Bi8fBszNj6VFwaPfnTFf6jH-xwCW9WLEtA HTTP/1.1" 404 341 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
13.58.95.193 router.jpleva.cz - [09/Jun/2022:01:32:37 +0200] "GET /.well-known/acme-challenge/MUfMoaUW1Bi8fBszNj6VFwaPfnTFf6jH-xwCW9WLEtA HTTP/1.1" 404 341 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
34.217.91.245 router.jpleva.cz - [09/Jun/2022:01:32:38 +0200] "GET /.well-known/acme-challenge/MUfMoaUW1Bi8fBszNj6VFwaPfnTFf6jH-xwCW9WLEtA HTTP/1.1" 404 341 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"

BUT, Let's debug still says - man, I can't connect, it must be firewall.... WTF?

Screenshot-2022-06-09_02.18.48@2x
Screenshot-2022-06-09_02.18.48@2x2242×966 210 KB

Sorry , but I'm out of ideas, how to fix this...

2

You should be seeing 4 requests. One is blocked. Check for 64.78.149.164 and 66.133.109.36 in your firewall.

1 Like
3

And those three are all returning "404" - so there is another issue after the firewall issue.

Dare I ask?
Are you using Apache?

3 Likes
4

The 404s are expected in this case because Let's Debug creates a certificate order on acme-staging-v02!

2 Likes
6

ok, new day and I want to check again and find out this:

root@turris:/# ~/acme -d router.jpleva.cz -r
[Thu Jun 9 09:22:13 CEST 2022] Renew: 'router.jpleva.cz'
[Thu Jun 9 09:22:13 CEST 2022] Renew to Le_API=https://acme-v02.api.letsencrypt.org/directory
[Thu Jun 9 09:22:13 CEST 2022] Skip, Next renewal time is: 2022-08-08T03:02:14Z
[Thu Jun 9 09:22:13 CEST 2022] Add '--force' to force to renew.

it renewed somehow itself tonight.

The only change I did was, adjusting the rule to not forwarding http to https for /.well-know/ URL what was probably the real problem, then. Other domains did have it already.

But it's really weird, that let's debug still complains about connection issue... I've double-check and I do not blocking any of the IPs you mentioned. It's really weird error...

7

Ok, I think, that there is something in-between LE and router itself.

Same test to server, where I do have 100% control over it as it's server hosting, there isn't any problem with, but on home router does. Looks like ISP maybe proactively blocking some part of traffic or something, otherwise I do not have idea, what's going on there.

Anyway, now it's working somehow again, so thank you for hint and tip for what I should go!

1 Like
8

Ahh!! I had some suspicion... Both IP can be found here...

https://view.sentinel.turris.cz/dynfw/

And my router is using those rules... I'll will try to report that...

So shortly:

  • blocking was caused by dynamic firewall rule
  • and real issue was cased by forgetting of adding exclusion of forwarding of /.well-known/ URL to HTTPS (as done for rest).

BTW - it would probably good to provide more information about those connection error (ex from what IP it fails connect to) and as there is more of them, probably something like 2 from 4 failed. It will help a lot.

2 Likes
9

I think the risk is that people will start whitelisting IPs if they are reported directly in error messages. This is something that has been discouraged so far because the IP pool changes frequently.

You're (at least) the third person on here who has encountered problems with Turris, though.

3 Likes
10

Yeah, that's right. Makes not sense too much, then.

Actually it's first time I had the issue (and was caused by something else obviously). But it's good to keep in mind that firewall. Anyway, definitely it helps prevent more troubles, then actually causing (at least from my point of view, have no issue with dynamic firewall so far, now it just points me to wrong direction - thats the only problem I had).

Thank you for help!

1 Like
11

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.