Timeout during connect (likely firewall problem) after changing internet provider

The thing I don't understand is that the Let's Encrypt validation server does see the HTTP to HTTPS redirect, probably also through IPv6, but afterwards has some undefined issue.. Whereas when I try to connect to HTTP on port 80 through IPv6, I get nowhere:

osiris@erazer ~ $ curl -LIv6 http://fotografeer.nl/
*   Trying 2a10:3781:1960::1:80...
* connect to 2a10:3781:1960::1 port 80 failed: Permission denied
* Failed to connect to fotografeer.nl port 80 after 38 ms: Permission denied
* Closing connection 0
curl: (7) Failed to connect to fotografeer.nl port 80 after 38 ms: Permission denied
osiris@erazer ~ $ 

Same goes for HTTPS on port 443 by the way..

Maybe the LE validation server caches HTTP to HTTPS? But that would be the first time I hear from such caching and it wouldn't explain why the authorization was successful with the modified rewrite rules to prevent the redirect for the challenge..

In any case, I urge @dsnpevl to look closely to his IPv6 connectivity once again as this could mean trouble for IPv6 enabled users trying to visit his site. (Although they probably fall back to IPv4, but I'm not sure.) You didn't add the IPv6 address of your home router to the AAAA record by any chance, right? Because with IPv6 there is no NAT so you should add the IPv6 address of the actual server.

3 Likes

You don't need RewriteEngine to do that. I would do something like this (don't copypaste, I am not an apache user I can't check if it works)

# Outside of a VirtualHost
<Location "/\.well-known/acme-challenge">
  DocumentRoot /var/www/html 
  # I use /var/www/acme and it's dedicated to just challenges, but you
  # need to set up certbot to recognize it, using http-01 and method webroot
</Location>

<Virtualhost *:80>
  ServerName example.com
  Redirect permanent / https://example.com
</VirtualHost>

<Virtualhost *:80>
  ServerName www.example.com
  Redirect permanent / https://www.example.com
</VirtualHost>

And one more block for each domain you want to redirect. (Please note that you should just use one, www or not www: pick which one to use and redirect the other.)

This way acme challenges will get answered for any domain, even one apache doesn't know about, just the dns and certbot need to know. But redirects to https will only work for matching ServerNames.

3 Likes

Same. @dsnpevl check if your webserver is listening on ipv6 (ss -tlpn | grep '\]:80')

and if your firewall is allowing it to, (ip6tables -S | grep ' 80 ') (also check your provider's firewall)

please.

ALSO, now that I think of it, if it's a residential provider they could just not have any ipv6 support at all and the only solution is to remove the AAAA record.

3 Likes

Don't forget the home router. Although NAT is not applicable, most home router devices have a restrictive IPv6 firewall for incoming connections.

3 Likes

Well, as I edited, most home ISPs don't even bother offering ipv6 connectivity where I am.

3 Likes

Luckily, ISPs in The Netherlands do :wink:

5 Likes

(Yeah, I have an actual glass strand in the house that brings me a 2500down 500up mbps link, and no ipv6. Italy.)

4 Likes

You're right. I had the home router IPv6 address in AAAA record. Now changed it to the webserver's IPv6 address. And there are no external ports opened for IPv6 on my home router either. Not sure yet on how to configure that.

The web server is running multiple virtual hosts on different port numbers. For IPv4, in the home router, there are port forwardings for 80 and 443 to the virtual host actual port numbers. Not sure if I can achieve something similar for IPv6.

It is all getting pretty complicated. I'm considering removing the AAAA record completely and only allow IPv4 traffic to the web server.

3 Likes

That's always an option if IPv6 won't work. The average visitor of your site probably has IPv4 connectivity. Not many users are IPv6 only, although I believe it's increasing in Asia.

5 Likes

I come to your house, knock on your door, you answer, I ask for some information, you send me away to another address - which is merely another door to your house (instead of just giving me the answer).
Then I get to that other door, knock and either no one answers or they don't have an answer for me.

Do you see the missed opportunity now?

If HTTP via IPv6 fails, LE will try HTTP via IPv4.
When HTTP via IPv4 redirects to HTTPS, it starts over.
If HTTPS via IPv6 connects, then it better have the answer [but it doesn't = FAIL].

2 Likes

Why should Boulder only retry using IPv4 when the protocol is HTTP? One should think the rules of retrying are the same for HTTP as for HTTPS.

2 Likes

Huh?
We seem to agree that it starts over with IPv6 as preferred.
[maybe my little "s" confused you - changed it to a big "S" - Who doesn't like a big "S" ? LOL]

2 Likes

It should check all of them. Don't forget you can have multiple A records and multiple AAAA records.

2 Likes

But it doesn't.
It is NOT a whichever one completes makes a WIN!
It's IPv6 first.
If that is unable to complete (no decision), then try IPv4.
[if the IPv6 decision was a fail, then everything stops]

2 Likes

I don't know.

It makes 4 connections, I would imagine it makes them to different IPs if it can, and any difference between them is a failure.

2 Likes

You say that the validation fails when the HTTPS connection is tried using IPV6 without retrying using IPv4, while you also state that when HTTP fails over IPv6, it does retry using IPv4. Why retrying over IPV4 with the HTTP protocol, but don't retry using IPV4 with the HTTPS protocol? That's just weird.

2 Likes

What is the difference between unable to complete and fail?

I just set firewall to block all inbound IPv6 traffic and a certbot --dry-run failed with timeout even though IPv4 remained fine. Interesting, if the --dry-run started but was still hung / waiting for timeout and I re-opened IPv6, the --dry-run would complete.

3 Likes

Because HTTP did not fail, it redirected to HTTPS.
Thus, HTTP is now off the table.
And we begin again with HTTPS (IPv6 first).

2 Likes

Nevermind.

2 Likes

Well, the documentation says it's "To keep our CA software simple":

To keep our CA software simple we only perform an IPv6 to IPv4 retry on the first request when validating “http-01” challenges. If you use redirects, the redirects will not get retry treatment.

6 Likes