Can't access the API over ipv6 tunnel


#1

Hello, I’m unable to renew my certs over ipv6 tunnel from tunnelbroker.net. After some debugging for the possible problem i’ve narrowed it down to no connectivity to acme-v01.api.letsencrypt.org. It doesn’t respond to ping6 (it should, it does from another box with native v6) and port 443 timesout. Its not a single ip block or something either, because the result is the same if i use my tunnel endpoint ip as a source or random ip from a routed /48. I’m sure there is no issue with the server, because i created another tunnel to another server with the same result. Everything else works over ipv6. Here are some traceroutes to the api using source IPs 2001:470:1f0a:b35::2 and 2001:470:7373::1

traceroute -6 acme-v01.api.letsencrypt.org
traceroute to acme-v01.api.letsencrypt.org (2a02:26f0:c000:183::3d5), 30 hops max, 80 byte packets
1 mony-1.tunnel.tserv6.fra1.ipv6.he.net (2001:470:1f0a:b35::1) 35.738 ms 35.695 ms 35.681 ms
2 ve399.core1.fra1.he.net (2001:470:0:69::1) 44.059 ms 44.048 ms 44.035 ms
3 de-cix.fra20.ip.tiscali.net (2001:7f8::cb9:0:1) 36.025 ms 38.185 ms 42.534 ms
4 * * *
5 * * *

30 * * *

traceroute -6 acme-v01.api.letsencrypt.org -s 2001:470:1f0a:b35::2
traceroute to acme-v01.api.letsencrypt.org (2a02:26f0:c000:183::3d5), 30 hops max, 80 byte packets
1 mony-1.tunnel.tserv6.fra1.ipv6.he.net (2001:470:1f0a:b35::1) 34.716 ms 34.676 ms 37.291 ms
2 ve399.core1.fra1.he.net (2001:470:0:69::1) 31.598 ms 46.689 ms 31.577 ms
3 de-cix.fra20.ip.tiscali.net (2001:7f8::cb9:0:1) 34.752 ms 37.608 ms 37.592 ms
4 * * *
5 * * *

30 * * *

curl -6 https://acme-v01.api.letsencrypt.org also gets nothing

Is there some firewall block from 2001:470::/32 from letsencrypt? I’ve also send an email to tunnelbroker’s support, but they are not responding.


#2

Try tracepath which would show any MTU problem along the way.


#3

The issue seems most likely related to the tunnel. Boulder will not issue for many addresses in the 2001: space (see https://github.com/letsencrypt/boulder/blob/master/bdns/dns.go#L132) as they are private or teredo NAT’d addresses, but you’re not getting that far. Can you do a tracepath or mtr report and see if we get any more data from that?


#4

Thanks for the replies
tracepath6 acme-v01.api.letsencrypt.org
1?: [LOCALHOST] 0.014ms pmtu 1480
1: mony-1.tunnel.tserv6.fra1.ipv6.he.net 38.639ms
1: mony-1.tunnel.tserv6.fra1.ipv6.he.net 36.385ms
2: ve399.core1.fra1.he.net 31.946ms
3: de-cix.fra20.ip.tiscali.net 33.099ms
4: no reply
5: no reply
6: no reply
7: no reply
8: no reply
9: no reply
10: no reply
11: no reply
12: no reply
13: no reply
14: no reply
15: no reply
16: no reply
17: no reply
18: no reply
19: no reply
20: no reply
21: no reply
22: no reply
23: no reply
24: no reply
25: no reply
26: no reply
27: no reply
28: no reply
29: no reply
30: no reply
Too many hops: pmtu 1480
Resume: pmtu 1480

I’ve also tried with 1280mtu for the test with no difference. I’ve also checked against the boulder src and I’m not from a blocked prefix, but yes i’m not even getting there, so it shouldn’t be the issue.

Edit: Another DNS server returned 2600:1404:18:297::3d5 AAAA for acme-v01.api.letsencrypt.org, It responds to ping, curl i even hardcoded it to /etc/hosts and I managed to create new cert. Obviously this is not a long term solution, just a test, but it seems that i only have an issue accessing the boulder instances at 2a02:26f0:c000:182::3d5 and 2a02:26f0:c000:183::3d5

EDIT2: Its definitely not a problem with the tunnel itself. I’m uploading a screenshot from HE’s(the company that runs tunnelbroker) looking glass and one of their routers also can’t access 2a02:26f0:c000:183::3d5


#5

Thanks for those follow ups. We’ll see if we can reach out to Akamai. That sounds like a problem with some of their cdn nodes or the routing immediately preceding those nodes.


#6

Hurricane Electric tunnel IPs don’t look special. HE’s netblock just happens to be 2001:470::/32, which they use for all of their services, tunnels and otherwise.

I tried this from a Hurricane Electric tunnel in Florida, and i can’t access those two IPs either.

(My DNS resolvers give totally different, nearby IPs for acme-v01.api.letsencrypt.org that do work, so the issue doesn’t impact service for me.)

Traceroute to Mony’s Akamai IPs from a couple other ISPs in the US goes through Cogent’s network.

Could this issue be due to the longstanding peering dispute between Cogent and Hurricane Electric? Maybe that Akamai PoP only uses one ISP?


#7

This is unrelated. Only a tiny part of the 2001: space is reserved. (The 2001::/32 is even redundant in the Boulder source because 2001::/23 covers that)


#8

Mony, is this still broken for you?


#9

Sorry for the delay, yes its still broken. I still get the same response from the ns servers (same ip addresses) and I still don’t have connectivity to them. I searched about the cogent vs he peering dispute that was mentioned by mnordhoff and it quite possibly could be the reason. I never new it existed but i guess we learn something new every day :smiley:


#10

I try test using he.net network is normal work. you may send email to ipv6 @ he.net for fix it.


#11

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.