Problems validating IPv6 against host running 6to4

Thanks for enabling IPv6 support. I’ve already successfully issued a certificate for an IPv6-only domain name. However, I still have troubles validating a domain name having just a 6to4 automatic tunneling address. Validation fails with “Could not connect” error.

When I tried to find the root cause, I observed that the validation address 2600:3000:2710:200::1d does not respond to ICMP echo requests, which makes troubleshooting almost impossible. There are also no DNS records yet.

1 Like

Can you provide your hostname, its IPv6 and IPv4 addresses, and the output of dig <hostname> and dig AAAA <hostname>? Thanks!

1 Like

I’ve successfuly obtained certifcate for: ip6.oskarcz.net
$ dig ip6.oskarcz.net AAAA +short
2002:55ef:e3b3::1
2001:1528:132:70::1

I had no success with hostname: 6to4.oskarcz.net
$ dig 6to4.oskarcz.net AAAA
6to4.oskarcz.net. 3545 IN CNAME flexi6to4.oskarcz.net.
flexi6to4.oskarcz.net. 3545 IN AAAA 2002:55ef:e3b3::1

Both addresses are operated by the same server. My guess is that there is a broken third party 6to4 gateway on the way between my server and LE.

1 Like

Thanks for the extra info. We’re looking into it, and so far it looks like what you say - a broken third party gateway. We’re digging a little deeper to see what we can find, though. Appreciate your help!

1 Like

Hmm. Could be worth asking some IPv6 experts to help think about this. The best IPv6 person I know in your area (assuming you’re in the Bay Area) is Cisco’s Stig Venaas - if you want somebody that is happy to sit down with a beer and discuss what’s best he’s your guy. For myself, with only a little expertise, it seems as though for 6to4 specifically it makes sense for Let’s Encrypt to perform the encapsulation gateway function itself (I mean not necessarily in Boulder, but on hardware under your control) so that you only have the same worries about hostile middle boxes as for an IPv4 applicant.

A 6to4 gateway gets to interpose itself between Let’s Encrypt (or at least one Let’s Encrypt site) and all 6to4 applicants. That’s acceptable if it’s your gateway, or if it’s operated by a trusted supplier that you’re contracting with, but it doesn’t seem acceptable if it’s a third party and especially if that third party isn’t under contract to you for the service. Thus I think Let’s Encrypt should ensure a specific node is configured as the 6to4 gateway, not the default anycast address, and that this specific node is operated by them or by a trusted supplier.

If that’s not practical for some reason it may be safer to forbid 6to4 (but not ordinary IPv6) which anyway makes up a very small percentage of all reachable IPv6 Internet hosts.

5 Likes

Thanks, that’s very helpful!

1 Like

@oskar, could you run traceroute6 2600:3000:2710:200::1d from your 6to4 host and give us the output?

1 Like

Here it comes:

# LANG=C traceroute6 2600:3000:2710:200::1d
traceroute to 2600:3000:2710:200::1d (2600:3000:2710:200::1d) from 2002:55ef:e3b3::1, port 33434, from port 56303, 30 hops max, 60 bytes packets
 1  2002:c058:6301::1 (2002:c058:6301::1)  0.547 ms  0.452 ms  0.483 ms 
 2  2001:1488:ffff::ffff (2001:1488:ffff::ffff)  0.681 ms  0.594 ms  2.195 ms 
 3  20ge1-3.core1.prg1.he.net (2001:7f8:14::6e:1)  0.838 ms  9.106 ms  11.477 ms 
 4  100ge8-1.core1.fra1.he.net (2001:470:0:213::1)  10.482 ms  11.786 ms  8.519 ms 
 5  100ge6-1.core1.lon2.he.net (2001:470:0:37::1)  29.605 ms  20.165 ms  20.141 ms 
 6  100ge1-1.core1.nyc4.he.net (2001:470:0:2cf::2)  93.726 ms  99.093 ms  88.546 ms 
 7  2001:428:601:e::1 (2001:428:601:e::1)  187.495 ms  117.512 ms  118.517 ms 
 8  2001:428::205:171:3:161 (2001:428::205:171:3:161)  152.425 ms  156.682 ms  152.725 ms 
 9  * * *         
10  * * *         
11  * * *         
12  * * *         
13  * * *         
14  * * *         
15  * * *         
16  * * *         
17  * * *         
18  * * *         
19  * * *         
20  * * *         
21  * * *         
22  * * *         
23  * * *         
24  * * *         
25  * * *         
26  * * *         
27  * * *         
28  * * *         
29  * * *         
30  * * * 

For comparsion, this is the traceroute from the native IPv6 address:

# LANG=C traceroute6 2600:3000:2710:200::1d 
traceroute to 2600:3000:2710:200::1d (2600:3000:2710:200::1d) from 2001:1528:132:70::1, port 33434, from port 56246, 30 hops max, 60 bytes packets
 1  2001:1528:132::1 (2001:1528:132::1)  0.816 ms  0.663 ms  0.660 ms 
 2  2001:1528:1:14:226:8800:6120:47c0 (2001:1528:1:14:226:8800:6120:47c0)  0.354 ms  0.274 ms  0.286 ms 
 3  10gigabitethernet1-8.core1.prg1.he.net (2001:470:1:2ce::1)  0.377 ms  12.905 ms  0.711 ms 
 4  100ge8-1.core1.fra1.he.net (2001:470:0:213::1)  7.834 ms  11.889 ms  14.092 ms 
 5  100ge6-1.core1.lon2.he.net (2001:470:0:37::1)  19.972 ms  19.915 ms  20.202 ms 
 6  100ge1-1.core1.nyc4.he.net (2001:470:0:2cf::2)  85.687 ms  92.770 ms  98.674 ms 
 7  2001:428:601:e::1 (2001:428:601:e::1)  88.846 ms  90.006 ms  87.374 ms 
 8  2001:428::205:171:3:161 (2001:428::205:171:3:161)  129.358 ms  127.122 ms  126.977 ms 
 9  2001:428:3801:210:0:25:0:2 (2001:428:3801:210:0:25:0:2)  128.014 ms  128.026 ms  129.612 ms 
10  2600:3000:2:b80::1 (2600:3000:2:b80::1)  127.916 ms  129.341 ms  128.206 ms 
11  2600:3000:2:c0::1 (2600:3000:2:c0::1)  129.692 ms  127.730 ms  127.997 ms 
12  2600:3000:1:111::2 (2600:3000:1:111::2)  153.959 ms  154.013 ms  153.928 ms 
13  2600:3000:3:4d0::2 (2600:3000:3:4d0::2)  152.663 ms  152.200 ms  152.341 ms 
14  2600:3000:3:38::2 (2600:3000:3:38::2)  155.251 ms  155.169 ms  155.441 ms 
15  2600:3000:3:730::2 (2600:3000:3:730::2)  152.073 ms  152.567 ms  151.972 ms 
16  2600:3000:2700:1073::4 (2600:3000:2700:1073::4)  153.598 ms  153.536 ms  153.474 ms 
17  * * *         
18  * * *         
19  * * *         
20  * * *

Hope this helps.

1 Like

Thanks for the extra info @Oskar!

After reviewing the literature and consulting with a networking expert, we’re following @tialaramex’s advice and disabling validation requests to addresses that start with the 6to4 anycast prefix 2002::/16. This is effective now, through a firewall rule, and we’ll be adding it into Boulder so we can provide an earlier error message.

Our reasoning is exactly as @tialaramex says: If we send 6to4 requests to the anycast prefix, it’s too easy for an untrusted third party to advertise that prefix and MITM our validation requests. We could run a 6to4 gateway inside our own datacenter, but since the 6to4 anycast prefix is deprecated, we think it’s not worth the extra operational overhead to support a very small number of hosts.

My recommendation to anyone operating a host that only has a 6to4 address is to use the DNS validation method.

1 Like

Thanks for answer. I don’t really care about 6to4, the sole fact that it didn’t work in the first place shows the exact reason why this technology has been deprecated.

But I guess you should apply similar ban to the Teredo well-known prefix 2001::/32, as there are also untrusted third-party gateways. Anyway, due to ephemeral nature of Teredo addresses, there would be virtually zero of then the public DNS, I guess.

But I still insist your servers should be polite and reply to ICMPv6 echo requests, like they do in the IPv4 world. I hope you also don’t block ICMPv6 “Packet Too Big” messages as such would break the Path-MTU discovery – see RFC 4890.

1 Like

But I guess you should apply similar ban to the Teredo well-known prefix 2001::/32, as there are also untrusted third-party gateways. Anyway, due to ephemeral nature of Teredo addresses, there would be virtually zero of then the public DNS, I guess.

That ban already exists :slight_smile: See line 132 of the bdns privateV6Networks map.

2 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.