I’ve disabled the BGP announcements to JointTransit in our network and suddenly it works again. Might be a problem with Let’s Encrypt, might be a problem with us. We are looking into it.
It’s still happening with Joint Transit enabled, can someone at Let’s Encrypt give us a traceroute from 2600:3000:2710:200::1d?
From us:
traceroute to 2600:3000:2710:200::1d (2600:3000:2710:200::1d), 30 hops max, 80 byte packets
1 2a06:2ec0:1::500 (2a06:2ec0:1::500) 0.314 ms 0.216 ms 0.369 ms
2 2a02:10:0:1::f7:3 (2a02:10:0:1::f7:3) 2.956 ms 1.421 ms 2.901 ms
3 * * *
4 * * *
5 100ge8-1.core1.dub1.he.net (2001:470:0:410::2) 24.536 ms 24.515 ms 24.450 ms
6 100ge5-2.core1.nyc5.he.net (2001:470:0:440::1) 74.218 ms 74.214 ms *
7 100ge4-2.core1.nyc4.he.net (2001:470:0:20a::1) 74.180 ms 74.168 ms 74.349 ms 8 2001:428:601:e::1 (2001:428:601:e::1) 74.887 ms 74.929 ms 74.917 ms
9 * * *
10 * * *
11 * * *
12 * * *
13 * * *
14 * * *
15 * * *
From others:
traceroute to 2600:3000:2710:200::1d (2600:3000:2710:200::1d), 30 hops max, 80 byte packets
1 2a03:4f00:20::1 (2a03:4f00:20::1) 4.051 ms 4.124 ms 4.288 ms
2 te0-0-24.asr1-arn01.breedbandnederland.nl (2a00:c080:0:2a::1) 4.078 ms 4.182 ms 4.287 ms
3 * * *
4 e2-2.rtr2-apl01.breedbandnederland.nl (2a00:c080:0:2b::2) 4.512 ms 4.545 ms 4.649 ms
5 te2-1.rtr1-apl01.breedbandnederland.nl (2a00:c080:0:26::1) 4.306 ms 4.530 ms 4.641 ms
6 te2-2.rtr1-hil01.breedbandnederland.nl (2a00:c080:0:25::1) 4.407 ms 4.236 ms 4.222 ms
7 te0-0-2-3.rtr1-ams01.breedbandnederland.nl (2a00:c080:0:15::2) 5.210 ms 5.321 ms 5.427 ms
8 * * *
9 * * *
10 100ge8-1.core1.dub1.he.net (2001:470:0:410::2) 17.567 ms 17.571 ms 17.604 ms
11 100ge5-2.core1.nyc5.he.net (2001:470:0:440::1) 77.342 ms 77.209 ms 77.290 ms
12 100ge4-2.core1.nyc4.he.net (2001:470:0:20a::1) 77.419 ms 77.462 ms 77.528 ms 13 2001:428:601:e::1 (2001:428:601:e::1) 77.627 ms 77.929 ms 77.982 ms
14 2001:428::205:171:200:219 (2001:428::205:171:200:219) 117.544 ms 117.283 ms 117.253 ms
15 2001:428:3801:208::2 (2001:428:3801:208::2) 118.638 ms 118.397 ms 118.563 ms
16 2600:3000:2:300::1 (2600:3000:2:300::1) 116.168 ms 115.945 ms 116.133 ms
17 2600:3000:1:230::2 (2600:3000:1:230::2) 127.470 ms 127.778 ms 127.743 ms
18 2600:3000:0:2::416 (2600:3000:0:2::416) 130.362 ms 130.469 ms 130.562 ms
19 2600:3000:3:720::2 (2600:3000:3:720::2) 127.475 ms 127.634 ms 127.368 ms
20 2600:3000:2700:1073::4 (2600:3000:2700:1073::4) 126.989 ms 127.052 ms 126.913 ms
21 * * *
22 * * *
23 * * *
edit: Can you let me know when this traceroute is being done? Traffic over joint transit is disabled since we can’t requests certificates when its on. When you want to make the traceroute I will have to enable Joint Transit first.
@rickjanssen
Here’s some ipv6 traceroutes from our firewalls to kcurv.nl.
tracert6 to kcurv.nl (2a06:2ec0:1::97), 30 hops max, 40/8 byte payload/paddata
1 2600:3000:2700:1073::1 (2600:3000:2700:1073::1) 0.545 ms 0.721 ms 0.329 ms
2 2600:3000:3:720::1 (2600:3000:3:720::1) 0.946 ms 0.774 ms 0.655 ms
3 2600:3000:0:2::415 (2600:3000:0:2::415) 0.944 ms 0.687 ms 0.569 ms
4 2600:3000:1:230::1 (2600:3000:1:230::1) 12.771 ms 12.594 ms 12.404 ms
5 2600:3000:2:328::2 (2600:3000:2:328::2) 11.973 ms 11.978 ms 11.991 ms
6 2001:1900:2100::3739 (lag-194.ear2.Denver1.Level3.net) 12.484 ms 12.559 ms 12.573 ms
7 2001:1900::3:45 (lo-22-v6.edge4.Chicago3.Level3.net) 40.760 ms 37.090 ms 37.189 ms
8 2001:1900:2100::2ab2 (IPTRIPLEPLA.edge4.Chicago3.Level3.net) 34.201 ms 34.183 ms 34.159 ms
9 2a03:9d40:fe00:5::1 (2a03:9d40:fe00:5::1) 60.129 ms 60.078 ms 59.998 ms
10 2a03:9d40:fe00:6::1 (2a03:9d40:fe00:6::1) 50.672 ms 57.718 ms 50.677 ms
11 2a03:9d40:fe00:10::1 (2a03:9d40:fe00:10::1) 68.726 ms 68.760 ms 68.709 ms
12 2a03:9d40:fe00:11::1 (2a03:9d40:fe00:11::1) 82.490 ms 82.571 ms 82.493 ms
13 * * *
14 * * *
15 * * *
and
tracert6 to kcurv.nl (2a06:2ec0:1::97), 30 hops max, 40/8 byte payload/paddata
1 2600:3000:1500:10f3::1 (2600:3000:1500:10f3::1) 0.998 ms 0.898 ms 0.500 ms
2 2600:3000:2:520::1 (2600:3000:2:520::1) 0.923 ms 0.630 ms 0.473 ms
3 2600:3000:0:2::ad (2600:3000:0:2::ad) 1.115 ms 0.801 ms 0.664 ms
4 2600:3000:0:2::aa (2600:3000:0:2::aa) 1.399 ms 0.908 ms 1.087 ms
5 2600:3000:0:2::8d (2600:3000:0:2::8d) 1.283 ms 1.293 ms 1.112 ms
6 2600:3000:2:328::2 (2600:3000:2:328::2) 0.496 ms 0.457 ms 0.459 ms
7 2001:1900:2100::3739 (lag-194.ear2.Denver1.Level3.net) 1.004 ms 1.033 ms 0.879 ms
8 2001:1900::3:45 (lo-22-v6.edge4.Chicago3.Level3.net) 25.500 ms 25.714 ms 25.560 ms
9 2001:1900:2100::2ab2 (IPTRIPLEPLA.edge4.Chicago3.Level3.net) 22.648 ms 22.656 ms 22.602 ms
10 2a03:9d40:fe00:5::1 (2a03:9d40:fe00:5::1) 48.502 ms 48.497 ms 48.456 ms
11 2a03:9d40:fe00:6::1 (2a03:9d40:fe00:6::1) 39.132 ms 39.189 ms 39.078 ms
12 2a03:9d40:fe00:10::1 (2a03:9d40:fe00:10::1) 57.168 ms 57.129 ms 57.090 ms
13 2a03:9d40:fe00:11::1 (2a03:9d40:fe00:11::1) 70.980 ms 70.875 ms 70.937 ms
14 * * *
15 * * *
16 * * *
...
29 * * 2a06:2ec0:1::97 (web0097.zxcs.nl) 114.710 ms
tracert6 to kcurv.nl (2a06:2ec0:1::97), 30 hops max, 40/8 byte payload/paddata
1 2600:3000:2700:1073::1 (2600:3000:2700:1073::1) 139.406 ms 6.996 ms 1.298 ms
2 2600:3000:3:720::1 (2600:3000:3:720::1) 1.180 ms 1.081 ms 1.006 ms
3 2600:3000:0:2::415 (2600:3000:0:2::415) 1.195 ms 0.902 ms 0.840 ms
4 2600:3000:1:230::1 (2600:3000:1:230::1) 12.832 ms 12.494 ms 12.750 ms
5 2600:3000:0:2::8e (2600:3000:0:2::8e) 12.671 ms 12.567 ms 12.564 ms
6 2600:3000:2:c0::2 (2600:3000:2:c0::2) 13.105 ms 13.079 ms 12.919 ms
7 2600:3000:2:b60::2 (2600:3000:2:b60::2) 12.551 ms 12.529 ms 12.511 ms
8 2001:1900:2100::3b99 (lag-101.ear3.Denver1.Level3.net) 13.664 ms 13.750 ms 13.629 ms
9 2001:1900::3:12e (lo-0-v6.ear1.Dallas1.Level3.net) 26.791 ms 26.752 ms 26.770 ms
10 2001:1900:4:3::41e (NTT-level3-200G.Dallas1.Level3.net) 27.625 ms 27.406 ms 2600:3002:2:160::2 (2600:3002:2:160::2) 32.613 ms
11 2001:428:2001:210:0:30:0:1 (2001:428:2001:210:0:30:0:1) 31.398 ms 31.436 ms 31.466 ms
12 2001:428:0:1:205:171:8:64 (2001:428:0:1:205:171:8:64) 31.127 ms 31.141 ms 36.464 ms
13 2001:418:0:4000::85 (ae-14.r11.dllstx09.us.bb.gin.ntt.net) 29.772 ms 29.807 ms 29.753 ms
14 2001:418:0:2000::13a (ae-0.r22.dllstx09.us.bb.gin.ntt.net) 30.461 ms 29.551 ms 29.495 ms
15 2001:418:0:2000::12 (ae-1.r22.asbnva02.us.bb.gin.ntt.net) 62.104 ms 62.177 ms 62.160 ms
16 2001:418:0:2000::3b2 (ae-0.r23.asbnva02.us.bb.gin.ntt.net) 62.168 ms 62.185 ms 62.167 ms
17 2001:418:0:6000::10e (ae-2.r25.amstnl02.nl.bb.gin.ntt.net) 139.241 ms 139.096 ms 139.306 ms
18 2001:728:0:2000::3a (ae-8.r03.amstnl02.nl.bb.gin.ntt.net) 139.684 ms 139.446 ms 139.674 ms
19 2001:728:1800:5000::2 (ge-100-0-0-30.r03.amstnl02.nl.ce.gin.ntt.net) 144.601 ms 140.380 ms 140.363 ms
20 2001:7b8:3:156::6 (2001:7b8:3:156::6) 136.858 ms 145.741 ms 136.811 ms
21 2a06:2ec0:1::97 (kcurv.nl) 136.624 ms 136.635 ms 136.688 ms
and
tracert6 to kcurv.nl (2a06:2ec0:1::97), 30 hops max, 40/8 byte payload/paddata
1 2600:3000:1500:10f3::1 (2600:3000:1500:10f3::1) 1.405 ms 0.490 ms 0.729 ms
2 2600:3000:2:520::1 (2600:3000:2:520::1) 0.719 ms 0.643 ms 0.620 ms
3 2600:3000:0:2::ad (2600:3000:0:2::ad) 1.140 ms 0.780 ms 0.656 ms
4 2600:3000:0:2::be (2600:3000:0:2::be) 0.908 ms 0.791 ms 0.532 ms
5 2600:3000:0:2::75 (2600:3000:0:2::75) 21.229 ms 21.249 ms 21.151 ms
6 2600:3000:0:2::46 (2600:3000:0:2::46) 21.167 ms 20.981 ms 20.983 ms
7 2600:3002:2:160::2 (2600:3002:2:160::2) 20.636 ms 20.612 ms 20.621 ms
8 2001:428:2001:210:0:30:0:1 (2001:428:2001:210:0:30:0:1) 19.387 ms 19.946 ms 19.438 ms
9 2001:428:0:1:205:171:8:64 (2001:428:0:1:205:171:8:64) 19.482 ms 19.431 ms 19.422 ms
10 2001:418:0:4000::85 (ae-14.r11.dllstx09.us.bb.gin.ntt.net) 18.048 ms 17.935 ms 17.915 ms
11 2001:418:0:2000::13a (ae-0.r22.dllstx09.us.bb.gin.ntt.net) 17.745 ms 18.393 ms 17.774 ms
12 2001:418:0:2000::12 (ae-1.r22.asbnva02.us.bb.gin.ntt.net) 50.098 ms 50.118 ms 50.106 ms
13 2001:418:0:2000::3b2 (ae-0.r23.asbnva02.us.bb.gin.ntt.net) 53.621 ms 50.390 ms 66.018 ms
14 2001:418:0:6000::10e (ae-2.r25.amstnl02.nl.bb.gin.ntt.net) 133.693 ms 134.896 ms 133.675 ms
15 2001:728:0:2000::3a (ae-8.r03.amstnl02.nl.bb.gin.ntt.net) 134.031 ms 134.038 ms 134.058 ms
16 2001:728:1800:5000::2 (ge-100-0-0-30.r03.amstnl02.nl.ce.gin.ntt.net) 135.818 ms 136.049 ms 135.029 ms
17 2001:7b8:3:156::6 (2001:7b8:3:156::6) 131.186 ms 139.068 ms 131.097 ms
18 2a06:2ec0:1::97 (web0097.zxcs.nl) 131.244 ms 131.439 ms 131.182 ms