Does LetsEncrypt block IP or hole subnets?

Server
1

We have been using LetsEncrypt for some time now. But all of a sudden we cannot connect to acme-staging.api.letsencrypt.org. I have tried at least 30 IPs from our subnet, and from diffrent servers (with and without firewall
).
ping acme-staging.api.letsencrypt.org
PING e981.dscb.akamaiedge.net (104.123.120.101) 56(84) bytes of data.
— e981.dscb.akamaiedge.net ping statistics —
8 packets transmitted, 0 received, 100% packet loss, time 7006ms

1 Like
2

Does LetsEncrypt block IP or whole subnets?

To my knowledge we have no such blocks in place. We favour rate limiting over outright blocking.

Do you have the same trouble reaching production, or only the staging server?

Are the different servers all in the same subnet? Can you share the output from each of these commands:

    curl http://ipv4.whatismyip.akamai.com/ ; echo
    curl http://ipv6.whatismyip.akamai.com/ ; echo
    dig +short whoami.ipv4.akahelp.net TXT
    dig +short whoami.ipv6.akahelp.net TXT
    dig +short whoami.ds.akahelp.net TXT
    dig +short whoami.ds.akahelp.net TXT
    dig +short whoami.ds.akahelp.net TXT
    mtr -c 20 -w -r acme-staging.api.letsencrypt.org

(You might need to install mtr for the last one).

1 Like
3

Thanks, for answer!

All servers are on the same subnet. It is the same for production and staging

My Output
[root@s60 domains]# curl http://ipv4.whatismyip.akamai.com/ ; echo
195.204.55.60
[root@s60 domains]# curl http://ipv6.whatismyip.akamai.com/ ; echo
curl: (7) Couldn’t connect to server

[root@s60 domains]# dig +short whoami.ipv4.akahelp.net TXT
"ns" “193.75.110.126”
[root@s60 domains]# dig +short whoami.ipv6.akahelp.net TXT
[root@s60 domains]# dig +short whoami.ds.akahelp.net TXT
"ns" “193.75.110.126”
[root@s60 domains]# dig +short whoami.ds.akahelp.net TXT
"ns" “193.75.110.126”
[root@s60 domains]# dig +short whoami.ds.akahelp.net TXT
"ns" “193.75.110.126”
[root@s60 domains]# mtr -c 20 -w -r acme-staging.api.letsencrypt.org
HOST: s60.interisp-hosting.com Loss% Snt Last Avg Best Wrst StDev

  1. static1.banetele-cust.com 0.0% 20 1.0 1.4 1.0 6.8 1.3
  2. te4-1-3.ar1.prinsg39.as2116.net 0.0% 20 0.5 0.6 0.5 1.3 0.2
  3. ae4.cr1.prinsg39.as2116.net 0.0% 20 1.3 1.6 1.3 3.1 0.6
  4. he16-0-1.cr1.san110.as2116.net 0.0% 20 8.7 9.6 8.6 15.7 2.1
  5. he5-0-0.br1.fn3.as2116.net 0.0% 20 8.8 8.2 8.1 8.8 0.2
  6. oso-b3-link.telia.net 0.0% 20 8.4 8.9 8.3 18.1 2.2
  7. ??? 100.0 20 0.0 0.0 0.0 0.0 0.0

Kai Rune

1 Like
4

Hi @krorten,

I’m having our ops team reach out to Akamai to see if they can help diagnose this or suggest further debugging. Thanks for your help so far.

One more easy question to get out of the way: You can confirm that the ACME endpoints are the only websites you’re having trouble reaching from these machines? I assume all other connectivity is working as expected?

5

Yes that is correct, this is the only endpoint we are having trouble with.

Thank you for your help!

1 Like
6

Hi @krorten,

Akamai replied to ask what range of IP addresses your server’s are connecting from. I notice that you already provided your src ip and we provided it to Akamai. Apologies.

Additionally could you visit https://acme-staging.api.letsencrypt.org.t1.re and provide the output? (This link is from Akamai and appears to have a broken TLS configuration you’ll have to click through…)

Thanks!

1 Like
7

This is my result on that page

ID: 9N2Q9T
Date: Mon Feb 27 10:29:03 2017
Host: acme-staging.api.letsencrypt.org
DNS IP: 173.194.98.5 (FI) (AS15169 Google Inc.)
Client IP: 195.204.55.2 (NO) (AS2116 Broadnet AS)
Protocol: HTTP/2.0
User Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36
Cookies: yes
Cipher: TLS 1.2/TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256

Getting Edge IP …

Failed to get Edge IP(Alert popup)

1 Like
8

Thanks - passed that information down the channel.

1 Like
9

Is it possible for you to get a reverse trace from Akamai to IP: 195.204.55.60 ?

1 Like
10

Still nothing? We have some certs that need renewal,

1 Like
11

I’m afraid I don’t have any substantive news. Akamai’s last update was that they believed your ISP or an upstream is blocking/filtering UDP and that they can’t help with that. From our end this answer isn’t satisfying and doesn’t seem to explain the behaviour you’re seeing. We’re still chasing down support to try and resolve this.

I’m sorry, I know this isn’t a very comforting update, believe me when I say the frustration with Akamai is mutual. I’ll let you know if there is any further progress.

1 Like
12

My ISP and their NOC has tried everything. The wired part is that they can use my gateway and from there they can connect to the endpoint. I was also given a secondary subnet for testing, but still no luck. Therefor I wanted to get a reverse trace from Akamai to IP: 195.204.55.60

1 Like
13

Akamai has provided the following reverse trace from Akamai to 195.204.55.60

tcptraceroute 195.204.55.60 -s 104.123.120.101
Selected device eth0, address 104.123.120.101, port 56823 for outgoing packets
Tracing the path to 195.204.55.60 on TCP port 80 (www), 30 hops max
 1  80-239-148-193.customer.teliacarrier.com (80.239.148.193)  0.194 ms  0.137 ms  0.173 ms
 2  oso-b4-link.telia.net (80.91.254.243)  0.215 ms  0.215 ms  0.202 ms
 3  broadnet-ic-305679-oso-b4.c.telia.net (213.248.84.234)  0.348 ms  0.277 ms  0.284 ms
 4  he16-1-1.cr3.san110.as2116.net (195.0.240.62)  2.616 ms  0.944 ms  0.999 ms
 5  he7-1-0.cr1.san110.as2116.net (193.75.1.68)  2.513 ms  0.928 ms  0.923 ms
 6  he7-0-1.cr1.prinsg39.as2116.net (195.0.240.219)  10.013 ms  8.434 ms  8.421 ms
 7  ae0.ar1.prinsg39.as2116.net (195.0.242.185)  10.166 ms  8.200 ms  8.196 ms
 8  te0-1-0.er1.tiller.as2116.net (195.0.245.96)  13.419 ms  8.706 ms  8.668 ms
 9  * * *
10  *
11  * * *
12  * * *
13  * * *
14  * * *
15  * * *
16  * * *
1 Like
14

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.