Let's Encrypt API is performing poorly

Help
1

I have been unable to renew the certificate for my domain because the acme-v02 API is intermittently unresponsive. I cannot reliably establish a connection to the API servers, making it impossible to renew my certificate.

I can reproduce these connection errors on my macbook, as well as my Synology which hosts my services.

My domain is: *.int.groundsfam.com

I ran this command: curl --connect-timeout 10 https://acme-v02.api.letsencrypt.org/directory

It produced this output: curl: (28) Failed to connect to acme-v02.api.letsencrypt.org port 443 after 10001 ms: Timeout was reached

My web server is (include version): Synology DSM 7.2.2-72806 Update 4

The operating system my web server runs on is (include version): Synology DSM 7.2.2-72806 Update 4

My hosting provider, if applicable, is: myself

I can login to a root shell on my machine (yes or no, or I don't know): yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): curl 8.7.1

2

I don't see any lag from my own test server. And, so far no service announcement from Let's Encrypt: https://letsencrypt.status.io/

Are you sure the problem is not on your end?

What do these show

curl https://www.cloudflare.com/cdn-cgi/trace

curl -I https://google.com
3 Likes
3

Thanks for the reply. It's hard to know if this problem is specific to me or not, but I do see some activity on down detector that indicates others might also be seeing degraded performance.

Here's what those show for me:

curl https://www.cloudflare.com/cdn-cgi/trace
fl=509f181
h=www.cloudflare.com
ip=192.63.67.72
ts=1756068053.762
visit_scheme=https
uag=curl/8.7.1
colo=ATL
sliver=010-tier1
http=http/2
loc=US
tls=TLSv1.3
sni=plaintext
warp=off
gateway=off
rbi=off
kex=X25519

curl -I https://google.com
HTTP/2 301
location: https://www.google.com/
content-type: text/html; charset=UTF-8
content-security-policy-report-only: object-src 'none';base-uri 'self';script-src 'nonce-SvPV4oS-OEqO2SSvM9QQeg' 'strict-dynamic' 'report-sample' 'unsafe-eval' 'unsafe-inline' https: http:;report-uri https://csp.withgoogle.com/csp/gws/other-hp
date: Sun, 24 Aug 2025 20:40:49 GMT
expires: Tue, 23 Sep 2025 20:40:49 GMT
cache-control: public, max-age=2592000
server: gws
content-length: 220
x-xss-protection: 0
x-frame-options: SAMEORIGIN
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
1 Like
4

Hmm. Do you have a version of traceroute that allows these options?

sudo traceroute -T -p 443 acme-v02.api.letsencrypt.org
3 Likes
5

Yep, that works on my synology (my macbook has a darwin traceroute with different options). Output there:

traceroute to acme-v02.api.letsencrypt.org (172.65.32.248), 30 hops max, 60 byte packets
 1  192.168.86.1 (192.168.86.1)  0.224 ms  0.295 ms  0.284 ms
 2  syn-192-063-066-001.res.spectrum.com (192.63.66.1)  10.730 ms  10.907 ms  12.539 ms
 3  lag-62.hcr02ausutxla.netops.charter.com (24.27.12.101)  9.356 ms  11.051 ms  9.439 ms
 4  lag-21.ausxtxir02r.netops.charter.com (24.28.90.74)  14.286 ms  10.813 ms  10.891 ms
 5  lag-22.rcr01hstqtx02.netops.charter.com (24.175.41.48)  20.913 ms  20.889 ms  20.977 ms
 6  lag-416-10.hstqtx0209w-bcr00.netops.charter.com (66.109.9.88)  22.477 ms lag-26-10.hstqtx0209w-bcr00.netops.charter.com (66.109.1.218)  21.621 ms lag-416-10.hstqtx0209w-bcr00.netops.charter.com (66.109.9.88)  21.435 ms
 7  lag-800.pr0.hou50.netops.charter.com (66.109.5.236)  23.977 ms  17.668 ms  15.758 ms
 8  syn-066-109-002-031.inf.spectrum.com (66.109.2.31)  43.228 ms  21.183 ms  21.275 ms
 9  172.65.32.248 (172.65.32.248)  20.095 ms *  22.566 ms
6

Same -T -p 443 options? Because that's a TCP connection to that domain.

Is the curl to /directory still failing?

Transient problems are not unusual. Not sure how reliable down detector is when reporting only a few outages. LE issues over 7 million certs per day. They have automated health monitoring but sometimes low volume problems sneak under their reporting thresholds.

2 Likes
7

Yep, sudo traceroute -T -p 443 acme-v02.api.letsencrypt.org is consistently working from what I can tell. curl https://acme-v02.api.letsencrypt.org/directory is sometimes hanging on the other hand.

2 Likes
8

"sometimes" ? Does more than 10s timeout allow "always" :slight_smile: Not that it is expected but part of isolating

What about curl to staging

curl https://acme-staging-v02.api.letsencrypt.org/directory

Does curl with -v show anything interesting to production /directory ?

curl -v https://acme-v02.api.letsencrypt.org/directory
3 Likes
9

"sometimes" ? Does more than 10s timeout allow "always" :slight_smile: Not that it is expected but part of isolating

No, it either works very quickly or never. I just used the 10s cutoff to get an error message to post here.

What about curl to staging

Same behavior.

Does curl with -v show anything interesting to production /directory

curl -v https://acme-v02.api.letsencrypt.org/directory
* Host acme-v02.api.letsencrypt.org:443 was resolved.
* IPv6: (none)
* IPv4: 172.65.32.248
*   Trying 172.65.32.248:443...
^C
10

I think I know for sure that this is not a problem with let's encrypt, but with my ISP (spectrum). I ran mtr and observed high packet loss at the last hop before reaching the let's encrypt API:

Host                                      Loss%   Snt   Last   Avg  Best  Wrst StDev
 [skipping earlier hosts with 0% packet loss]
 8. syn-066-109-002-031.inf.spectrum.com   41.5%    41   22.4  27.8  19.2  85.5  13.7
 9. 172.65.32.248                          36.6%    41   22.8  24.3  18.4  49.9   6.

This post can be marked resolved. Thanks for rubber ducking with me :slight_smile:

7 Likes
11

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.