Degraded performance post-maintenance?

Help
1

Are there known ongoing issues post the latest maintenance per the status page, which was moved to monitoring at 20:08 UTC? I'm seeing API connection errors from 18:30 onward that haven't resolved.

curl -sS -m 8 -o /dev/null -w '%{http_code} tls=%{time_appconnect}\n' -I https://acme-v02.api.letsencrypt.org/acme/new-nonce
200 tls=4.939589

curl -sS -m 8 -o /dev/null -w '%{http_code} tls=%{time_appconnect}\n' -I https://acme-v02.api.letsencrypt.org/acme/new-nonce
200 tls=2.922109

curl -sS -m 8 -o /dev/null -w '%{http_code} tls=%{time_appconnect}\n' -I https://acme-v02.api.letsencrypt.org/acme/new-nonce
200 tls=0.088846

curl -sS -m 8 -o /dev/null -w '%{http_code} tls=%{time_appconnect}\n' -I https://acme-v02.api.letsencrypt.org/acme/new-nonce
curl: (35) error:0A000126:SSL routines::unexpected eof while reading
000 tls=0.000000

curl -sS -m 8 -o /dev/null -w '%{http_code} tls=%{time_appconnect}\n' -I https://acme-v02.api.letsencrypt.org/acme/new-nonce
curl: (28) Operation timed out after 8001 milliseconds with 0 bytes received
000 tls=0.000000

I can reproduce this fairly broadly from different continents.

2

Thanks. I've updated status.io to indicate a partial service disruption, and are investigating.

3

Hi team, Nhia from Zendesk here. We are still seeing multiple Acme::Client::Error::RateLimited: Service busy; retry later errors following the latest update @ July 2, 2026 23:15 UTC for Let's Encrypt Status

Would those errors still be related?

Thanks for your attention in advance.

4

Thanks. Yes, it looks like we have some performance issues still. We will continue to investigate

5

We've rearranged some workloads to avoid maxed out CPUs, and our latency (& overload rate limits) are both looking better.

6

Time to start a third datacenter? "High Assurance Datacenter 3"?
Since LE seems to start outgrowing themselves now.

Guess that would require a new complete audit by the root programs? Or is it enough with just a physical security audit and then the rest of the security can "fly" under the previous datacenter's audits provided the software configuration of the servers are identical?

7

It wasn't a case of "outgrowing our current compute"; it was an issue with how jobs were allocated to current hosts. A component in Boulder (our software) appears to have been misbehaving and consuming excess CPU on a single instance, which degraded colocated services. That shouldn't have happened, and we're trying to figure out what went wrong still.

The costs of adding a third DC are significant, and would be counterproductive to increasing capacity: It would be a better use of funding to get additional servers for our existing pair of DCs than to have a third DC. Audit costs scale up with more DCs, but are going to be dwarfed by the costs of actually running them. A half-dozen app servers, a pair of DB servers, a pair of HSMs, firewalls and switches, a bunch of nvme storage, etc all add up pretty quick. Not to mention the human costs of our travel to and from another location, and the ongoing maintenance of all the extra servers.