Let's Encrypt is not available from Kazakhstan

Help
1

Hello! Users contacted us saying they cannot issue a certificate with an error:

Could not obtain directory: cURL error 35: Encountered end of file (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) for https://acme-v02.api.letsencrypt.org/directory

We started checking and found that the site https://acme-v02.api.letsencrypt.org/ is unavailable from the largest telecom operators in Kazakhstan, such as Kazakhtelecom, Beeline, etc. We contacted these operators and they said that there are no blockings on their side and that the blocking is most likely coming from Let's Encrypt. Is it possible to do something about this since thousands of users cannot get/update their SSL?

3 Likes
2

As a volunteer I don't have views to the internal networks of Let's Encrypt.

But, from similar issues in the past the output of below would be useful to know from an affected client

curl https://www.cloudflare.com/cdn-cgi/trace
6 Likes
3

Yes, the problem is of a global nature, the letsencrypt service is unavailable from almost all telecom operators in Kazakhstan with rare exceptions, now on the forum I see several parallel topics in the current ones.
curl https://www.cloudflare.com/cdn-cgi/trace
fl=281f15
h=www.cloudflare.com
ip=178.89.187.94
ts=1743733478.066
visit_scheme=https
uag=curl/7.29.0
colo=ALA
sliver=none
http=http/1.1
loc=KZ
tls=TLSv1.2
sni=plaintext
warp=off
gateway=off
rbi=off
kex=X25519

4 Likes
5

We are a hosting operator and we receive this response:
curl -4 -vvvvv https://acme-v02.api.letsencrypt.org/directory

  • Host acme-v02.api.letsencrypt.org:443 was resolved.
  • IPv6: (none)
  • IPv4: 172.65.32.248
  • Trying 172.65.32.248:443...
  • Connected to acme-v02.api.letsencrypt.org (172.65.32.248) port 443
  • ALPN: curl offers h2,http/1.1
  • (304) (OUT), TLS handshake, Client hello (1):
  • CAfile: /etc/ssl/cert.pem
  • CApath: none
  • (304) (IN), TLS handshake, Server hello (2):
  • (304) (IN), TLS handshake, Unknown (8):
4 Likes
6

$ curl https://www.cloudflare.com/cdn-cgi/trace
fl=967f13
h=www.cloudflare.com
ip=195.38.161.1
ts=1743749528.359
visit_scheme=https
uag=curl/8.7.1
colo=AKX
sliver=010-akx01
http=http/2
loc=KG
tls=TLSv1.3
sni=plaintext
warp=off
gateway=off
rbi=off
kex=X25519

И вот это
:2 ~ $ curl -4 https://acme-v02.api.letsencrypt.org/directory -vvv
12:54:07.167648 [0-x] == Info: [READ] client_reset, clear readers
12:54:07.204258 [0-0] == Info: Host acme-v02.api.letsencrypt.org:443 was resolved.
12:54:07.206208 [0-0] == Info: IPv6: (none)
12:54:07.207120 [0-0] == Info: IPv4: 172.65.32.248
12:54:07.208716 [0-0] == Info: [HTTPS-CONNECT] created with 1 ALPNs -> 0
12:54:07.210341 [0-0] == Info: [HTTPS-CONNECT] added
12:54:07.210673 [0-0] == Info: [HTTPS-CONNECT] connect, init
12:54:07.212245 [0-0] == Info: Trying 172.65.32.248:443...
12:54:07.213788 [0-0] == Info: [HTTPS-CONNECT] connect -> 0, done=0
12:54:07.215437 [0-0] == Info: [HTTPS-CONNECT] adjust_pollset -> 1 socks
12:54:07.218033 [0-0] == Info: [SSL] cf_connect()
12:54:07.227781 [0-0] == Info: ALPN: curl offers h2,http/1.1
12:54:07.228642 [0-0] => Send SSL data, 5 bytes (0x5)
0000: .....
12:54:07.229041 [0-0] == Info: TLSv1.3 (OUT), TLS handshake, Client hello (1):
12:54:07.229934 [0-0] => Send SSL data, 512 bytes (0x200)
0000: .........-kd...ri........T........W..J 'mU7.....c.....DTZ.2.....
0040: ....k.t.>.......,.0.........+./...$.(.k.#.'.g.....9.....3.....=.
0080: <.5./.....u...!.....acme-v02.api.letsencrypt.org................
00c0: .........................h2.http/1.1.........1.....0............
0100: .....................................+........-.....3.&.$... .q^
0140: .."......}.......!.~...h....!...................................
0180: ................................................................
01c0: ................................................................
12:54:07.237708 [0-0] == Info: [SSL] ossl_bio_cf_out_write(len=517) -> 517, err=0
12:54:07.238795 [0-0] == Info: [SSL] ossl_bio_cf_in_read(len=5) -> -1, err=81
12:54:07.239371 [0-0] == Info: [SSL] ossl_populate_x509_store, path=/etc/ssl/certs/ca-certificates.crt, blob=0
12:54:07.261166 [0-0] == Info: CAfile: /etc/ssl/certs/ca-certificates.crt
12:54:07.262130 [0-0] == Info: CApath: /etc/ssl/certs
12:54:07.262746 [0-0] == Info: [SSL] SSL_connect() -> err=-1, detail=2
12:54:07.263548 [0-0] == Info: [SSL] SSL_connect() -> want recv
12:54:07.264266 [0-0] == Info: [SSL] cf_connect() -> 0, done=0
12:54:07.265000 [0-0] == Info: [HTTPS-CONNECT] connect -> 0, done=0
12:54:07.265791 [0-0] == Info: [SSL] adjust_pollset, POLLIN fd=4
12:54:07.266509 [0-0] == Info: [HTTPS-CONNECT] adjust_pollset -> 1 socks
12:54:07.735952 [0-0] == Info: [SSL] cf_connect()
12:54:07.737203 [0-0] == Info: [SSL] ossl_bio_cf_in_read(len=5) -> 5, err=0
12:54:07.738135 [0-0] <= Recv SSL data, 5 bytes (0x5)
0000: ....z
12:54:07.738505 [0-0] == Info: [SSL] ossl_bio_cf_in_read(len=122) -> 122, err=0
12:54:07.738958 [0-0] == Info: TLSv1.3 (IN), TLS handshake, Server hello (2):
12:54:07.740311 [0-0] <= Recv SSL data, 122 bytes (0x7a)
0000: ...v.........h..U..R..I~E..&..N ...u.. 'mU7.....c.....DTZ.2.....
0040: ....k.t......+.....3.$... |.m...xo......(Rn..C.........j.n
12:54:07.743240 [0-0] == Info: [SSL] ossl_bio_cf_in_read(len=5) -> 5, err=0
12:54:07.744583 [0-0] <= Recv SSL data, 5 bytes (0x5)
0000: .....
12:54:07.745330 [0-0] == Info: [SSL] ossl_bio_cf_in_read(len=1) -> 1, err=0
12:54:07.746299 [0-0] == Info: [SSL] ossl_bio_cf_in_read(len=5) -> 5, err=0
12:54:07.747142 [0-0] <= Recv SSL data, 5 bytes (0x5)
0000: ....$
12:54:07.747904 [0-0] == Info: [SSL] ossl_bio_cf_in_read(len=36) -> 36, err=0
12:54:07.748797 [0-0] <= Recv SSL data, 1 bytes (0x1)
0000: .
12:54:07.749499 [0-0] == Info: TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
12:54:07.750445 [0-0] <= Recv SSL data, 19 bytes (0x13)
0000: .................h2
12:54:07.751403 [0-0] == Info: [SSL] ossl_bio_cf_in_read(len=5) -> 5, err=0
12:54:07.752254 [0-0] <= Recv SSL data, 5 bytes (0x5)
0000: .....
12:54:07.752978 [0-0] == Info: [SSL] ossl_bio_cf_in_read(len=2828) -> 1269, err=0
12:54:07.753919 [0-0] == Info: [SSL] ossl_bio_cf_in_read(len=1559) -> -1, err=81
12:54:07.754814 [0-0] == Info: [SSL] SSL_connect() -> err=-1, detail=2
12:54:07.755609 [0-0] == Info: [SSL] SSL_connect() -> want recv

12:54:07.756313 [0-0] == Info: [SSL] cf_connect() -> 0, done=0
12:54:07.756995 [0-0] == Info: [HTTPS-CONNECT] connect -> 0, done=0
12:54:07.757811 [0-0] == Info: [SSL] adjust_pollset, POLLIN fd=4
12:54:07.758528 [0-0] == Info: [HTTPS-CONNECT] adjust_pollset -> 1 socks
12:54:08.760674 [0-0] == Info: [SSL] cf_connect()
12:54:08.761245 [0-0] == Info: [SSL] ossl_bio_cf_in_read(len=1559) -> -1, err=81
12:54:08.761682 [0-0] == Info: [SSL] SSL_connect() -> err=-1, detail=2
12:54:08.762140 [0-0] == Info: [SSL] SSL_connect() -> want recv
12:54:08.763616 [0-0] == Info: [SSL] cf_connect() -> 0, done=0
12:54:08.764930 [0-0] == Info: [HTTPS-CONNECT] connect -> 0, done=0
12:54:08.765757 [0-0] == Info: [SSL] adjust_pollset, POLLIN fd=4
12:54:08.766480 [0-0] == Info: [HTTPS-CONNECT] adjust_pollset -> 1 socks
12:54:09.168959 [0-0] == Info: [SSL] cf_connect()
12:54:09.169625 [0-0] == Info: [SSL] ossl_bio_cf_in_read(len=1559) -> -1, err=81
12:54:09.170074 [0-0] == Info: [SSL] SSL_connect() -> err=-1, detail=2
12:54:09.170450 [0-0] == Info: [SSL] SSL_connect() -> want recv
12:54:09.170787 [0-0] == Info: [SSL] cf_connect() -> 0, done=0
12:54:09.171259 [0-0] == Info: [HTTPS-CONNECT] connect -> 0, done=0
12:54:09.172369 [0-0] == Info: [SSL] adjust_pollset, POLLIN fd=4
12:54:09.173379 [0-0] == Info: [HTTPS-CONNECT] adjust_pollset -> 1 socks
12:54:10.175757 [0-0] == Info: [SSL] cf_connect()
12:54:10.176751 [0-0] == Info: [SSL] ossl_bio_cf_in_read(len=1559) -> -1, err=81
12:54:10.177983 [0-0] == Info: [SSL] SSL_connect() -> err=-1, detail=2
12:54:10.178419 [0-0] == Info: [SSL] SSL_connect() -> want recv
12:54:10.178758 [0-0] == Info: [SSL] cf_connect() -> 0, done=0
12:54:10.179122 [0-0] == Info: [HTTPS-CONNECT] connect -> 0, done=0
12:54:10.179494 [0-0] == Info: [SSL] adjust_pollset, POLLIN fd=4
12:54:10.179837 [0-0] == Info: [HTTPS-CONNECT] adjust_pollset -> 1 socks
12:54:11.181694 [0-0] == Info: [SSL] cf_connect()
12:54:11.182153 [0-0] == Info: [SSL] ossl_bio_cf_in_read(len=1559) -> -1, err=81
12:54:11.182887 [0-0] == Info: [SSL] SSL_connect() -> err=-1, detail=2
12:54:11.183293 [0-0] == Info: [SSL] SSL_connect() -> want recv
12:54:11.183635 [0-0] == Info: [SSL] cf_connect() -> 0, done=0
12:54:11.183974 [0-0] == Info: [HTTPS-CONNECT] connect -> 0, done=0
12:54:11.185006 [0-0] == Info: [SSL] adjust_pollset, POLLIN fd=4
12:54:11.185777 [0-0] == Info: [HTTPS-CONNECT] adjust_pollset -> 1 socks

3 Likes
7

Hello!
We have same error.
Kazakhstan, provider Kazakhtelecom.
ip 45.136.57.5

4 Likes
8

Hi!
We have same error.
Kazakhstan, provider Kazakhtelecom.
curl https://www.cloudflare.com/cdn-cgi/trace
fl=967f12
h=www.cloudflare.com
ip=178.88.185.194
ts=1743760664.677
visit_scheme=https
uag=curl/7.88.1
colo=AKX
sliver=none
http=http/2
loc=KZ
tls=TLSv1.3
sni=plaintext
warp=off
gateway=off
rbi=off
kex=X25519

2 Likes
9

Hello! Same problem Kazakhtelecom.

1 Like
10

Let's Encrypt hasn't done anything to block access.

It may be some sort of network issue between Kazakhstan and Let's Encrypt.

I will begin investigating what I can see.

6 Likes
11

Apparently they tweaked something, because the work was restored! Thank you!

1 Like
12

We saw a nearly 50% traffic drop from Kazakhstan 2025-04-03 06:30 UTC and 2024-04-04 14:30 UTC, with two brief periods where traffic recovered.

It wasn't caused by anything on Let's Encrypt's side. Our traffic has otherwise been normal. As it's back to normal, no further investigation is going to happen.

image
image1260×796 76.6 KB

6 Likes
13

it worked for a bit and now there is no access again (

1 Like
14

after a short period of work, everything returned back

2 Likes
15

Hi, any news from Let's Encrypt? Is traffic being blocked at the ISP level in Kazakhstan?

1 Like
16

Dear mcpherrinm,

We are experiencing a critical issue with certificate validation that extends beyond Kazakhstan. There is a strong suspicion that the your implemented security measures may be overly aggressive.

Notably, the service briefly became accessible for approximately five minutes earlier today before failing again. This intermittent functionality suggests a potential automated blocking mechanism, possibly within your firewall.

Compounding the severity, a hosting provider managing approximately 10,000 domain names is unable to validate the certificate. Diagnostic logs are attached in previous messages above.

Our internal testing indicates that basic network utilities such as dig, traceroute, and ping are successfully querying the resource, further pointing towards a higher-level application or security layer issue, such as firewall blocking.

Furthermore, multiple hosting providers are failing to provide secure (HTTPS) access to our web pages. This widespread inability to establish secure connections is severely impacting user trust and our operational capabilities.

We request your immediate attention to identify the root cause of this widespread certificate validation failure and implement a swift resolution. The inability to provide secure access across various hosting platforms is a significant problem that requires immediate remediation.

Should any further information be required, please do not hesitate to contact us via the email address provided in our account registration profile.

Thank you for your prompt and focused attention to this critical matter.

1 Like
17

We're experiencing similar issues with a client server hosted on gohost.kz in Kazakhstan. We attempted to issue an SSL certificate using Let's Encrypt's ACME API from our web server, but the connection fails with the following error:

curl: (52) Empty reply from server

The output of the Cloudflare trace:

fl=740f191
h=www.cloudflare.com
ts=1744024433.671
visit_scheme=https
uag=curl/7.88.1
colo=SIN
sliver=none
http=http/2
loc=KZ
tls=TLSv1.3
sni=plaintext
warp=off
gateway=off
rbi=off
kex=X25519

Key technical details:

Network Configuration

  • Client IP: 109.248.170.246 (Kazakhstan, JSC "Kazakhtelecom" based on IP lookup)
  • Cloudflare colo: SIN (Singapore)
  • TLS connection: Established via Cloudflare but blocked at ISP level

ACME Failure Logs

{"level":"warn","ts":1744022902.2922828,"msg":"HTTP request failed; retrying",
"url":"https://acme-staging-v02.api.letsencrypt.org/directory",
"error":"context deadline exceeded (Client.Timeout exceeded while awaiting headers)"}

{"level":"warn","ts":1744022932.543171,"msg":"HTTP request failed; retrying",
"url":"https://acme-staging-v02.api.letsencrypt.org/directory",
"error":"context deadline exceeded..."}

Connectivity Tests

# Successful TLS handshake but failed HTTP/2 stream
$ curl -Iv https://acme-v02.api.letsencrypt.org/directory

* SSL connection: TLSv1.3 / TLS_AES_256_GCM_SHA384
* Server certificate: CN=acme-v02.api.letsencrypt.org (valid)
* Error: HTTP/2 stream 1 not closed cleanly (curl 18)

Our Observations

  1. Affects both staging (acme-staging-v02) and production (acme-v02) endpoints
  2. Occurs despite valid TLS certificate chain
  3. Cloudflare routing shows traffic exiting through Singapore (SIN) but still blocked.

Could Let's Encrypt investigate this matter further? If there are any regional restrictions or inadvertent blocks impacting Kazakhstan, it would be helpful to understand the cause and explore potential solutions. Thousands of users in Kazakhstan rely on Let's Encrypt for SSL certificates, and resolving this issue would significantly benefit the community.

18

Such problems are specifically with Kazakhtelecom. With another provider - Jusan Mobile, everything works

4 Likes
19

Such problems are not only with Kazakhtelecom, it does not work on Altel/Tele2 either, at the same time there are several operators through which it works, these are Jusan and Beeline

4 Likes
20

I confirm that the problem persists on different KZ hosting from different providers. At the same time from other locations no problems (personally from us) have been detected. I would like a sooner explanation of the problem.

1 Like
21

We continue to investigate, though as best we can tell the block is somewhere inside Kazakhstan and thus we have no ability to remove it.

7 Likes