Requesting acme-v02.api.letsencrypt.org/directory: Network is unreachable

Help
1

My domain is: disk.ekmob.ru (212.23.72.121)
I ran this command: sudo certbot --nginx
It produced this output: ValueError: Requesting acme-v02.api.letsencrypt.org/directory: Network is unreachable
nginx 1.18.0
ubuntu 20.04 lts
root yes
certbot 0.40.0 / OpenSSL 1.1.1f 31 Mar 2020

sudo traceroute acme-v02.api.letsencrypt.org -I
traceroute to acme-v02.api.letsencrypt.org (172.65.32.248), 30 hops max, 60 byte packets
 1  OPNsense.localdomain (192.168.0.1)  0.505 ms  0.467 ms  0.457 ms
 2  212.23.72.122 (212.23.72.122)  1.100 ms  1.231 ms  1.307 ms
 3  195.58.0.1 (195.58.0.1)  1.215 ms  1.414 ms  1.406 ms
 4  194.67.44.169 (194.67.44.169)  3.049 ms  3.031 ms  3.023 ms
 5  pe02.Ekaterinburg.gldn.net (79.104.240.161)  3.521 ms  3.514 ms  3.596 ms
 6  213.221.36.103 (213.221.36.103)  2.991 ms  3.043 ms  3.026 ms
 7  172.65.32.248 (172.65.32.248)  3.017 ms  3.091 ms  3.101 ms
2

Maybe an IPv6 issue?

1 Like
3

for details pls

4

@sergeybutakov, you aren't behind a firewall that restricts outgoing Internet connections, are you?

If not, this might be a configuration error of the CDN that Let's Encrypt is using to serve its API.

2 Likes
5

that's right, outgoing packets are not blocked on the gateway.

  • ufw disable on web-server

If the reason is in the CDN, help me.

6

@lestaff, could anyone help look into a possible CDN connectivity issue, or suggest other diagnostics for @sergeybutakov to use?

@sergeybutakov, did you also try something like this from your server?

curl -v https://acme-v02.api.letsencrypt.org/

It will probably show the same error, but might be useful. (And do you know how to use a packet sniffer like tcpdump or Wireshark? Sometimes the "network is unreachable" error is concealing a more specific ICMP error that could be seen with a network packet log.)

2 Likes
7

I did it with a WAN port, thinking the gateway.
This is the net output from my ip 212.23.72.121

curl -v https://acme-v02.api.letsencrypt.org/
*   Trying 172.65.32.248:443...
* TCP_NODELAY set
*   Trying 2606:4700:60:0:f53d:5624:85c7:3a2c:443...
8

We're not blocking that IP address, so something else is going on; if you can do so, I recommend using the diagnostics with OPNsense to see what's happening with the packets when you try to connect to the API.

2 Likes
9

Packet capture fails, equals zero.
tracert:

traceroute -w 2 -I  -n  -m '10' -s '212.23.72.121'   'acme-v02.api.letsencrypt.org'
traceroute to ca80a1adb12a4fbdac5ffcbc944e9a61.pacloudflare.com (172.65.32.248) from 212.23.72.121, 10 hops max, 48 byte packets
 1  212.23.72.122  0.531 ms  0.471 ms  0.332 ms
 2  195.58.0.1  1.034 ms  0.751 ms  0.722 ms
 3  194.67.44.169  2.491 ms  2.282 ms  2.224 ms
 4  79.104.240.161  3.420 ms  2.888 ms  2.739 ms
 5  213.221.36.103  2.783 ms  2.580 ms  3.696 ms
 6  172.65.32.248  2.638 ms  2.499 ms  2.421 ms
10

Even the IPv6 address fails?

1 Like
11

Yes

*   Trying 172.65.32.248:443...
* TCP_NODELAY set
*   Trying 2606:4700:60:0:f53d:5624:85c7:3a2c:443...
* TCP_NODELAY set
* Immediate connect fail for 2606:4700:60:0:f53d:5624:85c7:3a2c: Network is unreachable

ps: i am not using ipv6 on the server

12

I ask everyone to forgive me.
Thanks everyone for the response! You are a great team.

The problem was in reflection nat:
gateway / firewall / setting / advanced / reflection for port forwards: unchecked (unmarked)

curl -v https://acme-v02.api.letsencrypt.org/
*   Trying 172.65.32.248:443...
* TCP_NODELAY set
* Connected to acme-v02.api.letsencrypt.org (172.65.32.248) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):

But I can't issue a certificate.

sudo certbot --nginx -d info.ekmob.ru

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/contrib/pyopenssl.py", line 485, in wrap_socket
    cnx.do_handshake()
  File "/usr/lib/python3/dist-packages/OpenSSL/SSL.py", line 1915, in do_handshake
    self._raise_ssl_error(self._ssl, result)
  File "/usr/lib/python3/dist-packages/OpenSSL/SSL.py", line 1622, in _raise_ssl_error
    raise WantReadError()
OpenSSL.SSL.WantReadError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 376, in _make_request
    self._validate_conn(conn)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 996, in _validate_conn
    conn.connect()
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 366, in connect
    self.sock = ssl_wrap_socket(
  File "/usr/lib/python3/dist-packages/urllib3/util/ssl_.py", line 370, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/usr/lib/python3/dist-packages/urllib3/contrib/pyopenssl.py", line 488, in wrap_socket
    raise timeout("select timed out")
socket.timeout: select timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 719, in urlopen
    retries = retries.increment(
  File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 400, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/lib/python3/dist-packages/six.py", line 703, in reraise
    raise value
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 665, in urlopen
    httplib_response = self._make_request(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 379, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=conn.timeout)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 330, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='acme-v02.api.letsencrypt.org', port=443): Read timed out. (read timeout=45)

During handling of the above exception, another exception occurred:

requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='acme-v02.api.letsencrypt.org', port=443): Read timed out. (read timeout=45)
Please see the logfiles in /var/log/letsencrypt for more details.

/var/log/letsencrypt

2022-05-13 14:18:32,002:DEBUG:certbot.main:certbot version: 0.40.0
2022-05-13 14:18:32,002:DEBUG:certbot.main:Arguments: ['--nginx', '-d', 'info.ekmob.ru']
2022-05-13 14:18:32,002:DEBUG:certbot.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#manual,Plugi>
2022-05-13 14:18:32,018:DEBUG:certbot.log:Root logging level set at 20
2022-05-13 14:18:32,018:INFO:certbot.log:Saving debug log to /var/log/letsencrypt/letsencrypt.log
2022-05-13 14:18:32,020:DEBUG:certbot.plugins.selection:Requested authenticator nginx and installer nginx
2022-05-13 14:18:32,356:DEBUG:certbot.plugins.selection:Single candidate plugin: * nginx
Description: Nginx Web Server plugin
Interfaces: IAuthenticator, IInstaller, IPlugin
Entry point: nginx = certbot_nginx.configurator:NginxConfigurator
Initialized: <certbot_nginx.configurator.NginxConfigurator object at 0x7f2103593c70>
Prep: True
2022-05-13 14:18:32,357:DEBUG:certbot.plugins.selection:Selected authenticator <certbot_nginx.configurator.N>2022-05-13 14:18:32,357:INFO:certbot.plugins.selection:Plugins selected: Authenticator nginx, Installer nginx
2022-05-13 14:18:37,168:DEBUG:acme.client:Sending GET request to https://acme-v02.api.letsencrypt.org/direct>2022-05-13 14:18:37,171:DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): acme-v02.api.letsenc>
2022-05-13 14:19:23,005:DEBUG:certbot.log:Exiting abnormally:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/contrib/pyopenssl.py", line 485, in wrap_socket
    cnx.do_handshake()
  File "/usr/lib/python3/dist-packages/OpenSSL/SSL.py", line 1915, in do_handshake
    self._raise_ssl_error(self._ssl, result)
  File "/usr/lib/python3/dist-packages/OpenSSL/SSL.py", line 1622, in _raise_ssl_error
    raise WantReadError()
OpenSSL.SSL.WantReadError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 376, in _make_request
    self._validate_conn(conn)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 996, in _validate_conn
    conn.connect()
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 366, in connect
    self.sock = ssl_wrap_socket(
  File "/usr/lib/python3/dist-packages/urllib3/util/ssl_.py", line 370, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/usr/lib/python3/dist-packages/urllib3/contrib/pyopenssl.py", line 488, in wrap_socket
    raise timeout("select timed out")
socket.timeout: select timed out
During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 719, in urlopen
    retries = retries.increment(
  File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 400, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/lib/python3/dist-packages/six.py", line 703, in reraise
    raise value
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 665, in urlopen
    httplib_response = self._make_request(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 379, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=conn.timeout)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 330, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='acme-v02.api.letsencrypt.org', port=443): Rea>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/certbot", line 11, in <module>
    load_entry_point('certbot==0.40.0', 'console_scripts', 'certbot')()
  File "/usr/lib/python3/dist-packages/certbot/main.py", line 1382, in main
    return config.func(config, plugins)
  File "/usr/lib/python3/dist-packages/certbot/main.py", line 1125, in run
    le_client = _init_le_client(config, authenticator, installer)
  File "/usr/lib/python3/dist-packages/certbot/main.py", line 607, in _init_le_client
    acc, acme = _determine_account(config)
  File "/usr/lib/python3/dist-packages/certbot/main.py", line 522, in _determine_account
    acc, acme = client.register(
  File "/usr/lib/python3/dist-packages/certbot/client.py", line 176, in register
    acme = acme_from_config_key(config, key)
  File "/usr/lib/python3/dist-packages/certbot/client.py", line 46, in acme_from_config_key
    return acme_client.BackwardsCompatibleClientV2(net, key, config.server)
  File "/usr/lib/python3/dist-packages/acme/client.py", line 808, in __init__
    directory = messages.Directory.from_json(net.get(server).json())
  File "/usr/lib/python3/dist-packages/acme/client.py", line 1138, in get
    self._send_request('GET', url, **kwargs), content_type=content_type)
  File "/usr/lib/python3/dist-packages/acme/client.py", line 1088, in _send_request
    response = self.session.request(method, url, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='acme-v02.api.letsencrypt.org', port=443): Read ti>
2022-05-13 14:19:23,015:ERROR:certbot.log:An unexpected error occurred:

certbot is already the newest version (0.40.0-1ubuntu0.1)
python3-certbot-nginx is already the newest version (0.40.0-0ubuntu0.1)
OpenSSL 1.1.1f 31 Mar 2020

1 Like
13

Please show the full output of:

curl -4 https://acme-v02.api.letsencrypt.org/directory

then:

curl -6 https://acme-v02.api.letsencrypt.org/directory
1 Like
14 
curl -4 https://acme-v02.api.letsencrypt.org/directory
curl: (28) Operation timed out after 300608 milliseconds with 0 out of 0 bytes received

curl -6 https://acme-v02.api.letsencrypt.org/directory
curl: (7) Couldn't connect to server
15

How about to any other site...
Like:

curl -4 google.com
curl -6 google.com
1 Like
16 
curl -4 google.com
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="http://www.google.com/">here</A>.
</BODY></HTML>

curl -6 google.com
curl: (7) Couldn't connect to server
17

Please reconfirm your Internet IP, with:
curl -4 ifconfig.co

1 Like
18 
curl -4 ifconfig.co
212.23.72.121
19

This is very strange.

1 Like
20

A post was split to a new topic: Cloudflare dns cert issue