LetsEncrypt via Cpanel random timeouts


#1

We are having a problem with some of our servers where connections to Letsencypt times out randomly.
For example


$ curl -v -i https://acme-v01.api.letsencrypt.org/acme/authz/VPtRxUOkgR_O68N2Ym9z71yiT1EHbzca5ruj6ulaQs8

  • About to connect() to acme-v01.api.letsencrypt.org port 443 (#0)
  • Trying 23.13.129.221…
  • Connection timed out
  • Trying 2600:1417:3f:284::3a8e…
  • Failed to connect to 2600:1417:3f:284::3a8e: Network is unreachable
  • Trying 2600:1417:3f:289::3a8e…
  • Failed to connect to 2600:1417:3f:289::3a8e: Network is unreachable
  • Failed connect to acme-v01.api.letsencrypt.org:443; Network is unreachable
  • Closing connection 0
    curl: (7) Failed to connect to 2600:1417:3f:284::3a8e: Network is unreachable

We do not have ipv6 setup, so as soon as the ipv4 address timesout, its obvious that would fail with the ipv6 address it tries with.

It timesout one moment like above and just works , a few seconds later and the cycle continues.
And we can see the following in apache error logs


[Thu Jan 10 23:09:55.894714 2019] [ssl:error] [pid 1995097:tid 140700729415424] (101)Network is unreachable: [client 35.235.94.249:36156] AH01974: could not connect to OCSP responder ‘ocsp.int-x3.letsencrypt.org
[Fri Jan 11 00:13:02.031599 2019] [ssl:error] [pid 1996149:tid 140700670666496] (101)Network is unreachable: [client 66.249.79.192:38099] AH01974: could not connect to OCSP responder ‘ocsp.int-x3.letsencrypt.org
[Fri Jan 11 00:13:25.327621 2019] [ssl:error] [pid 1995210:tid 140700712630016] (101)Network is unreachable: [client 66.249.79.223:55793] AH01974: could not connect to OCSP responder ‘ocsp.int-x3.letsencrypt.org
[Fri Jan 11 00:13:44.460627 2019] [ssl:error] [pid 1995210:tid 140700821735168] (101)Network is unreachable: [client 112.202.111.109:63385] AH01974: could not connect to OCSP responder ‘ocsp.int-x3.letsencrypt.org
[Fri Jan 11 00:14:38.626841 2019] [ssl:error] [pid 1996274:tid 140700662273792] (101)Network is unreachable: [client 84.200.25.151:60282] AH01974: could not connect to OCSP responder ‘ocsp.int-x3.letsencrypt.org
[Fri Jan 11 00:15:35.487632 2019] [ssl:error] [pid 1995097:tid 140700771378944] (101)Network is unreachable: [client 66.249.79.187:65133] AH01974: could not connect to OCSP responder ‘ocsp.int-x3.letsencrypt.org
[Fri Jan 11 00:16:46.617615 2019] [ssl:error] [pid 1995210:tid 140700687451904] (101)Network is unreachable: [client 157.55.39.5:5633] AH01974: could not connect to OCSP responder ‘ocsp.int-x3.letsencrypt.org
[Fri Jan 11 00:18:31.926624 2019] [ssl:error] [pid 2723962:tid 140700830127872] (101)Network is unreachable: [client 62.210.215.119:32839] AH01974: could not connect to OCSP responder ‘ocsp.int-x3.letsencrypt.org
[Fri Jan 11 00:18:55.452626 2019] [ssl:error] [pid 1996430:tid 140700712630016] (101)Network is unreachable: [client 157.55.39.148:2784] AH01974: could not connect to OCSP responder ‘ocsp.int-x3.letsencrypt.org
[Fri Jan 11 00:19:06.715752 2019] [ssl:error] [pid 1995097:tid 140700796557056] (101)Network is unreachable: [client 66.249.79.116:50968] AH01974: could not connect to OCSP responder ‘ocsp.int-x3.letsencrypt.org
[Fri Jan 11 00:19:17.404614 2019] [ssl:error] [pid 1995209:tid 140700662273792] (101)Network is unreachable: [client 157.55.39.188:2029] AH01974: could not connect to OCSP responder ‘ocsp.int-x3.letsencrypt.org
[Fri Jan 11 00:19:42.762627 2019] [ssl:error] [pid 1996149:tid 140700821735168] (101)Network is unreachable: [client 66.249.79.114:47011] AH01974: could not connect to OCSP responder ‘ocsp.int-x3.letsencrypt.org
[Fri Jan 11 00:20:02.623638 2019] [ssl:error] [pid 1995998:tid 140700687451904] (101)Network is unreachable: [client 66.249.79.29:59837] AH01974: could not connect to OCSP responder ‘ocsp.int-x3.letsencrypt.org


These are shared servers and we have quite a number of customers using letsencrypt. I have gone over the rate limiting rules and it does not appear we are hitting them yet. IF we were, we should definitely know from the response.

Anyone else had similar issues ? We did take this with cpanel but they have bailed out of this in the end stating network issues outside of cpanel software. Any help here is appreciated.


#2

Have you tried using:
acme-v02.api.letsencrypt.org


#3

You need to talk to your NOC with basic reproduction steps (like intermittently failing curls from your servers’ IPv4 prefix). They will either confirm or disprove your complaint and you can go from there.