Renewal failing on Synology

My domain is: ruk.info
My IP address is: 77.192.72.161

I ran this command:
sudo /usr/syno/sbin/syno-letsencrypt new-cert -d testurl01.ruk.info -m email@gmail.com -v

It produced this output:

DEBUG: ==== start to new cert ====
DEBUG: Server: https://acme-v02.api.letsencrypt.org/directory
DEBUG: Email: email@gmail.com
DEBUG: Domain: testurl01.ruk.info
DEBUG: ==========================
DEBUG: setup acme url https://acme-v02.api.letsencrypt.org/directory
DEBUG: GET Request: https://acme-v02.api.letsencrypt.org/directory
DEBUG: GET Request: https://acme-v02.api.letsencrypt.org/acme/new-nonce
DEBUG: Found registed account. used old account. [/usr/syno/etc/letsencrypt/account/s4Onka/]
DEBUG: apply certs with type: RSA
DEBUG: Post JWS Request: https://acme-v02.api.letsencrypt.org/acme/new-order
DEBUG: Post Request: https://acme-v02.api.letsencrypt.org/acme/new-order
DEBUG: Post JWS Request: https://acme-v02.api.letsencrypt.org/acme/authz-v3/422884116277
DEBUG: Post Request: https://acme-v02.api.letsencrypt.org/acme/authz-v3/422884116277
DEBUG: dns-01 is not support for testurl01.ruk.info
DEBUG: Setup challenge for testurl01.ruk.info with type http-01
DEBUG: Incorrect port map rules result
DEBUG: Post JWS Request: https://acme-v02.api.letsencrypt.org/acme/chall-v3/422884116277/IT3vmw
DEBUG: Post Request: https://acme-v02.api.letsencrypt.org/acme/chall-v3/422884116277/IT3vmw
DEBUG: Post JWS Request: https://acme-v02.api.letsencrypt.org/acme/authz-v3/422884116277
DEBUG: Post Request: https://acme-v02.api.letsencrypt.org/acme/authz-v3/422884116277
DEBUG: Post JWS Request: https://acme-v02.api.letsencrypt.org/acme/authz-v3/422884116277
DEBUG: Post Request: https://acme-v02.api.letsencrypt.org/acme/authz-v3/422884116277
DEBUG: close port 80.
DEBUG: ==> start new-cert.
DEBUG: generate csr & private key
DEBUG: get new-cert
DEBUG: Post JWS Request: https://acme-v02.api.letsencrypt.org/acme/finalize/65255792/318178102967
DEBUG: Post Request: https://acme-v02.api.letsencrypt.org/acme/finalize/65255792/318178102967
{"error":100,"file":"client_network.cpp","msg":"Server is not reachable."}

My web server is (include version):
Synology Webstation 3.1.0-0339

The operating system my web server runs on is (include version):
DSM 7.1.1-42962 Update 5

My hosting provider, if applicable, is:
Self hosted

I can login to a root shell on my machine (yes or no, or I don't know):
yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):
I am usually using the synology certificate control panel

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):
n/a


It seems like the script is able to talk with acme-v02.api.letsencrypt.org over https for a while and then it stops.

$ curl https://acme-v02.api.letsencrypt.org/directory
curl: (28) Failed to connect to acme-v02.api.letsencrypt.org port 443 after 131104 ms: Connection timed out

$ echo | openssl s_client -connect acme-v02.api.letsencrypt.org:443 | head
depth=2 C = US, O = Internet Security Research Group, CN = ISRG Root X1
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = R11
verify return:1
depth=0 CN = acme-v02.api.letsencrypt.org
verify return:1
CONNECTED(00000003)
DONE
---
Certificate chain
 0 s:CN = acme-v02.api.letsencrypt.org
   i:C = US, O = Let's Encrypt, CN = R11
 1 s:C = US, O = Let's Encrypt, CN = R11
   i:C = US, O = Internet Security Research Group, CN = ISRG Root X1
---
Server certificate
-----BEGIN CERTIFICATE-----

$ echo | openssl s_client -connect google.com:443 | head
depth=2 C = US, O = Google Trust Services LLC, CN = GTS Root R1
verify return:1
depth=1 C = US, O = Google Trust Services, CN = WR2
verify return:1
depth=0 CN = *.google.com
verify return:1
CONNECTED(00000003)
---
Certificate chain
 0 s:CN = *.google.com
   i:C = US, O = Google Trust Services, CN = WR2
 1 s:C = US, O = Google Trust Services, CN = WR2
   i:C = US, O = Google Trust Services LLC, CN = GTS Root R1
 2 s:C = US, O = Google Trust Services LLC, CN = GTS Root R1
   i:C = BE, O = GlobalSign nv-sa, OU = Root CA, CN = GlobalSign Root CA
---
DONE

If I wait an hour or twho, then I can curl again:

$ curl -Ii https://acme-v02.api.letsencrypt.org/directory
HTTP/2 200 
server: nginx
date: Wed, 30 Oct 2024 07:56:07 GMT
content-type: application/json
content-length: 746
cache-control: public, max-age=0, no-cache
replay-nonce: l4CEvUUqciIO0M57SNV5-vqTeg4vZfTxjb5UYQXaADnVxSCdG30
x-frame-options: DENY
strict-transport-security: max-age=604800

$ curl https://acme-v02.api.letsencrypt.org/directory
{
  "NaLxQ-rcczo": "https://community.letsencrypt.org/t/adding-random-entries-to-the-directory/33417",
  "keyChange": "https://acme-v02.api.letsencrypt.org/acme/key-change",
  "meta": {
    "caaIdentities": [
      "letsencrypt.org"
    ],
    "termsOfService": "https://letsencrypt.org/documents/LE-SA-v1.4-April-3-2024.pdf",
    "website": "https://letsencrypt.org"
  },
  "newAccount": "https://acme-v02.api.letsencrypt.org/acme/new-acct",
  "newNonce": "https://acme-v02.api.letsencrypt.org/acme/new-nonce",
  "newOrder": "https://acme-v02.api.letsencrypt.org/acme/new-order",
  "renewalInfo": "https://acme-v02.api.letsencrypt.org/draft-ietf-acme-ari-03/renewalInfo",
  "revokeCert": "https://acme-v02.api.letsencrypt.org/acme/revoke-cert"
}

But if I request a new certificate, then it stops working again.

Could my IP address somehow be blocked or rate-limited?

NB:

  • renewal was working a few weeks ago
  • I need to renew now to add a subdomain in the subject alternative names
  • ports 80 and 443 are open and reachable from the outside: Let's Debug

Is there any Geo-Location blocking?
[your HTTP site has to be reachable from the whole Internet]

1 Like

I have no indication that there is geo-location blocking, is there any tool that can help me figure out? My impression is that it is acme-v02.api.letsencrypt.org that stops responding my requests.

I am reading "msg":"Server is not reachable." as the letsencrypt server is not reachable. Could it be the error message that the letsencrypt server is sending back to the synology script because my server is not reachable?

I read that the other way around - as LE can't reach your server.

It would NOT have gotten that far [finalize] without interaction with LE.

1 Like

I agree. Though, after we reach that far and that times out, subsequent launches fail much earlier during about 1 or 2 hours.

$ sudo /usr/syno/sbin/syno-letsencrypt new-cert -d testurl01.ruk.info -m email@gmail.com -v
DEBUG: ==== start to new cert ====
DEBUG: Server: https://acme-v02.api.letsencrypt.org/directory
DEBUG: Email: email@gmail.com
DEBUG: Domain: testurl01.ruk.info
DEBUG: ==========================
DEBUG: setup acme url https://acme-v02.api.letsencrypt.org/directory
DEBUG: GET Request: https://acme-v02.api.letsencrypt.org/directory
DEBUG: GET Request: https://acme-v02.api.letsencrypt.org/acme/new-nonce
{"error":100,"file":"client_network.cpp","msg":"Server is not reachable."}

And we circle back to...
Geo-Location blocking.

1 Like

All right. Do you know of any resources that could help me identify if my ISP is geoblocking my IP address?

Is that the latest version?

1 Like

It might not be the absolute latest version of the OS, but that's the one Synology recommends for my hardware. They do not force all OS updates on systems where they won't have an impact, and it shows as "Your DSM version is up to date".

It does look like you have inconsistent results reaching the LE ACME Server API.

Do you know why your system warned about the incorrect port map rules? Maybe your outbound requests to the LE server are being affected by inconsistent local routing?

That you see this same error for finalize and new-nonce looks like comms problem. And, no, there is no LE rate limit that could cause this.

1 Like

Hi @dotvav,

Possibly these might help:

2 Likes

Good catch, thanks. I had not paid attention to that. I'll see if I can find what it means.

Thanks!

3 Likes

And here Permanent link to this check report I am seeing this with HTTP
image

Edit

Also your DNS Servers do not preform well.

4 Likes

Many thanks for your help. It is eyes opening. I'll try and see if my router or synology box is having some kind of hiccups. Also I had no idea OVH had such a poor performance outside of the EU :dizzy_face:

3 Likes

Update: the Synology integration is weird, and I don't know why they chose to not leverage certbot in the first place. I've installed certbot on a different box, with the certbot-dns-ovh plugin, and it worked like a charm. I then manually imported the cert into my synology box. I might try to find a way to automate this last step, but maybe not given that I only have this one cert to update once every 3 months.

2 Likes