Please Help to unblock my ip

Help
1

My domain is: fiber.gozubuyukoglu.com

I ran this command: Sophos UTM (renew)
But tried with 3 different UTM s , when I use first ISP IP1:46.196.136.250 or ISP1 IP2:46.196.136.46 everything is fine. I can succesfully create/renew certificates.
However when I use ISP2 IP1:31.223.32.179 , while using this ISP2 I couldnt create or renew any certificates.

It produced this output:
2024:05:25-12:56:24 ag-vutm letsencrypt[15140]: D Renew certificate: Processing fiber.gozubuyukoglu.com
2024:05:25-12:56:24 ag-vutm letsencrypt[15140]: D Renew certificate: + Checking domain name(s) of existing cert... unchanged.
2024:05:25-12:56:24 ag-vutm letsencrypt[15140]: D Renew certificate: + Checking expire date of existing cert...
2024:05:25-12:56:24 ag-vutm letsencrypt[15140]: D Renew certificate: + Valid till Jan 1 00:00:01 2038 GMT (Longer than 30 days). Ignoring because renew was forced!
2024:05:25-12:56:24 ag-vutm letsencrypt[15140]: D Renew certificate: + Signing domains...
2024:05:25-12:56:24 ag-vutm letsencrypt[15140]: D Renew certificate: + Generating signing request...
2024:05:25-12:56:24 ag-vutm letsencrypt[15140]: D Renew certificate: + Requesting new certificate order from CA...
2024:05:25-12:56:24 ag-vutm letsencrypt[15140]: D Renew certificate: + Received 1 authorizations URLs from the CA
2024:05:25-12:56:24 ag-vutm letsencrypt[15140]: D Renew certificate: + Handling authorization for fiber.gozubuyukoglu.com
2024:05:25-12:56:24 ag-vutm letsencrypt[15140]: D Renew certificate: + 1 pending challenge(s)
2024:05:25-12:56:24 ag-vutm letsencrypt[15140]: D Renew certificate: + Deploying challenge tokens...
2024:05:25-12:56:24 ag-vutm letsencrypt[15140]: D Renew certificate: + Responding to challenge for fiber.gozubuyukoglu.com authorization...
2024:05:25-12:56:24 ag-vutm letsencrypt[15140]: D Renew certificate: + Cleaning challenge tokens...
2024:05:25-12:56:24 ag-vutm letsencrypt[15140]: D Renew certificate: + Challenge validation has failed :frowning:
2024:05:25-12:56:24 ag-vutm letsencrypt[15140]: I Renew certificate: command completed with exit code 256
2024:05:25-12:56:24 ag-vutm letsencrypt[15140]: E Renew certificate: COMMAND_FAILED: ERROR: Challenge is invalid! (returned: invalid) (result: ["type"] "http-01"
2024:05:25-12:56:24 ag-vutm letsencrypt[15140]: E Renew certificate: COMMAND_FAILED: ["status"] "invalid"
2024:05:25-12:56:24 ag-vutm letsencrypt[15140]: E Renew certificate: COMMAND_FAILED: ["error","type"] "urn:ietf:params:acme:error:connection"
2024:05:25-12:56:24 ag-vutm letsencrypt[15140]: E Renew certificate: COMMAND_FAILED: ["error","detail"] "31.223.32.179: Fetching http://fiber.gozubuyukoglu.com/.well-known/acme-challenge/TxeM-3Xm-irzgl3jf-7eb_JSiDeB3ZuXmi3dUFY2r58: Timeout during connect (likely firewall problem)"
2024:05:25-12:56:24 ag-vutm letsencrypt[15140]: E Renew certificate: COMMAND_FAILED: ["error","status"] 400
2024:05:25-12:56:24 ag-vutm letsencrypt[15140]: E Renew certificate: COMMAND_FAILED: ["error"] {"type":"urn:ietf:params:acme:error:connection","detail":"31.223.32.179: Fetching http://fiber.gozubuyukoglu.com/.well-known/acme-challenge/TxeM-3Xm-irzgl3jf-7eb_JSiDeB3ZuXmi3dUFY2r58: Timeout during connect (likely firewall problem)","status":400}
2024:05:25-12:56:24 ag-vutm letsencrypt[15140]: E Renew certificate: COMMAND_FAILED: ["url"] "https://acme-v02.api.letsencrypt.org/acme/chall-v3/355221697062/oX97XA"
2024:05:25-12:56:24 ag-vutm letsencrypt[15140]: E Renew certificate: COMMAND_FAILED: ["token"] "TxeM-3Xm-irzgl3jf-7eb_JSiDeB3ZuXmi3dUFY2r58"
2024:05:25-12:56:24 ag-vutm letsencrypt[15140]: E Renew certificate: COMMAND_FAILED: ["validationRecord",0,"url"] "http://fiber.gozubuyukoglu.com/.well-known/acme-challenge/TxeM-3Xm-irzgl3jf-7eb_JSiDeB3ZuXmi3dUFY2r58"
2024:05:25-12:56:24 ag-vutm letsencrypt[15140]: E Renew certificate: COMMAND_FAILED: ["validationRecord",0,"hostname"] "fiber.gozubuyukoglu.com"
2024:05:25-12:56:24 ag-vutm letsencrypt[15140]: E Renew certificate: COMMAND_FAILED: ["validationRecord",0,"port"] "80"
2024:05:25-12:56:24 ag-vutm letsencrypt[15140]: E Renew certificate: COMMAND_FAILED: ["validationRecord",0,"addressesResolved",0] "31.223.32.179"
2024:05:25-12:56:24 ag-vutm letsencrypt[15140]: E Renew certificate: COMMAND_FAILED: ["validationRecord",0,"addressesResolved"] ["31.223.32.179"]
2024:05:25-12:56:24 ag-vutm letsencrypt[15140]: E Renew certificate: COMMAND_FAILED: ["validationRecord",0,"addressUsed"] "31.223.32.179"
2024:05:25-12:56:24 ag-vutm letsencrypt[15140]: E Renew certificate: COMMAND_FAILED: ["validationRecord",0] {"url":"http://fiber.gozubuyukoglu.com/.well-known/acme-challenge/TxeM-3Xm-irzgl3jf-7eb_JSiDeB3ZuXmi3dUFY2r58","hostname":"fiber.gozubuyukoglu.com","port":"80","addressesResolved":["31.223.32.179"],"addressUsed":"31.223.32.179"}
2024:05:25-12:56:24 ag-vutm letsencrypt[15140]: E Renew certificate: COMMAND_FAILED: ["validationRecord"] [{"url":"http://fiber.gozubuyukoglu.com/.well-known/acme-challenge/TxeM-3Xm-irzgl3jf-7eb_JSiDeB3ZuXmi3dUFY2r58","hostname":"fiber.gozubuyukoglu.com","port":"80","addressesResolved":["31.223.32.179"],"addressUsed":"31.223.32.179"}]
2024:05:25-12:56:24 ag-vutm letsencrypt[15140]: E Renew certificate: COMMAND_FAILED: ["validated"] "2024-05-25T09:56:12Z")
2024:05:25-12:56:24 ag-vutm letsencrypt[15140]: I Renew certificate: sending notification WARN-603
2024:05:25-12:56:24 ag-vutm letsencrypt[15140]: [WARN-603] Let's Encrypt certificate renewal failed accessing Let's Encrypt service
2024:05:25-12:56:24 ag-vutm letsencrypt[15140]: D Renew certificate: disconnected from Confd
2024:05:25-12:56:24 ag-vutm letsencrypt[15140]: D Renew certificate: execution lock removed
2024:05:25-12:56:24 ag-vutm letsencrypt[15140]: I Renew certificate: execution completed (CSRs renewed: 0, failed: 1)

My web server is (include version): Sophos UTM 9.719-3

The operating system my web server runs on is (include version): Sophos UTM 9.719-3

My hosting provider, if applicable, is: KABLONET & TURK.NET

I can login to a root shell on my machine (yes or no, or I don't know): yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): yes, webadmin of Sophos UTM 9.719-3

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):

Possible "dehydrated"

----command example from logs below---
running command: /var/storage/chroot-reverseproxy/usr/dehydrated/bin/dehydrated -x -f /var/storage/chroot-reverseproxy/usr/dehydrated/conf/config -c --accept-terms --domain fiber.gozubuyukoglu.com

2

Your IP is most likely not blocked, it's more likely that fiber.gozubuyukoglu.com just doesn't accept http (TCP port 80) challenges.

4 Likes
3

It is accepting while creating or renewing certificates. This is automatically done by Sophos UTM.

4

46.196.136.46 this is another sophos fw with a public ip but dynamic, which is also not serving anything from 80 but can create or renew certificates.

5

The Let's Encrypt API Server can block IPs but this is rare. Yours is not blocked.

If your IP was blocked it would not get any response from the Let's Encrypt API Server. The above line in your log shows you do get a response so you are not blocked.

This error says the Let's Encrypt validation server (not the same as the API Server) timed out when trying its HTTP request to validate your domain.

You should review all the comms setting differences between this one and the other that works. Check port 80 configuration and routing from the outermost edge of your connection to the public internet to this device.

I don't know Sophos at all so cannot give specific advice on this. It probably has logs so you should check those. It can be difficult to debug systems that don't listen on port 80 except during the challenge.

Update:
From my own test server I reach an Apache server for both HTTP and HTTPS. Perhaps you have certain IPs blocked preventing requests. You should check any such firewall settings.

7 Likes
6

Found a temporary solution ,
on Sophos UTM
modified "/var/storage/chroot-reverseproxy/usr/dehydrated/conf/config" file
changed #IP_VERSION= to IP_VERSION=6 , so after now the system is updating certificated via only checking ipv6 (which has no blocking by anybody)

--------------------log below--------------------------
2024:05:25-19:59:02 ag-vutm letsencrypt[1807]: I Renew certificate: running command: /var/storage/chroot-reverseproxy/usr/dehydrated/bin/dehydrated -x -f /var/storage/chroot-reverseproxy/usr/dehydrated/conf/config -c --accept-terms --domain fiber.gozubuyukoglu.com
2024:05:25-19:59:21 ag-vutm letsencrypt[1807]: D Renew certificate: *** Command output ***: # INFO: Using main config file /var/storage/chroot-reverseproxy/usr/dehydrated/conf/config
2024:05:25-19:59:21 ag-vutm letsencrypt[1807]: D Renew certificate: + Fetching account URL...
2024:05:25-19:59:21 ag-vutm letsencrypt[1807]: D Renew certificate: + Creating chain cache directory /var/storage/chroot-reverseproxy/var/lib/dehydrated/cert_data/chains
2024:05:25-19:59:21 ag-vutm letsencrypt[1807]: D Renew certificate: Processing fiber.gozubuyukoglu.com
2024:05:25-19:59:21 ag-vutm letsencrypt[1807]: D Renew certificate: + Checking domain name(s) of existing cert... unchanged.
2024:05:25-19:59:21 ag-vutm letsencrypt[1807]: D Renew certificate: + Checking expire date of existing cert...
2024:05:25-19:59:21 ag-vutm letsencrypt[1807]: D Renew certificate: + Valid till Jan 1 00:00:01 2038 GMT (Longer than 30 days). Ignoring because renew was forced!
2024:05:25-19:59:21 ag-vutm letsencrypt[1807]: D Renew certificate: + Signing domains...
2024:05:25-19:59:21 ag-vutm letsencrypt[1807]: D Renew certificate: + Generating signing request...
2024:05:25-19:59:21 ag-vutm letsencrypt[1807]: D Renew certificate: + Requesting new certificate order from CA...
2024:05:25-19:59:21 ag-vutm letsencrypt[1807]: D Renew certificate: + Received 1 authorizations URLs from the CA
2024:05:25-19:59:21 ag-vutm letsencrypt[1807]: D Renew certificate: + Handling authorization for fiber.gozubuyukoglu.com
2024:05:25-19:59:21 ag-vutm letsencrypt[1807]: D Renew certificate: + 1 pending challenge(s)
2024:05:25-19:59:21 ag-vutm letsencrypt[1807]: D Renew certificate: + Deploying challenge tokens...
2024:05:25-19:59:21 ag-vutm letsencrypt[1807]: D Renew certificate: + Responding to challenge for fiber.gozubuyukoglu.com authorization...
2024:05:25-19:59:21 ag-vutm letsencrypt[1807]: D Renew certificate: + Challenge is valid!
2024:05:25-19:59:21 ag-vutm letsencrypt[1807]: D Renew certificate: + Cleaning challenge tokens...
2024:05:25-19:59:21 ag-vutm letsencrypt[1807]: D Renew certificate: + Requesting certificate...
2024:05:25-19:59:21 ag-vutm letsencrypt[1807]: D Renew certificate: + Using preferred chain with CN = ISRG Root X1
2024:05:25-19:59:21 ag-vutm letsencrypt[1807]: D Renew certificate: + Checking certificate...
2024:05:25-19:59:21 ag-vutm letsencrypt[1807]: D Renew certificate: + Done!
2024:05:25-19:59:21 ag-vutm letsencrypt[1807]: D Renew certificate: + Creating fullchain.pem...
2024:05:25-19:59:21 ag-vutm letsencrypt[1807]: D Renew certificate: + Done!
2024:05:25-19:59:21 ag-vutm letsencrypt[1807]: D Renew certificate: + Running automatic cleanup
2024:05:25-19:59:21 ag-vutm letsencrypt[1807]: I Renew certificate: command completed with exit code 0
2024:05:25-19:59:21 ag-vutm letsencrypt[1807]: D Renew certificate: resetting CSR's renew and error fields
2024:05:25-19:59:21 ag-vutm letsencrypt[1807]: I Renew certificate: temporary certificate exists, updating from /var/storage/chroot-reverseproxy/var/lib/dehydrated/cert_data/certs/fiber.gozubuyukoglu.com/fullchain.pem
2024:05:25-19:59:21 ag-vutm letsencrypt[1807]: D Renew certificate: setting CSR's certificate reference
2024:05:25-19:59:21 ag-vutm letsencrypt[1807]: I Renew certificate: updated certificate REF_UukVVOkTaIWb of CSR REF_CaCsrFibergozub
2024:05:25-19:59:21 ag-vutm letsencrypt[1807]: D Renew certificate: removing directory /var/storage/chroot-reverseproxy/var/lib/dehydrated/cert_data
2024:05:25-19:59:21 ag-vutm letsencrypt[1807]: D Renew certificate: disconnected from Confd

1 Like
7

The Let's Encrypt validation servers will use IPv6 first if you have an AAAA record in your DNS. If you don't block any IPv6 that is probably why it now succeeds.

You must have also added the AAAA record along with the change you described in Sophos. There was not an AAAA record in your DNS when you first posted here. We know this from the Verbose results of the Let's Debug test results.

The Let's Encrypt API server supports IPv4 and IPv6 whichever your client chooses to use. Accessing the API server was never a problem for you (in this thread).

I explain all this to help you understand what is happening. Hopefully this will lead to a more reliable solution for you.

2 Likes
8

Hi, @Arda,

I'm afraid I have bad news for you. You tried to post a followup to this thread that contained a copy of your private key, embedded in more of your Sophos UTM log output.

You need to stop using this private key, change it, and immediately replace any certificates that have used it. We will be revoking all such certificates within 24 hours. Once a private key has been shared over the Internet (even though your post was flagged and held for review), it is unsafe to use, and our compliance guidelines require us to block it.

I'm so sorry about this inconvenience. You had no reason to expect this would happen; I am very surprised that Sophos writes private key material to their logs. I'll reach out to them about this and request a fix to ensure it cannot happen again.

6 Likes
9

Thank you , I wil regenerate it ASAP.

5 Likes
10

I suspect my ISP is blocking that verification servers of yours...

11

should I also renew my letsencrypt account ?

1 Like
12

Sophos is not normally write priv to logs but unfortunately I opened debug mode via "cc set ca letsencrypt debug 1" for details

2 Likes
13

You might as well. I don't think the compromised key was used as your account key, just for a certificate, but it's harmless to replace your account and it would be the safest thing to do.

5 Likes
14

Ok, I think so but anyway maybe I also shared some account info...

15

Changed all of them and renewed certificates, they(old) should be revoked soon.

2 Likes
16

You might have inside information for this specific information, but for other people finding this thread some other time: if your account has rate limit exemptions coupled to it, you probably shouldn't replace your account itself, but only do an account key rollover, so that you'll retain your rate limit override(s).
That said, while account key rollovers are part of the ACME specifications, not every ACME client supports this unfortunately.

2 Likes
17

Speaking of the Rate Limits: testing and debugging are best done using the Staging Environment.

FYI -

https://tools.letsdebug.net/cert-search?m=domain&q=fiber.gozubuyukoglu.com&d=2160
"The Registered Domain (gozubuyukoglu.com) has used 32 of 50 weekly certificates."

image
image1257×987 27.8 KB

Edit:

Also https://zonemaster.net/en/result/fc3598a042c8ff5b shows Name Server Delegation Issues

image
image1052×718 13.6 KB

and here is another view of the DNS Zone with some ERRORs Hardenize Report: gozubuyukoglu.com

2 Likes
18

FWIW, Sophos has acknowledged the issue, so I expect they'll fix it in an upcoming release.

7 Likes
19

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.