One of two servers suddenly stops renewing

Help
1

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. https://crt.sh/?q=example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is:corp.networkingtechnology.org

I ran this command:certbot renew --dry-run

It produced this output:Failed

My web server is (include version):Apache 2.4.37

The operating system my web server runs on is (include version):Alma Linux 8.10

My hosting provider, if applicable, is:

I can login to a root shell on my machine (yes or no, or I don't know):yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):1.22.0

One server hosts 2 x phpBB boards, which are at present for internal use, so registrations are not accepted. Nothing has changed on this server other than every week I run dnf updates and keep them up to date. It's been working fine and the renew process has been fine.

Now it's stopped trying to renew. I ran LetsDebug and it tells me nothing is wrong.

The second server runs a Postfix/Dovecot mail server.
If I run certbot renew --dry run. It works fine

Both servers on the OPNSense firewall have rules that allow traffic of Ports 80 and 443 (along with the thousands of attacks I've suffered from the USA, China and Russia since Country Blocklists are no longer allowed).

I'm starting to wonder if Lets Encrypt is worth all the hassles it produces.

TWO servers on the same subnet, same OS, same everything. One renews perfectly. The other fails and LetsDebug says nothing is wrong - Except ONE of them stopped working for NO reason, because the server is stagnant other than updates.

2

Hi!

Post entire Certbot output.

This says your webserver is LiteSpeed:

$ curl -I http://corp.networkingtechnology.org/.well-known/acme-challenge/test
HTTP/1.1 404 Not Found
Connection: Keep-Alive
Keep-Alive: timeout=5, max=100
content-type: text/html
date: Fri, 13 Dec 2024 09:22:26 GMT
server: LiteSpeed
3 Likes
3

Well it loads as httpd, the version is Apache, so how can it be LiteSpeed? Who is screwing what up?

Any ideas on how to fix it?

4

Well, it's up to you to know how your web infra is set up. I can give a few pointers.

Is this the expected IP?

$ dig +short corp.networkingtechnology.org
192.81.170.6

Is this the expected page when visiting http://corp.networkingtechnology.org?

1421×832 95.4 KB

3 Likes
5

No, it isn't. HostUpon hosts Networkingtechnology.org and it has its own Let's Encrypt certificate.

Corp.networkingtechnology.org is a sub-domain of the above. The DNS for the subdomain is hosted by Dynu and I used their authentication to use DNS-01 for corp, support and writers.

NOTHING has changed on that server other than updates. It worked fine for some time and then just stopped renewal attempts. The fact it has valid certificates until January should tell you it was working.

So what's gone wrong? The entire corp domain is on a subnet 79.132.230.56/29 and the Certificate was created on Dynu.

Why has it suddenly decided to use the wrong IP address for the sub-domain?

6

Did you have a look at Dynu's control panel? Is an IP from 79.132.230.56/29 range assigned to the corp subdomain? If it is—then you need to talk to Dynu support for them to fix it.

3 Likes
7

Leave the blocks on 443 and exempt the /.well-known/acme-challenge path from redirection to 443. This will allow your HTTP-01 challenges to succeed on an unfiltered port 80 without preventing you from using geo-blocking on your HTTPS traffic.

3 Likes
8

Can you explain how to do that on OPNSense? I've tried several different ways and none of them has worked

9

The only thing done in your firewall is your geo-blocking. You need to handle the redirect exemption on your webserver. Here is one example that may help you.

          <IfModule mod_rewrite.c>
          RewriteEngine On
          # Exclude Let's Encrypt ACME from HTTPS redirect
          RewriteCond %{HTTPS} !=on
          RewriteCond %{REQUEST_URI} !^/.well-known/acme-challenge
          RewriteRule ^/(.*) https://%{SERVER_NAME}%{REQUEST_URI} [R=301,L]
          </IfModule>
2 Likes
10

Probably someone has messed up the DNS zone if 192.81.170.6 is not the correct IP address for corp.networkingtechnology.org. You can see at e.g. corp.networkingtechnology.org | DNSViz that all Dyno nameservers respond with that, apparently incorrect IP address.

2 Likes