Migrated to Cloudflare - Lets Encrypt Cannont renew

Help
1

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: *rclwiring.com

I ran this command: Using wacs.exe tool - R to renew

It produced this output: [IIS] app.rclwiring.com, (any host) - renewed 30 times, due now, 44 errors like "Authorization failed: {
"type": "urn:ietf:params:acme:error:unauthorized",
"detail": "XXX.XXX.XXX.XXX: Invalid response from http://app.rclwiring.com/.well-known/acme-challenge/2AnnxAYkV0D-5urLVdwdpLwlbyVt3Kn8t-DCTHwXtBo: 404",
"status": 403
}"

My web server is (include version): IIS

The operating system my web server runs on is (include version): Windows Server 2016

My hosting provider, if applicable, is: Cloudflare

I can login to a root shell on my machine (yes or no, or I don't know): Yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): Wordpress

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): Unsure.

Recently migrated from former domain to this current one, These are inherited sites so I'm not as webserver savvy as I should be.

2

I suspect it has to do with Cloudflares Flexible vs Full settings however I cannot get it to renew with either setting active.

3

An Apache server on Ubuntu is currently responding to requests to that domain name. Not IIS on a Windows Server

Has your public IP address changed? Does your DNS reflect the current IP?

Request to: app.rclwiring.com/209.214.204.234, Result: [Address=209.214.204.234,Address Type=IPv4,Server=Apache/2.4.41 (Ubuntu),HTTP Status=404],

3 Likes
4

Those are settings for when you "proxy" your domain name in Cloudflare.

But, the app.rclwiring.com is not proxied at the moment.

The DNS entries for that domain would be very different when proxied. Right now your DNS has just one IP address that an Apache server replies to

4 Likes
5

Thanks for the reply, these are old systems I inherited so I'll give as much info as I know. Public IP has remained the same, DNS records that I migrated mirror from the last registrar.

Understood so that setting has no effect. But as your previous comment says, you are correct an Apache server is responding.

1 Like
6

Has that always been true? I don't understand why you are using wacs on Windows Server to get the cert for an Apache server on Ubuntu

It is technically possible but takes considerable coordination amongst your tools for that to work. And, there are far easier ways.

I also see you now have your domain name proxied at Cloudflare. It is hard to give advice without knowing more about your objective.

3 Likes
7

Reaching out to the old guard to see if that is the case, as I said I inherited these systems. I was testing if a proxy would give me new results and it did not.

Would you mind sharing those easier ways as I'll likely be rebuilding this from the ground up so I understand it better.

I appreciate your time either way - goal was just to get the cert to renew - while it renews the others that weren't expired you are bringing to light things I was not aware of and am currently researching myself. Thank you again.

8

I could if you can explain why you are running a Windows tool to get a cert for an Apache system on Ubuntu :slight_smile:

Do you have access to that Ubuntu system?

4 Likes
9

I wish I could explain that but I currently do not have the knowledge of why.

I have access to the only ubuntu system I know of in my environment - so yes maybe? You really are leading the blind here apologies. What are you thinking?

1 Like
10

I am thinking there is a different tool used for that Apache / Ubuntu system

Can you run commands on that Ubuntu system? Like can you show output of these?

sudo ss -pant | grep -i listen | grep -Ei ':80|:443'
sudo apache2ctl -t -D DUMP_VHOSTS
curl -4 https://ifconfig.io
3 Likes
11

Its listening on ports 80 and 443, the site in question shows in the namevhost list and it shows my non-alias public ip.

12

This will work much better if you show the results of the commands. I, along with most of the other helpers here, are volunteers. We donate our personal time and expertise for free.

How you interpret and convey the results of those commands will be different than I do. And, some of those are just step 1 for discovering what your system looks like.

IP addresses and domain names are all known publicly. Your domain name appears in the certs which are logged in public transparency logs. IP addresses are in the public DNS system.

If you show the actual outputs of those commands I will try further to help you learn about your system. I know from experience on this forum that helping an unskilled person unwilling to share details often leads to very long debugging sessions which are not fun for anyone.

4 Likes
13

sudo ss -pant | grep -i listen | grep -Ei ':80|:443'
LISTEN 0 511 *:80 : users:(("apache2",pid=3301,fd=4),("apache2",pid=3299,fd=4),("apache2",pid=2871,fd=4),("apache2",pid=2761,fd=4),("apache2",pid=2756,fd=4),("apache2",pid=2755,fd=4),("apache2",pid=2752,fd=4),("apache2",pid=2748,fd=4),("apache2",pid=2746,fd=4),("apache2",pid=2709,fd=4),("apache2",pid=1399,fd=4))
LISTEN 0 511 *:443 : users:(("apache2",pid=3301,fd=6),("apache2",pid=3299,fd=6),("apache2",pid=2871,fd=6),("apache2",pid=2761,fd=6),("apache2",pid=2756,fd=6),("apache2",pid=2755,fd=6),("apache2",pid=2752,fd=6),("apache2",pid=2748,fd=6),("apache2",pid=2746,fd=6),("apache2",pid=2709,fd=6),("apache2",pid=1399,fd=6))
~$ sudo apache2ctl -t -D DUMP_VHOSTS

VirtualHost configuration:
*:443                  is a NameVirtualHost
         default server links.rclservicesgroup.com (/etc/apache2/sites-enabled/links.rclservicesgroup.com-le-ssl.conf:2)
         port 443 namevhost links.rclservicesgroup.com (/etc/apache2/sites-enabled/links.rclservicesgroup.com-le-ssl.conf:2)
                 alias links.rclservicesgroup.com
         port 443 namevhost links.rclsg.com (/etc/apache2/sites-enabled/links.rclsg.com-le-ssl.conf:2)
                 alias links.rclsg.com
         port 443 namevhost www.rclsg.com (/etc/apache2/sites-enabled/www.rclsg.com-le-ssl.conf:2)
                 alias www.rclsg.com
         port 443 namevhost www.rcltrackservices.com (/etc/apache2/sites-enabled/www.rcltrackservices.com-le-ssl.conf:2)
                 alias www.rcltrackservices.com
         port 443 namevhost www.rclts.com (/etc/apache2/sites-enabled/www.rclts.com-le-ssl.conf:2)
                 alias www.rclts.com
         port 443 namevhost www.rclwiring.com (/etc/apache2/sites-enabled/www.rclwiring.com-le-ssl.conf:2)
                 alias www.rclwiring.com
*:80                   is a NameVirtualHost
         default server links.rclservicesgroup.com (/etc/apache2/sites-enabled/000-default.conf:1)
         port 80 namevhost links.rclservicesgroup.com (/etc/apache2/sites-enabled/000-default.conf:1)
         port 80 namevhost links.rclservicesgroup.com (/etc/apache2/sites-enabled/links.rclservicesgroup.com.conf:1)
                 alias links.rclservicesgroup.com
         port 80 namevhost links.rclsg.com (/etc/apache2/sites-enabled/links.rclsg.com.conf:1)
                 alias links.rclsg.com
         port 80 namevhost www.rclservicesgroup.com (/etc/apache2/sites-enabled/www.rclservicesgroup.com.conf:1)
                 alias www.rclservicesgroup.com
         port 80 namevhost www.rclsg.com (/etc/apache2/sites-enabled/www.rclsg.com.conf:1)
                 alias www.rclsg.com
         port 80 namevhost www.rcltrackservices.com (/etc/apache2/sites-enabled/www.rcltrackservices.com.conf:1)
                 alias www.rcltrackservices.com
         port 80 namevhost www.rclts.com (/etc/apache2/sites-enabled/www.rclts.com.conf:1)
                 alias www.rclts.com
         port 80 namevhost www.rclwiring.com (/etc/apache2/sites-enabled/www.rclwiring.com.conf:1)
                 alias www.rclwiring.com

~$ curl -4 https://ifconfig.io
209.214.204.228

14

Upon further investigation the certificate failed to renew 8/19, same day the migration to Cloudflare was completed.

15

I don't see app.rclwiring.com in that VirtualHost list for Apache

And, the DNS for app is now 209.214.204.234
That is not the same as you show for

curl -4 https://ifconfig.io
209.214.204.228

There is probably another Apache / Ubuntu system. Do you know where the server handling the IP address ending in .234 is?

3 Likes
16

That is the only linux machine in my environment. .234 is NAT pointing to this linux box.

17

Okay, I was expecting to see a VirtualHost for your app subdomain. Without one your Apache will process the incoming requests for that domain name using the default server which is for links.rclservicesgroup.com

And, we can confirm that is what is happening by looking at the cert used for HTTPS requests to your app subdomain. It is an expired cert for that links domain

echo | openssl s_client -connect app.rclwiring.com:443 | head
Certificate chain
 0 s:CN = links.rclservicesgroup.com
   i:C = US, O = Let's Encrypt, CN = R3
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Feb 19 20:02:02 2024 GMT; NotAfter: May 19 20:02:01 2024 GMT

This is probably related to why the cert renewal is failing. (the missing VirtualHost for app)

Just based on the file names in your DUMP_VHOSTS it looks like this system is using Certbot to get certs. That is a common ACME Client.

Can you show output of this (fingers crossed)

sudo certbot certificates
3 Likes
18

Looks like your hunch is correct. How do I correct it?

Found the following certs:
Certificate Name: links.rclservicesgroup.com
Domains: links.rclservicesgroup.com
Expiry Date: 2024-05-19 20:02:01+00:00 (INVALID: EXPIRED)
Certificate Path: /etc/letsencrypt/live/links.rclservicesgroup.com/fullchain.pem
Private Key Path: /etc/letsencrypt/live/links.rclservicesgroup.com/privkey.pem
Certificate Name: links.rclsg.com
Domains: links.rclsg.com
Expiry Date: 2024-11-18 01:24:41+00:00 (VALID: 52 days)
Certificate Path: /etc/letsencrypt/live/links.rclsg.com/fullchain.pem
Private Key Path: /etc/letsencrypt/live/links.rclsg.com/privkey.pem
Certificate Name: www.rclsg.com
Domains: www.rclsg.com
Expiry Date: 2024-12-14 09:59:37+00:00 (VALID: 78 days)
Certificate Path: /etc/letsencrypt/live/www.rclsg.com/fullchain.pem
Private Key Path: /etc/letsencrypt/live/www.rclsg.com/privkey.pem
Certificate Name: www.rcltrackservices.com
Domains: www.rcltrackservices.com
Expiry Date: 2024-12-11 09:23:45+00:00 (VALID: 75 days)
Certificate Path: /etc/letsencrypt/live/www.rcltrackservices.com/fullchain.pem
Private Key Path: /etc/letsencrypt/live/www.rcltrackservices.com/privkey.pem
Certificate Name: www.rclts.com
Domains: www.rclts.com
Expiry Date: 2024-12-14 23:08:23+00:00 (VALID: 79 days)
Certificate Path: /etc/letsencrypt/live/www.rclts.com/fullchain.pem
Private Key Path: /etc/letsencrypt/live/www.rclts.com/privkey.pem
Certificate Name: www.rclwiring.com
Domains: www.rclwiring.com
Expiry Date: 2024-11-22 11:58:12+00:00 (VALID: 56 days)
Certificate Path: /etc/letsencrypt/live/www.rclwiring.com/fullchain.pem
Private Key Path: /etc/letsencrypt/live/www.rclwiring.com/privkey.pem

19

Hmm. While Certbot has certs for those domains it does not for your app subdomain. So, some other system must have been getting its certs. And, I can see in the history that domain got LE certs for a very long time.

So, fixing this depends on how you want your app domain to work. Should it be handled by that Apache? If so, what happened to its VirtualHost definitions? We can make new ones but something significant has changed and not just the DNS provider.

If app is supposed to be handled elsewhere then you need to change your network routing to requests get to the right place.

Can you provide more background on how you got to this state?

3 Likes
20

I was handed this system a month ago, transferred all sites of formers personal registrar to a company owned one. I will reach out to the former and ask some clarifying questions with the information you have provided.

This domain is non-essential but it seems there are some fundamental issues with what's currently implemented. As far as I am aware there have been no changes reported for this since 2022.

Quick run down what I have learned - (feel free to correct)
app domain is pointed to IIS for files but its root is in the lamp machine
app domain cannot renew due to it not being listed in the vhost list and defaulting to an expired domain.