Unable to Verify Domain with Certbot Despite Correct Webroot Configuration

Help
1

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: www.impossiblecreatures.com

I ran this command: sudo certbot certonly --webroot -w /var/www/certbot -d impossiblecreatures.com -d www.impossiblecreatures.com --debug-challenges

It produced this output: Saving debug log to /var/log/letsencrypt/letsencrypt.log
Requesting a certificate for impossiblecreatures.com and www.impossiblecreatures.com

Challenges loaded. Press continue to submit to CA.
Pass "-v" for more info about challenges.

Press Enter to Continue

Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
Domain: impossiblecreatures.com
Type: unauthorized
Detail: 2a02:4780:b:1106:0:96b:6224:2: Invalid response from http://impossiblecreatures.com/.well-known/acme-challenge/tMGHExVlJUVhaiBq0i-nUD5YfGYPqej51Jxmbsl5HCU: 404

Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded from the internet.

Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.
root@srv551730:~#

My web server is (include version): Node.js v16.20.2

The operating system my web server runs on is (include version): Ubuntu 22.04 with CloudPanel VPS

My hosting provider, if applicable, is: Hostinger

I can login to a root shell on my machine (yes or no, or I don't know): Yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): CloudPanel

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): certbot 2.11.0

Hello, I've encountered an issue while trying to issue a Let's Encrypt certificate using Certbot's webroot plugin on an Ubuntu server managed via CloudPanel. I have ensured the webroot directory is accessible over the internet, and a test file in the .well-known/acme-challenge/ directory can be accessed without any issue.

Problem:

When running the command:

sudo certbot certonly --webroot -w /var/www/certbot -d impossiblecreatures.com -d www.impossiblecreatures.com --debug-challenges

Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
Domain: impossiblecreatures.com
Type: unauthorized
Detail: Invalid response from http://impossiblecreatures.com/.well-known/acme-challenge/: 404

The complete response:

Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
Domain: impossiblecreatures.com
Type: unauthorized
Detail: 2a02:4780:b:1106:0:96b:6224:2: Invalid response from http://impossiblecreatures.com/.well-known/acme-challenge/tMGHExVlJUVhaiBq0i-nUD5YfGYPqej51Jxmbsl5HCU: 404

Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded from the internet.

Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.
root@srv551730:~#

I've also Placed a test.txt file in /var/www/certbot/.well-known/acme-challenge/ which is accessible via http://impossiblecreatures.com/.well-known/acme-challenge/test.txt

Attempted to issue a certificate using the webroot method also.

Configuration:

Webroot Path: /var/www/certbot
Nginx Configuration for Webroot:

location ^~ /.well-known/acme-challenge/ {
root /var/www/certbot;
allow all;
}

Additional Context:

I am able to see the challenge file created by Certbot in the webroot directory before it is cleaned up post-challenge failure, ie in debug mode, i also was able to point to this in the chrome browser and download it, sugggesting there's no issues seeing and accessing it, ie permissions, DNS issues etc

The domain is pointing correctly to the server, and there are no visible issues with DNS configurations.

The Nginx server blocks for HTTP and HTTPS are configured correctly to redirect and handle SSL requests.

What I Need Help With:

Understanding why Let's Encrypt cannot access the challenge file.

Any additional configurations or steps I might have missed that could resolve this issue.

Various paths:

sftp://root@157.173.210.7/var/log/letsencrypt/letsencrypt.log

sftp://root@157.173.210.7/var/www/certbot/.well-known/acme-challenge/test.txt

sftp://root@157.173.210.7/home/impossiblecreatures/htdocs/impossiblecreatures.com/index.html

sftp://root@157.173.210.7/etc/nginx/sites-available/impossiblecreatures.com.conf

sftp://root@157.173.210.7/etc/nginx/nginx.conf

See the followingfiles which includes the NGINX.conf and impossiblecreatures.conf from available sites, logs and so forth among other information that might be useful to see.

https://drive.google.com/drive/folders/1xasLxFKDNIZDAdVFwH1GrQnJo3VjGIVl?usp=sharing

This could be complete user error on my part, I feel i have a sound understanding of the concepts, but maybe im overlooking something. still learning! :slight_smile:

Thank you kindly!

David

2

Hello @everlite1knight, welcome to the Let's Encrypt community. :slightly_smiling_face:

Using the online tool Let's Debug yields these results https://letsdebug.net/impossiblecreatures.com/2077121

MultipleIPAddressDiscrepancy
WARNING
impossiblecreatures.com has multiple IP addresses in its DNS records. While they appear to be accessible on the network, we have detected that they produce differing results when sent an ACME HTTP validation request. This may indicate that some of the IP addresses may unintentionally point to different servers, which would cause validation to fail.
[Address=2a02:4780:b:1106:0:96b:6224:2,Address Type=IPv6,Server=LiteSpeed,HTTP Status=404] vs [Address=157.173.210.7,Address Type=IPv4,Server=nginx,HTTP Status=404]

Also for DNS the is only an IPv4 Address for www.impossiblecreatures.com yet impossiblecreatures.com has both an IPv4 Address and an IPv6 Address

image
image1043×808 10.6 KB

All IP Address for a give domain name need to respond the same.

2 Likes
3

Hi @everlite1knight,

Probably Hostinger automatically assigning an IPv6 Address (often undesirably);
likely you want to remove the IPv6 Address by deleting the DNS AAAA Records.

IPv4 show Server: nginx

>curl -4 -Ii http://impossiblecreatures.com/.well-known/acme-challenge/sometestfile
HTTP/1.1 404 Not Found
Server: nginx
Date: Mon, 01 Jul 2024 23:49:11 GMT
Content-Type: text/html
Content-Length: 146
Connection: keep-alive
Vary: Accept-Encoding

IPv6 shows server: LiteSpeed platform: hostinger

>curl -6 -Ii http://impossiblecreatures.com/.well-known/acme-challenge/sometestfile
HTTP/1.1 404 Not Found
Connection: Keep-Alive
Keep-Alive: timeout=5, max=100
content-type: text/html
content-length: 150
date: Mon, 01 Jul 2024 23:49:17 GMT
server: LiteSpeed
platform: hostinger
content-security-policy: upgrade-insecure-requests
3 Likes
4

Thanks for the great reply! Strange i think LiteSpeed might have been installed as their template. Strangely i dont see LiteSpeed in the processes or even a directory for it .. I was going to try and remove it .. but as yet it appears to not exist! ... will continue digging. Thanks again, sincerely i didn't expect such a great response! :slight_smile:

3 Likes
5

Check the DNS settings. We see these extra IPv6 addresses often with Hostinger

2 Likes
6

Thanks, that seems to be issue. I deleted the following from the DNS list:

AAAA @ 0 2a02:4780:b:1106:0:96b:6224:2 1800

And now its successful! Thank you! :slight_smile: I am curious though what was happening with the LiteSpeed, how this wasn't present in the OS.

3 Likes
7

I believe the IPv6 address pointed to a LiteSpeed server operated by Hostinger. Sort of like a parking page. They may even setup a parking site for both IPv4 and IPv6 and then only later modify the IPv4 once you setup your own.

I am not sure of the details but something along those lines. That's a good question to ask of them :slight_smile:

2 Likes
8

oh i tried! lol it was met with a lot of resistance and links to their help page and finally saying; "we're only here for hosting issues and we have limited knowledge with the VPS servers."

2 Likes
9

Ah i thought the certificates would remove these messages!
image

Any suggestions? :frowning:

1 Like
10

Maybe restart your browser. Your site and certs look fine. And, you redirect HTTP to HTTPS as you should.

Sometimes browsers cache old lookups

1 Like
11

You probably should also add a --deploy-hook to your Certbot renewal profile.

nginx needs at least a reload to see a new cert. Whatever command you use to reload nginx just add in place of (command-here)

sudo certbot reconfigure --cert-name impossiblecreatures.com --deploy-hook '(command-here)'

The deploy-hook runs whenever you get a fresh cert. Very useful for auto-renew so nginx gets reloaded each cert renew.

If you reload or restart nginx frequently (like daily) you could skip this and just remember there will be delay in starting to use a new cert.

1 Like
12

Ah great tip! Thanks for that, will do that now! :slight_smile:

2 Likes
13

Are you still seeing "insecure" notices?

1 Like
14

No, it has actually gone away now. Perhaps it was a cache thing maybe. Thank you! :slight_smile:

1 Like
15

oh thats strange, correction, i am seeing it again. But i wasnt for a while. ummm .. will dig into that a little more.

16

Is it failing only certain pages? Perhaps you have some links using http and not https. Or even using a specific IP address.

You should click around on the Error screen and see what certificate is being used. And see if it gives further explanation for exactly what it thinks is insecure. You may have to search on a browser forum for a guide to find the details

2 Likes
17

from what i can see all the pages are using https, i dont see anything as http. I assume you mean internal links within the website map? and not links that jump to external third party websites? ie links in a page.

18

If you open browser tools while browsing your site you'll see your site is using localhost:5000 for some of the content (images) so those won't load for other people and they will give you various browser errors.

In general check your network tab on the browser when doing a refresh to see which URLs you are trying to access.

3 Likes
19

Hey, thanks for pointing that out! completely slipped by me, will replace those and see if it address the issue.

Thanks again.

3 Likes
20

I think the security warnings are gone now. Perhaps ensuring the production environment / variables were been used corrected this issue! fingers crossed i dont see it again :slight_smile:

1 Like