Challenge failed for domain hotrobotinc.com http-01

[hotrobot]

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is:
hotrobotinc.com

I ran this command:
certbot --nginx --agree-tos --redirect --hsts --staple-ocsp --email myemail@gmail.com -d hotrobotinc.com -v

It produced this output:
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator nginx, Installer nginx
Requesting a certificate for hotrobotinc.com
Performing the following challenges:
http-01 challenge for hotrobotinc.com
Waiting for verification...
Challenge failed for domain hotrobotinc.com
http-01 challenge for hotrobotinc.com

Certbot failed to authenticate some domains (authenticator: nginx). The Certificate Authority reported these problems:
Domain: hotrobotinc.com
Type: unauthorized
Detail: 15.197.142.173: Invalid response from http://hotrobotinc.com/.well-known/acme-challenge/HD3WOwVOyOaZe8qOsKO9ZOeGtanHPEom483rNKacZ8M: 404

Hint: The Certificate Authority failed to verify the temporary nginx configuration changes made by Certbot. Ensure the listed domains point to this nginx server and that it is accessible from the internet.

Cleaning up challenges
Some challenges have failed.

My web server is (include version):
nginx/1.14.1

The operating system my web server runs on is (include version):
CentOS Stream 8

My hosting provider, if applicable, is:
Kamatera

I can login to a root shell on my machine (yes or no, or I don't know):
Yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):
No

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):
certbot 2.6.0

[rg305]

Hi @hotrobot, and welcome to the LE community forum

Well... It's not easy, and quite uncommon, to see "--nginx" fail with "404".
So, you are either a master at that OR something else it at hand:
[I'm betting on the latter]

curl -Ii http://hotrobotinc.com/.well-known/acme-challenge/Test_File-1234
HTTP/1.1 405 Not Allowed
Server: awselb/2.0 <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< ? ? ?
Date: Wed, 02 Aug 2023 21:24:26 GMT
Content-Length: 0
Connection: keep-alive
WAFRule: 0

[hotrobot]

Hi, thanks for having us. It is really bizarre. I have looked for hours trying to figure it out, and can't seem to get my fingers on the root cause. I was thinking it is a permissions issue because it seems like that file is not getting created. nginx is the owner of the server root folder, though, so I don't know how that could be the case. I've verified I can access the url from my personal computer, I have also verified with nmap as well as ncap and got no information there, other than ncat is unable to find a crt.

[MikeMcQ]

Your DNS is not pointing directly to your nginx server.

Instead it is pointing to an Amazon AWS service (ELB). We sometimes see this for hosting services that use URL Redirect or URL Forwarding options in the DNS config.

Do you have anything like that setup? If so, change your DNS config to point to your nginx public IP

[hotrobot]

But, if DNS wasn't setup properly, wouldn't attempting to navigate to hotrobotinc.com on my personal computer not hit the server?

[MikeMcQ]

Something might work on your local network. And, those redirect services can work with HTTP. But, they won't work with HTTPS.

But, your DNS is not pointing directly to your nginx.

Here is the response using the ACME HTTP Challenge. Note the 'Server' is an AWS EC2 instance.

 curl -iL hotrobotinc.com/.well-known/acme-challenge/Test123
HTTP/1.1 404 Not Found
Date: Wed, 02 Aug 2023 22:28:42 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 125
Connection: keep-alive
Server: ip-10-123-122-81.ec2.internal
X-Request-Id: 1b096943-a9bf-4b0a-995a-523064da4b37

Here is a request to your "home" page. It sends directly to an IP address which won't work with Let's Encrypt certs because those names must appear in the cert itself.

curl -i hotrobotinc.com
HTTP/1.1 301 Moved Permanently
Date: Wed, 02 Aug 2023 22:26:26 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 56
Connection: keep-alive
Location: http://103.101.202.71
Server: ip-10-123-123-234.ec2.internal
X-Request-Id: 49095563-f256-4988-add3-571a57249709

<a href="http://103.101.202.71">Moved Permanently</a>.

[hotrobot]

Oh, ok. so the domain we have with godaddy is redirected to the server, which is a no go. I will look into pointing that domain name directly at our server.

[hotrobot]

Thank you so much, we're golden now.
So what had happened waa:
When I went to update the A record on godaddy it was greyed out and said unable to edit, so I thought I had to use forwarding and set it up as such, which obviously didn't work. But, following your comment, I deleted forwarding, ADDED the correct A record, which just overwrites the one that is there. Not a very intuitive user interface for such a huge company. Anyways, hopefully this saves someone some headache in the future. Thanks again for your help!!!