Certbot resolves wrong IP

Help
1

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: spinsjoy-lp.com

I ran this command: certbot certonly --nginx -d spinsjoy-lp.com,www.spinsjoy-lp.com --dry-run

It produced this output:
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator nginx, Installer nginx
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for spinsjoy-lp.com
http-01 challenge for www.spinsjoy-lp.com
Waiting for verification...
Cleaning up challenges
Failed authorization procedure. www.spinsjoy-lp.com (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://www.spinsjoy-lp.com/.well-known/acme-challenge/u7dYFihzZSgjFF7wXF7xcIVdUSrsN3LEaNkG4AEg9Fo [107.154.249.223]: "<html style="height:100%"><META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW"><meta name="format-detection" content="telephone", spinsjoy-lp.com (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://spinsjoy-lp.com/.well-known/acme-challenge/IcPvV5auOXjjU2tYgEHf3qCpLRDzI6BPGyVJvgV-efM [107.154.212.223]: "<html style="height:100%"><META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW"><meta name="format-detection" content="telephone"

IMPORTANT NOTES:

My web server is (include version): nginx/1.18.0

The operating system my web server runs on is (include version): Debian Buster

My hosting provider, if applicable, is: GCP

I can login to a root shell on my machine (yes or no, or I don't know): yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): certbot 0.31.0

Very strange because I found here that some of people had that issue already. For example here
But it all seems goes to one point. Wrong DNS entry. Which in my case isn't the problem, my DNS is showing correct IP's

dig +short A spinsjoy-lp.com @pdns05.domaincontrol.com. ; dig +short A spinsjoy-lp.com @pdns06.domaincontrol.com.
107.154.212.223
107.154.248.223
107.154.212.223
107.154.248.223

I don't know how he get that 107.154.249.223 which is the issue here. From link above from other thread there is information that it uses our DNS from server where we run that command. But it seems that's not true.

2

Hi @eset

that's your DNS configuration, see your check, created yesterday - https://check-your-website.server-daten.de/?q=spinsjoy-lp.com

Host Type IP-Address is auth. ∑ Queries ∑ Timeout
spinsjoy-lp.com A 107.154.212.223 London/England/United Kingdom (GB) - Incapsula Inc Hostname: 107.154.212.223.ip.incapdns.net yes 1 0
A 107.154.248.223 London/England/United Kingdom (GB) - Incapsula Inc Hostname: 107.154.248.223.ip.incapdns.net yes 1 0
AAAA

That's the ip of your error message:

Looks like you have a wrong ip address defined.

3

The www subdomain is a CNAME for qcxnnga.x.incapdns.net.

4

Yes, but that's the same ip 107.154.248.223.

5

Nope, it resolves to 107.154.249.223:

Although the apex domain resolves to the correct 107.154.212.223 IP address and also returns HTML in stead of the token, so the DNS issue isn't the only issue here. It's probably also a nginx configuration issue.

6

Ah, the check is one day old, didn't rechecked the domain.

7

Check what DNS gives you

And I paste it here
dig +short A spinsjoy-lp.com @pdns05.domaincontrol.com. ; dig +short A spinsjoy-lp.com @pdns06.domaincontrol.com.
107.154.212.223
107.154.248.223
107.154.212.223
107.154.248.223
So we stick to those IP addresses because they are valid
107.154.212.223
107.154.248.223

The issue here is 107.154.249.223 which isn't the valid one. The third octet makes difference here 249 instead of 248 and that's the issue.

9

Rechecked your domain:

Host Type IP-Address is auth. ∑ Queries ∑ Timeout
spinsjoy-lp.com A 107.154.212.223 London/England/United Kingdom (GB) - Incapsula Inc Hostname: 107.154.212.223.ip.incapdns.net yes 1 0
A 107.154.248.223 London/England/United Kingdom (GB) - Incapsula Inc Hostname: 107.154.248.223.ip.incapdns.net yes 1 0
AAAA yes
www.spinsjoy-lp.com CNAME qcxnnga.x.incapdns.net yes 1 0
A 107.154.248.223 London/England/United Kingdom (GB) - Incapsula Inc Hostname: 107.154.248.223.ip.incapdns.net yes

That's your current configuration.

But checking

http://spinsjoy-lp.com/.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de

there is a http status 200, not the expected http status 404 - Not Found.

Looks like your application doesn't handle files correct.

What says

nginx -T
10

Sorry I don't understand what my nginx has to do with it. Osiris has a good point that cname resolves somehow to wrong IP which certbot have my in the error output. 107.154.249.223 != 107.154.248.223

regarging your URL status to that address.. ehm you said there is 200 instead of 404? Strange.. is obvious 404

<html>
<head><script src="/f-nowne-Macbeth-a-Daggersonry-selfe-to-the-labou" async></script><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx</center>
<script async type="text/javascript" src="/_Incapsula_Resource?SWJIYLWA=719d34d31c8e3a6e6fffd425f7e032f3&ns=1&cb=82697774"></script>
</body>
</html>```
11

Looks like you don't know your own system.

Downloading

http://www.spinsjoy-lp.com/.well-known/acme-challenge/1234

there is something with an iframe and a JavaScript.

<html style="height:100%"><head><META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">
<meta name="format-detection" content="telephone=no"><meta name="viewport" content="initial-scale=1.0"><meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
<script type="text/javascript" src="/_Incapsula_Resource?SWJIYLWA=719d34d31c8e3a6e6fffd425f7e032f3"></script></head>

<body style="margin:0px;height:100%">
<iframe id="main-iframe" src="/_Incapsula_Resource?CWUDNSAI=20&xinfo=13-283075958-0%20NNNN%20RT%281617091540176%200%29%20q%280%20-1%20-1%200%29%20r%280%20-1%29%20B10%2814%2c0%2c0%29%20U18&incident_id=730000360308573455-955972240243558669&edet=10&cinfo=0e0000008e27&rpinfo=0" frameborder=0 width="100%" height="100%" marginheight="0px" marginwidth="0px">Request unsuccessful. Incapsula incident ID: 730000360308573455-955972240243558669</iframe>
</body></html>

The browser executes the JavaScript and redirects to a 404 page.

1 Like
12

Certbot shows you two errors: one for the www subdomain with an incorrect IP address but also an error for the apex domain with the correct IP address: so the CNAME is not the only issue here.

13

Also, what the @#+&@£& is this JavaScript file?

http://spinsjoy-lp.com/f-nowne-Macbeth-a-Daggersonry-selfe-to-the-labou

Is your server hacked or something?

14

It's behind CDN and it's not my system it's developer system I have in Nginx special location layer 7 routing to include .well-known

15

It's Imperva Bot Protection. Like I said it's not the issue here. I have a lot of sites behind that CDN and Certbot works. Like I said IP address is the issue here as this gives the wrong IP. I'm pretty sure it will work correctly if it will be fixed. And the rest of the configuration which you found that's part of CDN configuration and I have over 22 sites there and they work easily with certbot being behind the CDN provider. So no, my website isn't hacked, quite opposite , it's secured :slight_smile:

16

I'm 80 % certain it won't. We'll see :slight_smile:

17

Here is a prove that .well-known locationw works well

curl -Is  http://spinsjoy-lp.com/.well-known/acme-challenge/dupa.html
HTTP/1.1 200 OK
Server: nginx
Date: Tue, 30 Mar 2021 08:35:49 GMT
Content-Type: text/html
Content-Length: 5
Last-Modified: Tue, 29 Dec 2020 23:01:28 GMT
Connection: keep-alive
ETag: "5febb548-5"
Accept-Ranges: bytes
Set-Cookie: visid_incap_2420280=e3vg3ub+TFqbibx6nlcnE+XiYmAAAAAAQUIPAAAAAABrKgzbtdLx72rWj/vay0i6; expires=Tue, 29 Mar 2022 10:04:56 GMT; HttpOnly; path=/; Domain=.spinsjoy-lp.com
Set-Cookie: nlbi_2420280=PYRJflxvAjptyVifH+SF5QAAAADDvbN6SNY+cbhRlvseV+mU; path=/; Domain=.spinsjoy-lp.com
Set-Cookie: incap_ses_447_2420280=W/4pESbJCS2HAPGKfBA0BuXiYmAAAAAAY7t5PE7ptA3FUJ+DjNhunw==; path=/; Domain=.spinsjoy-lp.com
Set-Cookie: ___utmvmLEuXokwZ=kdfLDQGNuTD; path=/; Max-Age=900
Set-Cookie: ___utmvaLEuXokwZ=PCNvXXj; path=/; Max-Age=900
Set-Cookie: ___utmvbLEuXokwZ=WZv
    XHGORalz: rtX; path=/; Max-Age=900
X-CDN: Imperva
X-Iinfo: 5-64758536-64758537 NNNY CT(0 -1 0) RT(1617093349088 0) q(0 0 0 -1) r(0 0) U6

It fails because of the DNS and yeah one of them should work and second not but I never face output when only one check worked. Each time if it comes to CDN if there are multiple IP address if one of them fails , second also gaves error. And the nginx config is automated using ansible so there is no way that configuration difference much. What difference is the server_name output. Thats all.

18

I see, so the CDN decides it returns an error even if it could have succeded. That would be the 20 % margin I build into my prediction :grin:

19

does certbot servers have a IP addresses which I can whitelist? I Believe it's because Captcha on CDN

20

Certbot is one of many ACME clients which can be used to get certificates from any CA using the ACME protocol. So there are no "certbot servers". You probably mean the Let's Encrypt validation servers and for that the answer is: no. Please see the following FAQ question:

21

Yeah I found that the issue is not with nginx but with CDN. We received extra security layer and that cut off Lets Encrypt Validation Servers :confused: