DNS' problem: SERVFAIL looking up A

Help
1

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. https://crt.sh/?q=example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: da-demo.xyz

I tried installing and renewing SSL Certificates from my DirectAdmin panel.

It produced this output:

CANNOT EXECUTE YOUR REQUEST
        Requesting new certificate order...
        Processing authorization for ftp.da-demo.xyz...
        Waiting for domain verification...
        Challenge status: invalid. Challenge error: "type": "http-01",  "status": "invalid",  "error": {    "type": "urn:ietf:params:acme:error:dns",    "detail": "DNS problem: SERVFAIL looking up A for ftp.da-demo.xyz",    "status": 400  . Exiting...

Screenshot is https://i.imgur.com/HNBfI9c.png

My web server is (include version): Apache (2.4) + Nginx (1.17.7) as a reverse proxy of Apache

The operating system my web server runs on is (include version): CloudLinux release 7.7

I can log in to a root shell on my machine (yes or no, or I don’t know): Yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel): DirectAdmin Version 1.595

1 Like
2

Is da-demo.xyz really your domain name? You registered and own it?

8.9.37.209 is your IP address?

The DNS looks okay to me – aside from the fact that there’s only one server. Maybe there’s a routing issue between you and some of Let’s Encrypt’s validation servers? Or your DNS server has some kind of rate limiting?

1 Like
3

Hi @mnordhoff,

Yes, I own da-demo.xyz. And the IP is also under my control.

Indeed, DNS looks pretty Okay.

I have temporarily disabled ConfigServer Security & Firewall (CSF) in my server.

I don’t have any Rate Limits set in the DNS server. My primary domain (the one resolves the Nameservers) is using Cloudflare. I put Cloudflare as DNS only as well, but still the same issue.

https://letsdebug.net/da-demo.xyz/89469 indicates the same error. But all other DNS checking tools says no faults.

1 Like
4

This might not mean anything, but from my ISP, I cannot talk to your server at all. It doesn’t seem to find a route out of my ISP at all.

$ dig @8.9.37.209 da-demo.xyz

; <<>> DiG 9.11.5-P4-5.1ubuntu2.1-Ubuntu <<>> @8.9.37.209 da-demo.xyz
; (1 server found)
;; global options: +cmd
;; connection timed out; no servers could be reached

and

$ mtr --report -n -c 10 8.9.37.209
Start: 2019-12-31T17:12:35+1100
HOST: x1                          Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- 192.168.1.1                0.0%    10    2.2   1.7   1.3   2.9   0.5
  2.|-- 10.20.21.123               0.0%    10   12.5  14.1  12.4  17.8   2.0
  3.|-- 203.219.155.2              0.0%    10   22.9  24.1  22.5  25.5   1.0
  4.|-- 202.7.171.153              0.0%    10   24.1  27.2  24.1  30.3   2.4
  5.|-- 203.29.134.125             0.0%    10   22.4  22.9  22.2  24.1   0.6
  6.|-- ???                       100.0    10    0.0   0.0   0.0   0.0   0.0

So I would agree that the problem has something to do with networking.

Is this a Vultr server in New York? I would be interested to know if you were able to add a second IP address from an entirely different /8 and point your domains at it, whether it avoids the issue?

1 Like
5

Yes, it is, @_az.

But Vultr support says that this is not a DNS issue. It is a permission issue on the root of the sites.

DirectAdmin support said:

Resolver DNS works from your server just fine. The strange part is that server is actually getting the request and the request is sent back, but the LE server is not seeing that.

1 Like
6

Let's Encrypt sends simultaneous requests from 4 different datacentres.

As of today, in the live API, only 1 one of these results affects the outcome of the validation, and the other 3 results are informative only.

When you see that the response is being sent to Let's Encrypt, it can mislead you into believing that the request has succeeded. It may actually be the case that the one important datacentre failed while one or more of the others succeeded.

I would still be interested in seeing whether binding an IP from another range and using that IP for your domain and nameserver has any effect.

1 Like
7

The error message disagrees.

1 Like
8

Hi @_az,

I will definitely connect a new IP and check as you suggested. I am waiting for a confirmation from DirectAdmin support team.

I have received another response from DirectAdmin team.

well, requests to challenges(these should be served as data files, no php interpretation etc. etc. which could raise response time) definitely are quite snappy and takes nowhere near 30s, unless there's issues with connectivity from somewhere reaching those and there's packet loss it shouldn't be an issue.

Also there's requests logged by webserver, these should be logged upon response is sent:
3.14.255.131 - - [31/Dec/2019:15:24:52 +0530] "GET /.well-known/acme-challenge/0RSxW5R_UIsx_05sWNxO5ny2kVJqowbm1vGYp8ctpMc HTTP/1.0" 200 428 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
34.209.232.166 - - [31/Dec/2019:15:24:52 +0530] "GET /.well-known/acme-challenge/0RSxW5R_UIsx_05sWNxO5ny2kVJqowbm1vGYp8ctpMc HTTP/1.0" 200 428 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
18.194.58.132 - - [31/Dec/2019:15:24:52 +0530] "GET /.well-known/acme-challenge/0RSxW5R_UIsx_05sWNxO5ny2kVJqowbm1vGYp8ctpMc HTTP/1.0" 200 428 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
3.14.255.131 - - [31/Dec/2019:15:24:54 +0530] "GET /.well-known/acme-challenge/0RSxW5R_UIsx_05sWNxO5ny2kVJqowbm1vGYp8ctpMc HTTP/1.0" 200 428 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
34.222.229.130 - - [31/Dec/2019:15:24:54 +0530] "GET /.well-known/acme-challenge/0RSxW5R_UIsx_05sWNxO5ny2kVJqowbm1vGYp8ctpMc HTTP/1.0" 200 428 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
52.28.236.88 - - [31/Dec/2019:15:24:54 +0530] "GET /.well-known/acme-challenge/0RSxW5R_UIsx_05sWNxO5ny2kVJqowbm1vGYp8ctpMc HTTP/1.0" 200 428 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
3.14.255.131 - - [31/Dec/2019:15:24:56 +0530] "GET /.well-known/acme-challenge/0RSxW5R_UIsx_05sWNxO5ny2kVJqowbm1vGYp8ctpMc HTTP/1.0" 200 428 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
34.209.232.166 - - [31/Dec/2019:15:24:56 +0530] "GET /.well-known/acme-challenge/0RSxW5R_UIsx_05sWNxO5ny2kVJqowbm1vGYp8ctpMc HTTP/1.0" 200 428 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
52.28.236.88 - - [31/Dec/2019:15:24:56 +0530] "GET /.well-known/acme-challenge/0RSxW5R_UIsx_05sWNxO5ny2kVJqowbm1vGYp8ctpMc HTTP/1.0" 200 428 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
3.14.255.131 - - [31/Dec/2019:15:24:58 +0530] "GET /.well-known/acme-challenge/0RSxW5R_UIsx_05sWNxO5ny2kVJqowbm1vGYp8ctpMc HTTP/1.0" 200 428 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
34.209.232.166 - - [31/Dec/2019:15:24:58 +0530] "GET /.well-known/acme-challenge/0RSxW5R_UIsx_05sWNxO5ny2kVJqowbm1vGYp8ctpMc HTTP/1.0" 200 428 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
18.194.58.132 - - [31/Dec/2019:15:24:59 +0530] "GET /.well-known/acme-challenge/0RSxW5R_UIsx_05sWNxO5ny2kVJqowbm1vGYp8ctpMc HTTP/1.0" 200 428 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
3.14.255.131 - - [31/Dec/2019:15:25:00 +0530] "GET /.well-known/acme-challenge/0RSxW5R_UIsx_05sWNxO5ny2kVJqowbm1vGYp8ctpMc HTTP/1.0" 200 428 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
34.209.232.166 - - [31/Dec/2019:15:25:00 +0530] "GET /.well-known/acme-challenge/0RSxW5R_UIsx_05sWNxO5ny2kVJqowbm1vGYp8ctpMc HTTP/1.0" 200 428 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
52.28.236.88 - - [31/Dec/2019:15:25:01 +0530] "GET /.well-known/acme-challenge/0RSxW5R_UIsx_05sWNxO5ny2kVJqowbm1vGYp8ctpMc HTTP/1.0" 200 428 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
3.14.255.131 - - [31/Dec/2019:15:25:03 +0530] "GET /.well-known/acme-challenge/0RSxW5R_UIsx_05sWNxO5ny2kVJqowbm1vGYp8ctpMc HTTP/1.0" 200 428 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
34.222.229.130 - - [31/Dec/2019:15:25:03 +0530] "GET /.well-known/acme-challenge/0RSxW5R_UIsx_05sWNxO5ny2kVJqowbm1vGYp8ctpMc HTTP/1.0" 200 428 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
52.28.236.88 - - [31/Dec/2019:15:25:03 +0530] "GET /.well-known/acme-challenge/0RSxW5R_UIsx_05sWNxO5ny2kVJqowbm1vGYp8ctpMc HTTP/1.0" 200 428 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
3.14.255.131 - - [31/Dec/2019:15:25:09 +0530] "GET /.well-known/acme-challenge/0RSxW5R_UIsx_05sWNxO5ny2kVJqowbm1vGYp8ctpMc HTTP/1.0" 200 428 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
34.209.232.166 - - [31/Dec/2019:15:25:09 +0530] "GET /.well-known/acme-challenge/0RSxW5R_UIsx_05sWNxO5ny2kVJqowbm1vGYp8ctpMc HTTP/1.0" 200 428 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
52.28.236.88 - - [31/Dec/2019:15:25:09 +0530] "GET /.well-known/acme-challenge/0RSxW5R_UIsx_05sWNxO5ny2kVJqowbm1vGYp8ctpMc HTTP/1.0" 200 428 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"

It seems fishy that requests are being repeated as if response not received.

Do you mind for us to install third party letsencrypt client like certbot on server, just to fully rule out issues with data formed be da letsencrypt client? Though up to know there has been no reports on this and currently it looks like something happening to traffic which leaves server.

I will keep this thread updated with their response. In the meantime, could you confirm if this log helps to identify something?

1 Like
9

Those requests are all from the non-primary validation servers.

From Let’s Encrypt’s perspective, they all succeeded, but they do not affect the result of the overall validation.

The two primary validation servers have other IP addresses.

1 Like
10

I got this resolved.

Believe it or not, the solution was permanently to remove my primary domain from Cloudflare.

I too wonder, obviously we don’t suspect a big dude like Cloudflare.

Let me explain a bit.

I was using Cloudflare for the primary domain. This domain used to be the authoritative Nameserver to the box.

Since more than a week now, it takes more than 30 seconds to start rendering the page, especially when I force reload or open from Incognito window.

I thought that would be my new ISPs issue. And when I Pause Cloudflare and use it as a DNS Only mode, the speed becomes normal.

Trust me, I toggled this Pause Cloudflare and Resume Cloudflare multiple times with decent interval between to double-check this.

Below is a hypothetical scenario or a coincidence.

At https://www.whatsmydns.net/#A/any-hosted-domain most of them are green, except the servers from the USA.

But the primary domain’s NS were all green. All the domains hosted on this box returned the same error in this test from the USA test networks.

But now, after I change the NS of my primary domain, https://www.whatsmydns.net/#A/any-hosted-domain shows green to all the USA test systems along with other regions.

Let’s Encrypts servers or main validation, and issuance servers are/may located in the USA.

Thus this failure. Who knows. After a week or so, I will try to reconnect with Cloudflare if I get another nameserver.

For the record, my present NS from Cloudflare is cory.ns.cloudflare.com and emily.ns.cloudflare.com.

1 Like
11

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.