Domain Control Validation failed for Valid & owned domains


#1

Domain is: support.cologlobal.com & support.novacarthosting.com

We have implemented our own Java Certificate V2 client and doing certificate operations through ACME v2 successfully for more than a year.

The following 2 domains were failing Domain Control Validation saying DNS error.

  1. support.cologlobal.com
  2. support.novacarthosting.com

Error Status : 400
Error detail : DNS problem: query timed out looking up A for support.cologlobal.com
Error type : urn:ietf:params:acme:error:dns

Error Status : 400
Error detail : DNS problem: SERVFAIL looking up A for support.novacarthosting.com
Error type : urn:ietf:params:acme:error:dns

Some tests we did on the domains:

; <<>> DiG 9.10.3-P4-Ubuntu <<>> support.novacarthosting.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 29380
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;support.novacarthosting.com. IN A

;; ANSWER SECTION:
support.novacarthosting.com. 900 IN CNAME desk.cs.zohohost.com.
desk.cs.zohohost.com. 29 IN A 8.39.54.74

;; Query time: 477 msec
;; SERVER: 192.168.100.11#53(192.168.100.11)
;; WHEN: Fri Nov 30 10:48:52 IST 2018
;; MSG SIZE rcvd: 103

We couldn’t find any problems related to DNS for those domains.

  • The status is status: NOERROR and not SERVFAIL
  • Also resolved ‘A’ record from different locations and the response is fast only.

https://dnschecker.org/#A/support.novacarthosting.com
https://dnschecker.org/#A/support.cologlobal.com

Facing this issue for the past 4 days and couldn’t get any clue to resolve this.
Kindly help us in debugging this issue ASAP as these sites were facing Certificate exception.


#2

The DNS servers don’t support TCP:

https://unboundtest.com/m/A/support.cologlobal.com/PWQ2EVQV

https://unboundtest.com/m/A/support.novacarthosting.com/Y3HMWX4D

http://dnsviz.net/d/support.cologlobal.com/XADS2A/dnssec/

Let’s Encrypt decreased the EDNS size from 4096 to 512 bytes two weeks ago, making TCP fallback more common.

I’m not sure why TCP fallback would happen in this case, but https://unboundtest.com/ – which is similar to the Let’s Encrypt resolvers – seems to be doing it regardless.

Try fixing TCP?

Edit: Ah-ha, the culprit is ginormous (okay, ~1100 byte) authority and additional sections in the CNAME response, demonstrated by “dig +norecurse support.cologlobal.com @ns2.hspheredns.com”.


#3

Might be indirectly caused by the nameserver being configured to recurse:

$ dig @ns1.hspheredns.com google.com +short
74.125.138.113
74.125.138.138
74.125.138.139
74.125.138.100
74.125.138.101
74.125.138.102

And yeah, raising edns-buffer-size to 4096 makes OP’s domain succeed.


#4

Thanks @mnordhoff, @_az - I thoroughly appreciate your DNS debugging! It’s a huge help.

@Devarajan Issuing a certificate for these names will require the authoritative nameserver providers to support TCP fallback or to reduce the size of the responses being returned. Do you operate these nameservers yourself or is it a customer?

One other note: We recommend that integrators attempt renewal of certificates when they are ~30d from expiry. Is your system attempting renewals with enough lead-time?


#5

thank you @mnordhoff and @_az for your help… :slight_smile:

Issuing a certificate for these names will require the authoritative nameserver providers to support TCP fallback or to reduce the size of the responses being returned. Do you operate these nameservers yourself or is it a customer?

@cpu those domains are our customer domains and we have informed them about the issue…

Is your system attempting renewals with enough lead-time?

yes… we are renewing certificates before 30 days of their expiry… Since these domains failed DCV during renewal schedule, they are deleted and certificate reissued and pushed to SSL offloader for deployment. So the certificate exception for those domains.

We will make necessary changes to prevent this. :slightly_smiling_face: