@joegold100 I spoke w/the Let's Debug developer (@Nummer378) and he has quite a lot to share. He was going offline so asked me to share it. Mind you, these are results from Let's Debug so may not reflect exactly what Let's Encrypt sees. But, I think this is likely pointing to the underlying problem. Note especially that he reproduced an error using @1.1.1.1 too.
Let's Debug and Let's Encrypt both use unbound for resolving. However, their configurations may well be slightly different. Further, LE validates from several locations around the world mostly simultaneously which can cause issues not seen by a system doing single queries.
This post focuses on the failures. The next post focuses on why some domains fail and not others.
=====================================================
(... some earlier log debugging omitted for brevity ...)
Why does it not have any nameservers? The logs clearly show it trying to query several nameservers:
Mar 09 23:16:33 letsdebug letsdebug-server[1448376]: [1773098193] libunbound[1448376:0] info: response for ns2.ecservers.net. AAAA IN
Mar 09 23:16:33 letsdebug letsdebug-server[1448376]: [1773098193] libunbound[1448376:0] info: reply from <net.> 192.26.92.30#53
Mar 09 23:16:33 letsdebug letsdebug-server[1448376]: [1773098193] libunbound[1448376:0] info: query response was REFERRAL
Mar 09 23:16:33 letsdebug letsdebug-server[1448376]: [1773098193] libunbound[1448376:0] info: processQueryTargets: ns2.ecservers.net. AAAA IN
Well, no real idea but there's this one suspicious line in the logs:
info: skipping target due to dependency cycle (harden-glue: no may fix some of the cycles) ns2.ecservers.net. A IN
So something seems to go wrong while resolving the nameservers themselves. And then I suddenly could reproduce even on cloudflare's 1.1.1.1:
dig NS ecservers.net @1.1.1.1
; <<>> DiG 9.20.18-1~deb13u1-Debian <<>> NS ecservers.net @1.1.1.1
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 11697
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
; EDE: 23 (Network Error): (54.67.108.165:53 returned REFUSED for ecservers.net NS)
;; QUESTION SECTION:
;ecservers.net. IN NS
;; Query time: 344 msec
;; SERVER: 1.1.1.1#53(1.1.1.1) (UDP)
;; WHEN: Mon Mar 09 23:27:54 UTC 2026
;; MSG SIZE rcvd: 102
dig NS ns6.ecservers.net +dnssec @1.1.1.1
; <<>> DiG 9.20.18-1~deb13u1-Debian <<>> NS ns6.ecservers.net +dnssec @1.1.1.1
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 50460
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 1232
; EDE: 23 (Network Error): (54.67.108.165:53 returned REFUSED for ecservers.net DNSKEY)
;; QUESTION SECTION:
;ns6.ecservers.net. IN NS
;; ANSWER SECTION:
ns6.ecservers.net. 86400 IN NS ns1.ecservers.net.
ns6.ecservers.net. 86400 IN NS ns2.ecservers.net.
;; Query time: 340 msec
;; SERVER: 1.1.1.1#53(1.1.1.1) (UDP)
;; WHEN: Mon Mar 09 23:28:54 UTC 2026
;; MSG SIZE rcvd: 146
After some further digging, I found a severly broken configuration:
ecservers.net is hosted on ns{1}.ecservers.net according to the net nameservers. That's a cycle (which unbound doesn't like), as one needs to know the nameserver for ecservers.net to get the nameservers for ns1/2.ecservers.net. The .net nameservers return a "glue record" with the A/AAAA IPs to solve this problem - many standard nameservers will just accept the glue and move on, if everything else is in order. But from experience, unbound likes to be picky and tries to validate the glues, which fails since the ns1/2.ecservers.net nameservers seem to drop all DNS responses for "ecservers.net":
dig A ecservers.net @54.67.108.165
;; communications error to 54.67.108.165#53: timed out
;; communications error to 54.67.108.165#53: timed out
;; communications error to 54.67.108.165#53: timed out
; <<>> DiG 9.20.18-1~deb13u1-Debian <<>> A ecservers.net @54.67.108.165
;; global options: +cmd
;; no servers could be reached
(At least from Let's Debugs perspective)
So the nameservers do not reply for their main domain, but according to delegation they should. This will cause issues when unbound tries to validate the referral and also grabbing DNSSEC keys for ecservers.net. Therefore Let's Debug won't accept the nameserver IPs and thus can't query the final domain name, as it has no validated nameserver to ask.
