DNSKEY Missing: validation failure

Jens · July 20, 2024, 11:38am

My domain is:

www.developmentscout.com

I ran this command:

certbot certonly --dry-run --webroot --agree-tos --email struck.jens@gmail.com -d developmentscout.com -d www.developmentscout.com -d www.developmentscout.de -d developmentscout.de -d development-scout.com -d www.development-scout.com -w /var/www/vhosts/developmentscout.com/htdocs/

It produced this output:

Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
Domain: developmentscout.com
Type: dns
Detail: DNS problem: looking up A for developmentscout.com: DNSSEC: DNSKEY Missing: validation failure <developmentscout.com. A IN>: No DNSKEY record from 2001:67c:192c::add:5 for key developmentscout.com. while building chain of trust; DNS problem: looking up AAAA for developmentscout.com: DNSSEC: DNSKEY Missing: validation failure <developmentscout.com. AAAA IN>: No DNSKEY record from 2001:67c:192c::add:5 for key developmentscout.com. while building chain of trust

My web server is (include version):

Apache/2.4.61 (Debian)

The operating system my web server runs on is (include version):

Debian-1205-bookworm-amd64-base

My hosting provider, if applicable, is:

Hetzner

I can login to a root shell on my machine (yes or no, or I don't know):

yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):

no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):

certbot 2.1.0

Osiris · July 20, 2024, 11:59am

Your DNSSEC is severely broken. See e.g. developmentscout.com | DNSViz.

Jens · July 20, 2024, 12:33pm

Thanks Osiris for you answer. But what does this mean exactly? Have used the same setup with OVH but moving the domains today to Hetzner and got this errors? Any ideas?

Osiris · July 20, 2024, 12:55pm

I guess that move didn't go without troubles. I think there are 2 issues:

error: the DS RR in the .com. zone hasn't been updated (or removed, depending on the DNSSEC situation at Hetzner);
warning: AAAA glue RRs for hydrogen.ns.hetzner.com. and oxygen.ns.hetzner.com. are missing from the .com. nameservers and there are no glue RRs for helium.ns.hetzner.de. entirely.

Looks like Hetzner is not configured for DNSSEC entirely. Either enable DNSSEC at Hetzner and update the DS RR in the .com. zone (your DNS registrar needs to do that I think) or remove the DS RR from the .com. zone entirely. (Also your registrar I think.)

Jens · July 20, 2024, 1:11pm

Thanks again for helping me Osiris. Hetzner doesn't has DNSSEC (https://docs.hetzner.com/dns-console/dns/general/dnssec/), so if I have understand everything correctly, they could not enable DNSSEC, right?

I don't have a DS RR entry, so how can I remove this?

Osiris · July 20, 2024, 1:17pm

Yes, you should either change to a DNS provider which is capable in general (as I believe a DNS provider not being able to provide DNSSEC is frankly not capable) or disable DNSSEC by removing the DS RR from the .com. zone.

"You" do have it, it's just in the .com. zone where it belongs. Probably a relic from your previous DNS provider. See:

osiris@erazer ~ $ dig @j.gtld-servers.net. developmentscout.com DS

; <<>> DiG 9.16.42 <<>> @j.gtld-servers.net. developmentscout.com DS

(...)

;; QUESTION SECTION:
;developmentscout.com.		IN	DS

;; ANSWER SECTION:
developmentscout.com.	86400	IN	DS	3034 8 2 F65FA984AA0CAE7D1BE6BE4780A7A6409A6EDC75F708DCFF3FE49D15 556A85F3

(...)

That's the job of the DNS registrar. This can be the same as the DNS service provider where you can edit the DNS zone of your domain, but it can also be a completely different company. Looking at the WHOIS info of your domain name, the registrar seems to be Hetzner also. I.e.: Hetzner should be able to remove this DS resource record from the .com zone. If they don't offer this in their DNS registrar options, then maybe their customer service can.

Please note that DNSSEC enforcing nameservers will refuse to resolve your domain name entirely, so it's not only a matter of Let's Encrypt certificates, but for your site to be reachable for a large part of the internet at large.

See e.g. the DNS output of the DNS resolver of my ISP:

osiris@erazer ~ $ dig developmentscout.com

; <<>> DiG 9.16.42 <<>> developmentscout.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 251
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;developmentscout.com.		IN	A

;; Query time: 27 msec
;; SERVER: ...#53(...)
;; WHEN: Sat Jul 20 15:17:59 CEST 2024
;; MSG SIZE  rcvd: 49

osiris@erazer ~ $

See the SERVFAIL status.

Jens · July 20, 2024, 2:22pm

O.K. Osiris. Tried to contact Hetzner because your are right. The developmentscout.com. 86400 IN DS 3034 8 2 F65FA984AA0CAE7D1BE6BE4780A7A6409A6EDC75F708DCFF3FE49D15 556A85F3
record comes from the SOA entry directly from Hetzner, which I cannot delete (lock symbol...).
Unfortunately their support for that is only available on Monday, so I try to switch to Cloudflare now.

Osiris · July 20, 2024, 2:24pm

I'm not sure I follow. Maybe Hetzner has put that info together with the SOA RR, but it's not part of it.

As a registrar? Or as DNS service provider? Because they would need to be the registrar to be able to remove/modify the DS RR in the .com zone as far as I know (but I'm not 100 % familair with DS RR).

Jens · July 20, 2024, 2:33pm

As far as I know Hetzner is both. The registar and the DNS provider. The switch should be because I'm able to change the NS nameserver records to cloudflare and the have their own chances to create a SSL certificate.

That's my current DNS zone file:

$ORIGIN germanonlinepublisher.com.
$TTL 86400
; SOA Records
@ IN SOA hydrogen.ns.hetzner.com. dns.hetzner.com. 2024072004 86400 10800 3600000 3600
; NS Records
@ 60 IN NS jessica.ns.cloudflare.com.
@ 60 IN NS clyde.ns.cloudflare.com
; MX Records
@ 60 IN MX 5 server4
; A Records
@ 60 IN A 148.251.46.178
localhost 60 IN A 127.0.0.1
server4 60 IN A 148.251.46.178
www 60 IN A 148.251.46.178
; AAAA Records
@ 60 IN AAAA 2a01:4f8:202:1ac::2
server4 60 IN AAAA 2a01:4f8:202:1ac::2
www 60 IN AAAA 2a01:4f8:202:1ac::2
; CNAME Records
loopback 60 IN CNAME localhost
; TXT Records
@ 60 IN TXT "google-site-verification=yoOZUNZAu5JwG_ty2Zph3QScH5qsTaQ4g96e_ILguLI"
@ 60 IN TXT "v=spf1 a mx ip4:148.251.46.178 -all"
_dmarc 60 IN TXT "v=DMARC1;p=none;"
acy._domainkey 60 IN TXT "v=DKIM1;s=email;p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCm2I6KQ4WUbRvMIUzEYiH0c0hdp+i6iAstvxTXKUUgR/M/2Ack4L1dtXsPx5rATSw3NKadhb5PsyrT8LbfPfUG35bXxMpl4DVFAYY+GSqZ8AbFFEyMHH/3rbPlXxJdp/usRoZS9nevxnMFUbmHkFce+vGNzzAjizaBccTK32A8rwIDAQAB;t=s;"
server4 60 IN TXT "v=spf1 a mx ip4:148.251.46.178 -all"

So I thought the DS RR comes from it.

Osiris · July 20, 2024, 2:37pm

I don't see how the DS RR would come from the SOA RR with what you have provided: I can't see it anywhere?

It is, yes.

Which switch? DNS registrar? DNS provider? Or both?

If you just switch DNS provider to Cloudflare without also switching DNS registrar and remove the erroneous DS RR from the .com zone, the switch wouldn't help you at all. It would just provide DNSSEC errors with some additional Cloudflare nameserver stuff.

Jens · July 20, 2024, 2:43pm

Got it. But when you don't see the DS RR anywhere, where does it come from and how can I remove the erroneous DS RR from the .com zone?

Nummer378 · July 20, 2024, 2:44pm

The actual management is of course done by the TLD zone operator itself. There are various protocols to manage data in a parent zone automatically. For DS records there's RFC 8078 - Managing DS Records from the Parent via CDS/CDNSKEY, if the child and parent zone both support it. Of course the most reliable way is via the registar, since they will almost certainly have API access to the TLD zone.

Based on a whois lookup, Hetzner is indeed your domain registar. I have looked through their UI, but couldn't find any obvious setting. I suggest you talk to their support, I presume they can fix that DS issue for you in a few minutes. Just tell them that you would like the DS record removed from the parent com zone.

(Though I should mention: If you're moving your nameservers to Cloudflare: Cloudflare does support DNSSEC, and AFAIK they do try to manage the DS automatically, if possible)

Jens · July 20, 2024, 2:50pm

Alright. As I mentioned, I'm trying to switch to Cloudflare right now (takes a little bit for my nameserver change). And on Monday I will contact Hetzner because of the weird DS record

Osiris · July 20, 2024, 2:51pm

It came from one of your previous DNS service providers or DNS registrars. That company put the DS RR in the .com zone when they enabled DNSSEC. You may or may not have had any control over that.

How to remove it depends on Hetzner. For example, my DNS service provider and registrar (same company) simply has a statement "DNSSEC: active" but no way to change that.

IMO a DNS registrar or service provider should remove the DS RR once the domain name has been transfered to a different DNS service provider, as they would know DNSSEC would break otherwise. But I don't know if that's common or not.

Jens · July 20, 2024, 2:58pm

Yep. OVH has DNSSEC and I have not switch it Off during the domain transfer process. So it could be that this information has been transfered to Hetzner and they don't removed it automatically.

linkp · July 20, 2024, 2:59pm

You won't be able to successfully complete that until the DS record is removed from the parent zone.

Jens · July 20, 2024, 2:59pm

Weir thing is that with the domain presseservicebuero.de I could create a Let's Encrypt SSL certificate without any issues. Same setup...

Osiris · July 20, 2024, 3:01pm

Semi-offtopic: I was just made aware the correct spelling is "registRar", so I've edited all my posts to add some extra R's English is not my native language

Jens · July 20, 2024, 3:02pm

No problem. Same to me English is not my first language also

Osiris · July 20, 2024, 3:02pm

No, that domain name is registrered and managed by Cloudflare, not Hetzner. And has DNSSEC enabled without major errors. (It does have some errors at presseservicebuero.de | DNSViz though, but I guess not severe enough to break DNSSEC when resolving the domain.)