# Certbot renewal dns-01 challenge failure, During secondary validation: DNS problem

**URL:** <https://community.letsencrypt.org/t/certbot-renewal-dns-01-challenge-failure-during-secondary-validation-dns-problem/184428>\
**Category:** Help\
**Created:** [September 13, 2022, 2:33pm UTC](https://community.letsencrypt.org/t/certbot-renewal-dns-01-challenge-failure-during-secondary-validation-dns-problem/184428 "2022-09-13T14:33:40Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![ariah](https://avatars.discourse-cdn.com/v4/letter/a/90db22/32.png) [@ariah](https://community.letsencrypt.org/u/ariah)\
**Post date:** [September 13, 2022, 2:33pm UTC](https://community.letsencrypt.org/t/certbot-renewal-dns-01-challenge-failure-during-secondary-validation-dns-problem/184428/1 "2022-09-13T14:33:40Z")

</div>

My domain is: [https://dashboard.panorama9.com/](https://dashboard.panorama9.com/)

I ran this command:  
certbot certonly --dns-rfc2136 --dns-rfc2136-credentials -v

It produced this output:  
Certbot failed to authenticate some domains (authenticator: dns-rfc2136). The Certificate Authority reported these problems:  
Domain: [dashboard.panorama9.com](http://dashboard.panorama9.com)  
Type: dns  
Detail: During secondary validation: DNS problem: NXDOMAIN looking up TXT for \_acme-challenge.dashboard.panorama9.com - check that a DNS record exists for this domain

The version of my client is certbot 1.21.0

we have two bind name servers [ns1.panorama9.com](http://ns1.panorama9.com) and [ns2.panorama9.com](http://ns2.panorama9.com)

while renewal is ongoing i can validate that the TXT record is there with dig, whether using our primary, secondary dns or the public google dns  
dig @ns1.panorama9.com \_acme-challenge.dashboard.panorama9.com TXT +short

but i get the output from above that the secondary validation failed  
Is this related to Multi-Perspective Validation ? that some or multiple validation from some regions failed ?

I have used --dns-rfc2136-propagation-seconds to increase time up to 3 hours, i still get the same error.

---

<div class="post-metadata">

**Author:** ![rg305](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/rg305/32/91314_2.png) [@rg305](https://community.letsencrypt.org/u/rg305)\
**Post date:** [September 13, 2022, 4:39pm UTC](https://community.letsencrypt.org/t/certbot-renewal-dns-01-challenge-failure-during-secondary-validation-dns-problem/184428/2 "2022-09-13T16:39:38Z")

</div>

Hi @ariah, and welcome to the LE community forum 🙂

> [@ariah](#):
>
> Is this related to Multi-Perspective Validation ?

Probably (see **secondary** validation):

> [@ariah](#):
>
> Detail: During secondary validation: DNS problem: NXDOMAIN

---

<div class="post-metadata">

**Author:** ![ariah](https://avatars.discourse-cdn.com/v4/letter/a/90db22/32.png) [@ariah](https://community.letsencrypt.org/u/ariah)\
**Post date:** [September 13, 2022, 4:50pm UTC](https://community.letsencrypt.org/t/certbot-renewal-dns-01-challenge-failure-during-secondary-validation-dns-problem/184428/3 "2022-09-13T16:50:52Z")

</div>

> [@rg305](#):
>
> Hi @ariah, and welcome to the LE community forum 🙂

Thank you @rg305

> [@rg305](#):
>
> Probably (see **secondary** validation):

right, it says NXDOMAIN, but when i dig the TXT record from any server i get NOERROR along side the value of the record. is the certficate authority querying some dns that have yet to be propoganded ? as i mentioned before, i did wait for 3 hours. And i can't think of anything on how to further debug this

---

<div class="post-metadata">

**Author:** ![petercooperjr](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/petercooperjr/32/84698_2.png) [@petercooperjr](https://community.letsencrypt.org/u/petercooperjr)\
**Post date:** [September 13, 2022, 4:51pm UTC](https://community.letsencrypt.org/t/certbot-renewal-dns-01-challenge-failure-during-secondary-validation-dns-problem/184428/4 "2022-09-13T16:51:16Z")

</div>

Do you have the challenge token there at the moment? I'm currently seeing NXDOMAIN as well.

---

<div class="post-metadata">

**Author:** ![ariah](https://avatars.discourse-cdn.com/v4/letter/a/90db22/32.png) [@ariah](https://community.letsencrypt.org/u/ariah)\
**Post date:** [September 13, 2022, 4:55pm UTC](https://community.letsencrypt.org/t/certbot-renewal-dns-01-challenge-failure-during-secondary-validation-dns-problem/184428/5 "2022-09-13T16:55:50Z")

</div>

@petercooperjr it does now

```nohighlight
dig @ns1.panorama9.com _acme-challenge.dashboard.panorama9.com TXT +short
"qrFzGWwfKPr5E1FrvOlr6n4BoQ5cUFEq6mMHqoVJ8d4"

```

---

<div class="post-metadata">

**Author:** ![petercooperjr](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/petercooperjr/32/84698_2.png) [@petercooperjr](https://community.letsencrypt.org/u/petercooperjr)\
**Post date:** [September 13, 2022, 5:00pm UTC](https://community.letsencrypt.org/t/certbot-renewal-dns-01-challenge-failure-during-secondary-validation-dns-problem/184428/6 "2022-09-13T17:00:22Z")

</div>

Not from here

```nohighlight
$ dig @ns1.panorama9.com _acme-challenge.dashboard.panorama9.com TXT +norecurse

; <<>> DiG 9.16.22-RH <<>> @ns1.panorama9.com _acme-challenge.dashboard.panorama9.com TXT +norecurse
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 59051
;; flags: qr aa; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

```

---

<div class="post-metadata">

**Author:** ![petercooperjr](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/petercooperjr/32/84698_2.png) [@petercooperjr](https://community.letsencrypt.org/u/petercooperjr)\
**Post date:** [September 13, 2022, 5:05pm UTC](https://community.letsencrypt.org/t/certbot-renewal-dns-01-challenge-failure-during-secondary-validation-dns-problem/184428/7 "2022-09-13T17:05:25Z")

</div>

And I don't know if this is the root cause, but your DNS delegation isn't right either:

[https://dnsviz.net/d/\_acme-challenge.dashboard.panorama9.com/servers/](https://dnsviz.net/d/_acme-challenge.dashboard.panorama9.com/servers/)

> Warnings
> 
> - ` com to panorama9.com: The following NS name(s) were found in the authoritative NS RRset, but not in the delegation NS RRset (i.e., in the com zone): ns2.panorama9.org, ns1.panorama9.org`
> - ` com to panorama9.com: The following NS name(s) were found in the delegation NS RRset (i.e., in the com zone), but not in the authoritative NS RRset: ns1.panorama9.com, ns2.panorama9.com`

That is, the `.com` zone says your nameservers end in `.com`:

```nohighlight
$ dig -t NS panorama9.com. +norecurse @a.gtld-servers.net.

; <<>> DiG 9.16.22-RH <<>> -t NS panorama9.com. +norecurse @a.gtld-servers.net.
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 62877
;; flags: qr; QUERY: 1, ANSWER: 0, AUTHORITY: 2, ADDITIONAL: 3

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;panorama9.com. IN NS

;; AUTHORITY SECTION:
panorama9.com. 172800 IN NS ns1.panorama9.com.
panorama9.com. 172800 IN NS ns2.panorama9.com.

;; ADDITIONAL SECTION:
ns1.panorama9.com. 172800 IN A 46.51.187.235
ns2.panorama9.com. 172800 IN A 79.125.105.180

;; Query time: 0 msec
;; SERVER: 2001:503:a83e::2:30#53(2001:503:a83e::2:30)
;; WHEN: Tue Sep 13 17:19:00 UTC 2022
;; MSG SIZE rcvd: 110

```

**But** , your nameserver says that they actually end in `.org`:

```nohighlight
$ dig -t NS panorama9.com. +norecurse @46.51.187.235

; <<>> DiG 9.16.22-RH <<>> -t NS panorama9.com. +norecurse @46.51.187.235
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 34825
;; flags: qr aa; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 3

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;panorama9.com. IN NS

;; ANSWER SECTION:
panorama9.com. 3600 IN NS ns1.panorama9.org.
panorama9.com. 3600 IN NS ns2.panorama9.org.

;; ADDITIONAL SECTION:
ns1.panorama9.org. 1068 IN A 46.51.187.235
ns2.panorama9.org. 1780 IN A 79.125.105.180

;; Query time: 67 msec
;; SERVER: 46.51.187.235#53(46.51.187.235)
;; WHEN: Tue Sep 13 17:19:21 UTC 2022
;; MSG SIZE rcvd: 123

```

---

<div class="post-metadata">

**Author:** ![ariah](https://avatars.discourse-cdn.com/v4/letter/a/90db22/32.png) [@ariah](https://community.letsencrypt.org/u/ariah)\
**Post date:** [September 13, 2022, 5:20pm UTC](https://community.letsencrypt.org/t/certbot-renewal-dns-01-challenge-failure-during-secondary-validation-dns-problem/184428/8 "2022-09-13T17:20:06Z")

</div>

> [@petercooperjr](#):
>
> And I don't know if this is the root cause, but your DNS delegation isn't right either:

I can't say either, but afaik, until recently the dns servers were helping validating tokens correctly, and there was no change there.  
the .org servers are the one registered as NS on the hosting provider while .com are alias they both point to the same servers

> [@petercooperjr](#):
>
> Not from here
> 
> ```nohighlight
> 
> ```

this is more interesting, could it be geo related ? i've tried query on different laptops and i get the token correctly(can you please make the query again ? it'll be there for couple minutes)

---

<div class="post-metadata">

**Author:** ![MikeMcQ](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/mikemcq/32/52772_2.png) [@MikeMcQ](https://community.letsencrypt.org/u/MikeMcQ)\
**Post date:** [September 13, 2022, 5:21pm UTC](https://community.letsencrypt.org/t/certbot-renewal-dns-01-challenge-failure-during-secondary-validation-dns-problem/184428/9 "2022-09-13T17:21:52Z")

</div>

> [@ariah](#):
>
> i've tried query on different laptops and i get the token correctly(can you please make the query again ? it'll be there for couple minutes)

I can't see it either. You can try this site. It uses similar method as Let's Encrypt server uses  
[https://unboundtest.com/](https://unboundtest.com/)

---

<div class="post-metadata">

**Author:** ![petercooperjr](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/petercooperjr/32/84698_2.png) [@petercooperjr](https://community.letsencrypt.org/u/petercooperjr)\
**Post date:** [September 13, 2022, 5:23pm UTC](https://community.letsencrypt.org/t/certbot-renewal-dns-01-challenge-failure-during-secondary-validation-dns-problem/184428/10 "2022-09-13T17:23:52Z")

</div>

> [@ariah](#):
>
> this is more interesting, could it be geo related ? i've tried query on different laptops and i get the token correctly(can you please make the query again ? it'll be there for couple minutes)

From a server in AWS's us-east-1 region:

```nohighlight
[ec2-user@ip-172-31-59-142 ~]$ dig @46.51.187.235 _acme-challenge.dashboard.panorama9.com TXT +norecurse

; <<>> DiG 9.16.22-RH <<>> @46.51.187.235 _acme-challenge.dashboard.panorama9.com TXT +norecurse
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 3256
;; flags: qr aa; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;_acme-challenge.dashboard.panorama9.com. IN TXT

;; AUTHORITY SECTION:
panorama9.com. 3600 IN SOA ns1.panorama9.com. dns.jomax.net. 2016114767 28800 7200 604800 86400

;; Query time: 67 msec
;; SERVER: 46.51.187.235#53(46.51.187.235)
;; WHEN: Tue Sep 13 17:21:18 UTC 2022
;; MSG SIZE rcvd: 121

[ec2-user@ip-172-31-59-142 ~]$ dig @79.125.105.180 _acme-challenge.dashboard.panorama9.com TXT +norecurse

; <<>> DiG 9.16.22-RH <<>> @79.125.105.180 _acme-challenge.dashboard.panorama9.com TXT +norecurse
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 43328
;; flags: qr aa; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;_acme-challenge.dashboard.panorama9.com. IN TXT

;; AUTHORITY SECTION:
panorama9.com. 3600 IN SOA ns1.panorama9.com. dns.jomax.net. 2016114767 28800 7200 604800 86400

;; Query time: 67 msec
;; SERVER: 79.125.105.180#53(79.125.105.180)
;; WHEN: Tue Sep 13 17:21:30 UTC 2022
;; MSG SIZE rcvd: 121

```

From unboundtest, which uses a similar unbound configuration to that which Let's Encrypt uses (though I don't know what region it queries from):

[https://unboundtest.com/m/TXT/\_acme-challenge.dashboard.panorama9.com/NQOFTN56](https://unboundtest.com/m/TXT/_acme-challenge.dashboard.panorama9.com/NQOFTN56)

```nohighlight
Query results for TXT _acme-challenge.dashboard.panorama9.com

Response:
;; opcode: QUERY, status: NXDOMAIN, id: 11351
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:
;_acme-challenge.dashboard.panorama9.com.	IN TXT

;; AUTHORITY SECTION:
panorama9.com.	0	IN	SOA	ns1.panorama9.com. dns.jomax.net. 2016114767 28800 7200 604800 86400

```

All just saying NXDOMAIN. Are you sure you're updating the DNS servers that the outside world sees, and not some sort of visibly-internal-only server? Though the fact that you see a "secondary validation" message implies that the primary validation did see the record. Hmm…

---

<div class="post-metadata">

**Author:** ![ariah](https://avatars.discourse-cdn.com/v4/letter/a/90db22/32.png) [@ariah](https://community.letsencrypt.org/u/ariah)\
**Post date:** [September 13, 2022, 5:34pm UTC](https://community.letsencrypt.org/t/certbot-renewal-dns-01-challenge-failure-during-secondary-validation-dns-problem/184428/11 "2022-09-13T17:34:15Z")

</div>

> [@petercooperjr](#):
>
> All just saying NXDOMAIN. Are you sure you're updating the DNS servers that the outside world sees, and not some sort of visibly-internal-only server? Though the fact that you see a "secondary validation" message implies that the primary validation did see the record. Hmm…

yes, 79.125.105.180 and 46.51.187.235 are both aws bind servers registered as NS, and as you said the primary validation passed, but one or more of the validation from different regions didn't, which may mean that those dns weren't propagated with new records, hence why i did try to wait from 10 minutes to couple hours to no avail

---

<div class="post-metadata">

**Author:** ![Bruce5051](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/bruce5051/32/76576_2.png) [@Bruce5051](https://community.letsencrypt.org/u/Bruce5051)\
**Post date:** [September 13, 2022, 5:35pm UTC](https://community.letsencrypt.org/t/certbot-renewal-dns-01-challenge-failure-during-secondary-validation-dns-problem/184428/12 "2022-09-13T17:35:49Z")

</div>

I see

```nohighlight
> $ nslookup
> > server ns1.panorama9.com
> Default server: ns1.panorama9.com
> Address: 46.51.187.235#53
> > dashboard.panorama9.com
> Server: ns1.panorama9.com
> Address: 46.51.187.235#53
> 
> Name: dashboard.panorama9.com
> Address: 35.225.59.74
> > set q=soa
> > dashboard.panorama9.com
> Server: ns1.panorama9.com
> Address: 46.51.187.235#53
> 
> *** Can't find dashboard.panorama9.com: No answer
> > panorama9.com
> Server: ns1.panorama9.com
> Address: 46.51.187.235#53
> 
> panorama9.com
> origin = ns1.panorama9.com
> mail addr = dns.jomax.net
> serial = 2016114818
> refresh = 28800
> retry = 7200
> expire = 604800
> minimum = 86400
> >

```

---

<div class="post-metadata">

**Author:** ![petercooperjr](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/petercooperjr/32/84698_2.png) [@petercooperjr](https://community.letsencrypt.org/u/petercooperjr)\
**Post date:** [September 13, 2022, 5:37pm UTC](https://community.letsencrypt.org/t/certbot-renewal-dns-01-challenge-failure-during-secondary-validation-dns-problem/184428/13 "2022-09-13T17:37:09Z")

</div>

And yet I do see the record when querying 1.1.1.1 (Cloudflare) or 8.8.8.8 (Google), even after their TTL expires, but I get NXDOMAIN via 64.6.64.6 (Verisign) or 208.67.222.222 (OpenDNS) (or when I try to check the authoritative servers myself).

So it does look like your DNS servers are giving different responses to different parts of the Internet, somehow.

---

<div class="post-metadata">

**Author:** ![Bruce5051](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/bruce5051/32/76576_2.png) [@Bruce5051](https://community.letsencrypt.org/u/Bruce5051)\
**Post date:** [September 13, 2022, 5:38pm UTC](https://community.letsencrypt.org/t/certbot-renewal-dns-01-challenge-failure-during-secondary-validation-dns-problem/184428/14 "2022-09-13T17:38:17Z")

</div>

And I see 2 different IP Addresses being used around the world

> **[Check website performance and response: Check host - online website monitoring](https://check-host.net/check-http?host=https:%2F%2Fdashboard.panorama9.com)**
>
> Website checking for speed and availability with servers around the world: website monitoring with useful tools, Check IP, Check website

 ![image](https://global.discourse-cdn.com/letsencrypt/original/3X/6/b/6bfc03c159cdd1c7c2ed39245c0c627a4e55c3ab.png)

---

<div class="post-metadata">

**Author:** ![ariah](https://avatars.discourse-cdn.com/v4/letter/a/90db22/32.png) [@ariah](https://community.letsencrypt.org/u/ariah)\
**Post date:** [September 13, 2022, 5:40pm UTC](https://community.letsencrypt.org/t/certbot-renewal-dns-01-challenge-failure-during-secondary-validation-dns-problem/184428/15 "2022-09-13T17:40:16Z")

</div>

those are load balancers ip for the dashboard, in different regions: eu and us

---

<div class="post-metadata">

**Author:** ![petercooperjr](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/petercooperjr/32/84698_2.png) [@petercooperjr](https://community.letsencrypt.org/u/petercooperjr)\
**Post date:** [September 13, 2022, 5:42pm UTC](https://community.letsencrypt.org/t/certbot-renewal-dns-01-challenge-failure-during-secondary-validation-dns-problem/184428/16 "2022-09-13T17:42:48Z")

</div>

So are you saying that your DNS server intentionally gives different answers for different regions for some queries? Because if that's the case, then definitely check that your TXT update is updating the responses for all regions.

---

<div class="post-metadata">

**Author:** ![Bruce5051](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/bruce5051/32/76576_2.png) [@Bruce5051](https://community.letsencrypt.org/u/Bruce5051)\
**Post date:** [September 13, 2022, 5:46pm UTC](https://community.letsencrypt.org/t/certbot-renewal-dns-01-challenge-failure-during-secondary-validation-dns-problem/184428/17 "2022-09-13T17:46:40Z")

</div>

Since Let's Encrypt use [Multi-Perspective Validation Improves Domain Validation Security - Let's Encrypt](https://letsencrypt.org/2020/02/19/multi-perspective-validation.html)

---

<div class="post-metadata">

**Author:** ![ariah](https://avatars.discourse-cdn.com/v4/letter/a/90db22/32.png) [@ariah](https://community.letsencrypt.org/u/ariah)\
**Post date:** [September 13, 2022, 5:48pm UTC](https://community.letsencrypt.org/t/certbot-renewal-dns-01-challenge-failure-during-secondary-validation-dns-problem/184428/18 "2022-09-13T17:48:08Z")

</div>

> [@petercooperjr](#):
>
> So are you saying that your DNS server intentionally gives different answers for different regions for some queries? Because if that's the case, then definitely check that your TXT update is updating the responses for all regions.

that's most likely it, let me check on this and get back to you, thanks for the breakthrough

---

<div class="post-metadata">

**Author:** ![rg305](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/rg305/32/91314_2.png) [@rg305](https://community.letsencrypt.org/u/rg305)\
**Post date:** [September 13, 2022, 7:45pm UTC](https://community.letsencrypt.org/t/certbot-renewal-dns-01-challenge-failure-during-secondary-validation-dns-problem/184428/19 "2022-09-13T19:45:33Z")

</div>

Keep in mind that even though the problem seems to have started very recently, the root cause for it may have been implemented up to 60 days ago.

---

<div class="post-metadata">

**Author:** ![ariah](https://avatars.discourse-cdn.com/v4/letter/a/90db22/32.png) [@ariah](https://community.letsencrypt.org/u/ariah)\
**Post date:** [September 14, 2022, 10:54pm UTC](https://community.letsencrypt.org/t/certbot-renewal-dns-01-challenge-failure-during-secondary-validation-dns-problem/184428/20 "2022-09-14T22:54:19Z")

</div>

Sorry for late reply(just got access to the server)

So, yes that was it, we do have multiple zones, one for US regions, and the other zone for the rest, we were always updating the us one, and it was working file until recently(idk if Multi-Perspective Validation was implemented recently hence why validating from multiple regions ?)

Using certbot, is it possible to do the dns-challenge by adding the record simultaneously to the two zones ? or must we configure one zone to transfer data to the other ?

[Next page](https://community.letsencrypt.org/t/certbot-renewal-dns-01-challenge-failure-during-secondary-validation-dns-problem/184428.md?page=2)