Max 10 domains in DNS verification

kongen · July 21, 2021, 10:40am

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

I'm setting up a mailserver with 12 domains and I'm trying to validate with DNS validation.
The DNS records are based on a template so I have to put all 12 "_acme-challenge." in the same place. It looks like certbot only can handle 10 "_acme-challenge." entries as the last two domains are failing and keep getting new values for "_acme-challenge." where the first 10 keep the same value?

Also it says that it found one value (and 9 more)... there are 12.

I guess I could split the DNS template into two, but it is a little complex setup so I would prefer to keep it in one.

Is there another way to get around this?

Best regards
Ulrik

My domain is: ezida.dk cybersite.dk furehus.dk guilds.dk kofod.me odds-on.dk oddsetligaen.dk periconsult.com periconsult.dk ring.gl same-game.dk samegame.dk xn--sbakken2800-ggb.dk

I ran this command: certbot certonly --manual --preferred-challenges=dns

It produced this output:
Failed authorization procedure. samegame.dk (dns-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Incorrect TXT record "37aetltdGOaYf77iA0KmHnJoiaJeKLk5HMjdrmpPmZc" (and 9 more) found at _acme-challenge.samegame.dk, xn--sbakken2800-ggb.dk (dns-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Incorrect TXT record "k5r9RBAIotXUvS5ySVL9owGK8_zZdm9j0CHAZKOqgBY" (and 9 more) found at _acme-challenge.xn--sbakken2800-ggb.dk

IMPORTANT NOTES:

The following errors were reported by the server:

Domain: samegame.dk
Type: unauthorized
Detail: Incorrect TXT record
"37aetltdGOaYf77iA0KmHnJoiaJeKLk5HMjdrmpPmZc" (and 9 more) found at
_acme-challenge.samegame.dk

Domain: xn--sbakken2800-ggb.dk
Type: unauthorized
Detail: Incorrect TXT record
"k5r9RBAIotXUvS5ySVL9owGK8_zZdm9j0CHAZKOqgBY" (and 9 more) found at
_acme-challenge.xn--sbakken2800-ggb.dk

To fix these errors, please make sure that your domain name was
entered correctly and the DNS A/AAAA record(s) for that domain
contain(s) the right IP address.

My web server is (include version):
none - dns verification

The operating system my web server runs on is (include version):
Ubuntu 18.04

My hosting provider, if applicable, is: None/own server

I can login to a root shell on my machine (yes or no, or I don't know): Yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): NO

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):

certbot 0.27.0

rg305 · July 21, 2021, 2:39pm

Try changing their order.
If any have recently passed the challenge, LE may have them cached and would skip them.

jvanasco · July 22, 2021, 7:20pm

The LetsEncrypt Server, Boulder, caches and reuses a few things:

Valid Authorizations. Cached authorizations last for 30 days from the time of validation.
Pending Authorizations. Pending authorizations live for 7 days and can be reused across orders.

What is most-likely happening is the following:

You created an AcmeOrder for 12 domains
Your client quits after the first challenge fails. That probably keeps happening to the same 2 domains. The failed domain will receive a new authorization on the next attempt, but the other domains will not change authorizations because they are still pending.

In terms of DNS issues and records, your provider could be using some sort of caching system. That messes up a lot of users. You can use https://unboundtest.com/ to see what letsencrypt sees.

Osiris · July 22, 2021, 7:23pm

I'm failing to see how this is a certbot issue. I've tested certbot just now:

server ~ # DOMAINS=$(echo le-dns-test-{1..20}.example.com | tr ' ' ,)
server ~ # echo $DOMAINS
le-dns-test-1.example.com,le-dns-test-2.example.com,le-dns-test-3.example.com,le-dns-test-4.example.com,le-dns-test-5.example.com,le-dns-test-6.example.com,le-dns-test-7.example.com,le-dns-test-8.example.com,le-dns-test-9.example.com,le-dns-test-10.example.com,le-dns-test-11.example.com,le-dns-test-12.example.com,le-dns-test-13.example.com,le-dns-test-14.example.com,le-dns-test-15.example.com,le-dns-test-16.example.com,le-dns-test-17.example.com,le-dns-test-18.example.com,le-dns-test-19.example.com,le-dns-test-20.example.com
server ~ # certbot certonly --staging --dns-rfc2136 --dns-rfc2136-credentials /etc/letsencrypt/rfc-2136.ini --dns-rfc2136-propagation-seconds 2 -d $DOMAINS
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Requesting a certificate for le-dns-test-1.example.com and 19 more domains
Waiting 2 seconds for DNS changes to propagate

Certbot failed to authenticate some domains (authenticator: dns-rfc2136). The Certificate Authority reported these problems:
  Domain: le-dns-test-13.example.com
  Type:   dns
  Detail: During secondary validation: DNS problem: query timed out looking up TXT for _acme-challenge.le-dns-test-13.example.com

  Domain: le-dns-test-17.example.com
  Type:   dns
  Detail: During secondary validation: DNS problem: query timed out looking up CAA for le-dns-test-17.example.com

Hint: The Certificate Authority failed to verify the DNS TXT records created by --dns-rfc2136. Ensure the above domains are hosted by this DNS provider, or try increasing --dns-rfc2136-propagation-seconds (currently 2 seconds).

Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.
server ~ #

Note that while I did had two failing challenges due to separate issues, 18 out of the 20 challenges were succesful! (And a second run succesfully validated the two remaining challenges.)

I think it's more likely this is a limitation of your DNS servers method of adding TXT records to your DNS zone.

kongen · July 23, 2021, 7:50am

I split it into two DNS templates with 10 validation values in the first and the remaining 2 in the nexe, and then it worked fine.

Changing the order did not work as certbot sorts the domains, so it was always the last 2 that could not validate.

The problem simply is that certbot can not handle if there are more than 10 _acme-challenge TXT records in one DNS lookup.

Osiris · July 23, 2021, 7:51am

That's just simply not true, as I've shown above?

Nummer378 · July 23, 2021, 8:15am

You're putting a lot of TXT records under the same name (unneccessarily), due to an issue with your zone configuration.

This cannot scale nicely, as DNS records will always start to see rejections by resolvers as they hit certain size limitations. See also the statement from the https://letsencrypt.org/docs/challenge-types/#dns-01-challenge:

You can have multiple TXT records in place for the same name. For instance, this might happen if you are validating a challenge for a wildcard and a non-wildcard certificate at the same time. However, you should make sure to clean up old TXT records, because if the response size gets too big Let’s Encrypt will start rejecting it.

This is not a limitation from certbot, but Let's Encrypts resolvers. And given enough names, any resolver will have the same issue.

Note that you can delegate challenges to a different zone using CNAMEs. This could solve your template problem. Without the duplicate TXT records everything should be fine, as demonstrated by Osiris.

kongen · July 23, 2021, 8:21am

It's not on the same domain, it is a template applied to 13 domains, so each validation value has to be there.

But as I said, I fixed it by splitting the template into two. 10 domains in one and 3 in another identical, except for the _acme-challenge's

Osiris · July 23, 2021, 8:35am

I did another test on the Let's Encrypt staging server, but now with 20+ TXT records for a single hostname, as that's more the same as this current issue. Here are the results:

DNS configuration:

$ORIGIN .
$TTL 900        ; 15 minutes
example.com        IN SOA  example.net. root.example.net. (
                                2021072303 ; serial
                                7200       ; refresh (2 hours)
                                3600       ; retry (1 hour)
                                604800     ; expire (1 week)
                                21600      ; minimum (6 hours)
                                )
                        NS      example.net.
                        A       x.x.x.x
                        MX      10 mx.example.com.
                        AAAA    y:y:y:y::y
$ORIGIN example.com.
*                       A       x.x.x.x
                        AAAA    y:y:y:y::y
_acme-challenge.le-dns-test-21  TXT     abcdefghijklmnopqrstuvwxyzQWERTY01
_acme-challenge.le-dns-test-21  TXT     abcdefghijklmnopqrstuvwxyzQWERTY02
_acme-challenge.le-dns-test-21  TXT     abcdefghijklmnopqrstuvwxyzQWERTY03
_acme-challenge.le-dns-test-21  TXT     abcdefghijklmnopqrstuvwxyzQWERTY04
_acme-challenge.le-dns-test-21  TXT     abcdefghijklmnopqrstuvwxyzQWERTY05
_acme-challenge.le-dns-test-21  TXT     abcdefghijklmnopqrstuvwxyzQWERTY06
_acme-challenge.le-dns-test-21  TXT     abcdefghijklmnopqrstuvwxyzQWERTY07
_acme-challenge.le-dns-test-21  TXT     abcdefghijklmnopqrstuvwxyzQWERTY08
_acme-challenge.le-dns-test-21  TXT     abcdefghijklmnopqrstuvwxyzQWERTY09
_acme-challenge.le-dns-test-21  TXT     abcdefghijklmnopqrstuvwxyzQWERTY10
_acme-challenge.le-dns-test-21  TXT     abcdefghijklmnopqrstuvwxyzQWERTY11
_acme-challenge.le-dns-test-21  TXT     abcdefghijklmnopqrstuvwxyzQWERTY12
_acme-challenge.le-dns-test-21  TXT     abcdefghijklmnopqrstuvwxyzQWERTY13
_acme-challenge.le-dns-test-21  TXT     abcdefghijklmnopqrstuvwxyzQWERTY14
_acme-challenge.le-dns-test-21  TXT     abcdefghijklmnopqrstuvwxyzQWERTY15
_acme-challenge.le-dns-test-21  TXT     abcdefghijklmnopqrstuvwxyzQWERTY16
_acme-challenge.le-dns-test-21  TXT     abcdefghijklmnopqrstuvwxyzQWERTY17
_acme-challenge.le-dns-test-21  TXT     abcdefghijklmnopqrstuvwxyzQWERTY18
_acme-challenge.le-dns-test-21  TXT     abcdefghijklmnopqrstuvwxyzQWERTY19
_acme-challenge.le-dns-test-21  TXT     abcdefghijklmnopqrstuvwxyzQWERTY20
_acme-challenge.le-dns-test-21  TXT     WzU8e4bHf3nZskXBhAatceyZcqY52xHkc-SsN8Tp7mg

A test with dig:

osiris@erazer ~ $ dig @example.net +norecurse _acme-challenge.le-dns-test-21.example.com TXT

; <<>> DiG 9.16.12 <<>> @example.net +norecurse _acme-challenge.le-dns-test-21.example.com TXT
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 8111
;; flags: qr aa; QUERY: 1, ANSWER: 21, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
; COOKIE: d70f135542c3bf560000000060fa6dbf284f8dc779f42a97 (good)
;; QUESTION SECTION:
;_acme-challenge.le-dns-test-21.example.com. IN TXT

;; ANSWER SECTION:
_acme-challenge.le-dns-test-21.example.com. 900 IN      TXT "abcdefghijklmnopqrstuvwxyzQWERTY09"
_acme-challenge.le-dns-test-21.example.com. 900 IN      TXT "abcdefghijklmnopqrstuvwxyzQWERTY08"
_acme-challenge.le-dns-test-21.example.com. 900 IN      TXT "abcdefghijklmnopqrstuvwxyzQWERTY07"
_acme-challenge.le-dns-test-21.example.com. 900 IN      TXT "abcdefghijklmnopqrstuvwxyzQWERTY18"
_acme-challenge.le-dns-test-21.example.com. 900 IN      TXT "abcdefghijklmnopqrstuvwxyzQWERTY12"
_acme-challenge.le-dns-test-21.example.com. 900 IN      TXT "abcdefghijklmnopqrstuvwxyzQWERTY02"
_acme-challenge.le-dns-test-21.example.com. 900 IN      TXT "abcdefghijklmnopqrstuvwxyzQWERTY04"
_acme-challenge.le-dns-test-21.example.com. 900 IN      TXT "abcdefghijklmnopqrstuvwxyzQWERTY16"
_acme-challenge.le-dns-test-21.example.com. 900 IN      TXT "abcdefghijklmnopqrstuvwxyzQWERTY10"
_acme-challenge.le-dns-test-21.example.com. 900 IN      TXT "abcdefghijklmnopqrstuvwxyzQWERTY05"
_acme-challenge.le-dns-test-21.example.com. 900 IN      TXT "abcdefghijklmnopqrstuvwxyzQWERTY14"
_acme-challenge.le-dns-test-21.example.com. 900 IN      TXT "abcdefghijklmnopqrstuvwxyzQWERTY03"
_acme-challenge.le-dns-test-21.example.com. 900 IN      TXT "abcdefghijklmnopqrstuvwxyzQWERTY01"
_acme-challenge.le-dns-test-21.example.com. 900 IN      TXT "abcdefghijklmnopqrstuvwxyzQWERTY06"
_acme-challenge.le-dns-test-21.example.com. 900 IN      TXT "abcdefghijklmnopqrstuvwxyzQWERTY11"
_acme-challenge.le-dns-test-21.example.com. 900 IN      TXT "abcdefghijklmnopqrstuvwxyzQWERTY13"
_acme-challenge.le-dns-test-21.example.com. 900 IN      TXT "abcdefghijklmnopqrstuvwxyzQWERTY15"
_acme-challenge.le-dns-test-21.example.com. 900 IN      TXT "abcdefghijklmnopqrstuvwxyzQWERTY20"
_acme-challenge.le-dns-test-21.example.com. 900 IN      TXT "WzU8e4bHf3nZskXBhAatceyZcqY52xHkc-SsN8Tp7mg"
_acme-challenge.le-dns-test-21.example.com. 900 IN      TXT "abcdefghijklmnopqrstuvwxyzQWERTY17"
_acme-challenge.le-dns-test-21.example.com. 900 IN      TXT "abcdefghijklmnopqrstuvwxyzQWERTY19"

;; AUTHORITY SECTION:
example.com.    900     IN      NS      example.net.

;; Query time: 27 msec
;; SERVER: x.x.x.x#53(x.x.x.x)
;; WHEN: Fri Jul 23 10:28:47 CEST 2021
;; MSG SIZE  rcvd: 1143

osiris@erazer ~ $

Certbot:

server ~ # certbot certonly --staging --manual --preferred-challenge=dns -d le-dns-test-21.example.com
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Requesting a certificate for le-dns-test-21.example.com

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Please deploy a DNS TXT record under the name:

_acme-challenge.le-dns-test-21.example.com.

with the following value:

WzU8e4bHf3nZskXBhAatceyZcqY52xHkc-SsN8Tp7mg

Before continuing, verify the TXT record has been deployed. Depending on the DNS
provider, this may take some time, from a few seconds to multiple minutes. You can
check if it has finished deploying with aid of online tools, such as the Google
Admin Toolbox: https://toolbox.googleapps.com/apps/dig/#TXT/_acme-challenge.le-dns-test-21.example.com.
Look for one or more bolded line(s) below the line ';ANSWER'. It should show the
value(s) you've just added.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Press Enter to Continue

Successfully received certificate.
Certificate is saved at: /etc/letsencrypt/live/le-dns-test-21.example.com/fullchain.pem
Key is saved at:         /etc/letsencrypt/live/le-dns-test-21.example.com/privkey.pem
This certificate expires on 2021-10-21.
These files will be updated when the certificate renews.

NEXT STEPS:
- The certificate will need to be renewed before it expires. Certbot can automatically renew the certificate in the background, but you may need to take steps to enable that functionality. See https://certbot.org/renewal-setup for instructions.
server ~ #

With this I can only confirm that this is not an issue with certbot nor with the Let's Encrypt validation servers: they both happily accept more than 20 TXT records for the same hostname.

Therefore, I must conclude it has to be an issue with your DNS service providers method of adding TXT records to your zone.

kongen · July 23, 2021, 8:55am

That is exactly what I did? And I could see all 13 TXT records in "dig"?
I'm confused now Except if you got lucky and your validation key was one of the 10 that was checked.

I just added 3 TESTING TXT records and all 13 TXT records show?

% dig _acme-challenge.odds-on.dk TXT 

; <<>> DiG 9.10.6 <<>> _acme-challenge.odds-on.dk TXT
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 40189
;; flags: qr rd ra; QUERY: 1, ANSWER: 13, AUTHORITY: 13, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;_acme-challenge.odds-on.dk.	IN	TXT

;; ANSWER SECTION:
_acme-challenge.odds-on.dk. 7029 IN	TXT	"heU9FTi5DH8D7Euci8d2yunUwQA3vvU83HXVKhbm0Xc"
_acme-challenge.odds-on.dk. 7029 IN	TXT	"TESTING 2"
_acme-challenge.odds-on.dk. 7029 IN	TXT	"_mxnRNglQCRNTrCpxRobIl2KgTQ75pTGYK-xH4XuLpE"
_acme-challenge.odds-on.dk. 7029 IN	TXT	"k3nyMUALZc-pPp2wEHq8BUxwzhOnRbjuMT7XE2Wr4R0"
_acme-challenge.odds-on.dk. 7029 IN	TXT	"TESTING 3"
_acme-challenge.odds-on.dk. 7029 IN	TXT	"p7AP6-oLBKeoVRuYfEje71G64Se6FKe6nHdQ5J9ADpc"
_acme-challenge.odds-on.dk. 7029 IN	TXT	"VpFzgMVukfBs-1cG1QTTWYV4r8BAfqs5CVgQ38jkmtM"
_acme-challenge.odds-on.dk. 7029 IN	TXT	"XWg69u7rtWNSlpQnpqlkzSEZ6K1lxRf9vT9S6WX3Abo"
_acme-challenge.odds-on.dk. 7029 IN	TXT	"piFdSXdv1ZjH8NKUcs-IfKWas-nA7MhDfrwFvO1HNOA"
_acme-challenge.odds-on.dk. 7029 IN	TXT	"Tp-lk8QpjuJQZmtLwZwOmzQl3RYQMcNAkRG2xU_Qvok"
_acme-challenge.odds-on.dk. 7029 IN	TXT	"_OwRXvTbN8A-04ayWmjbU0YAMFyOOpqbCPB1gdTlRgc"
_acme-challenge.odds-on.dk. 7029 IN	TXT	"0bqr64nIuDq-TT8u70Gg7pxGWkOEkqIxFFK3d8yDMho"
_acme-challenge.odds-on.dk. 7029 IN	TXT	"TESTING 1"

Osiris · July 23, 2021, 9:00am

Well, that's indeed a possibility, when I ran dig a few times, the order of the resulting TXT records was differently every time.. I could add a ton of TXT records of course, lemme try that.

kongen · July 23, 2021, 9:05am

Don't spend all your time on this I did fix it for me by splitting the template, so it is not urgent for me, that it is solved.

Osiris · July 23, 2021, 9:05am

It seems Unbound (the resolver software used by LE) doesn't like 100 TXT records for a single hostname:

Error running query: dns: failed to unpack truncated message

kongen · July 23, 2021, 9:06am

it should be enough with 11 values for 11 domains

Osiris · July 23, 2021, 9:08am

Well, more than 10 values with a single value per domain and more than 10 values for a single domain were already succesful earlier (20 values). But as you said, I might have been lucky on that single try.

76 values seems to work, I'll try a few times.

OK, I think I hit a caching issue at the LE side here when I tried the 76 values for a second time (but with the updated real challenge of course):

Detail: Incorrect TXT record "abcdefghijklmnopqrstuvwxyzQWERTY34" (and 75 more) found at _acme-challenge.le-dns-test-21.example.com

But this does confirm that the Let's Encrypt validation server does not have a limit of just 10 TXT records, as shown in the "and 75 more" from the error message.

kongen · July 23, 2021, 9:16am

I never managed to make it say more than (and 9 more), that is why I assumed it was a limit of 10. When I split it into two (10 and 3) it worked right away.

system · August 22, 2021, 9:17am

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
DNS Challenge keeps failing Help	7	2482	October 29, 2022
Failed validation limit Help	14	1256	November 28, 2023
Just one _acme-challenge value for all domains in same DNS validation Feature Requests	23	5846	August 22, 2021
_acme-challenge Help	17	1073	May 3, 2024
Certbot mixed-case_acmE-ChalLENGE fails Help	8	880	July 11, 2021

Max 10 domains in DNS verification

Related topics