Can I specify the DNS resolver to be used for DNS challenge with certbot similar to traefik?

My operating system is (include version): Ubuntu 24.04

I installed Certbot with (snap, OS package manager, pip, certbot-auto, etc):

I'm actually run SWAG docker implementation which I'm aware runs certbot within a container. (bear with me). I'm not looking for docker help as the issue has to do with certbot and specifically with the inability to specify a DNS resolver.

I ran this command and it produced this output:

Certbot failed to authenticate some domains (authenticator: dns-cloudflare). The Certificate Authority reported these problems:
  Domain: disco.<domain>.com
  Type:   unauthorized
  Detail: No TXT record found at _acme-challenge.disco.<domain>.com

Hint: The Certificate Authority failed to verify the DNS TXT records created by --dns-cloudflare. Ensure the above domains are hosted by this DNS provider, or try increasing --dns-cloudflare-propagation-seconds (currently 10 seconds).

Certbot's behavior differed from what I expected because:

So I'm aware what causing my problem. My linux server is setup via systemd-resolved with the DNS names resolved locally using pfSense unbound. If I do a simple dns lookup with drill you'll see:

drill disco.<domain>.com
;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 57770
;; flags: qr rd ra ; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;; disco.<domain>.com.	IN	A

;; ANSWER SECTION:
disco.<domain>.com.	3600	IN	A	10.0.1.161

;; AUTHORITY SECTION:

;; ADDITIONAL SECTION:

;; Query time: 1 msec
;; SERVER: 127.0.0.53
;; WHEN: Wed Dec 18 01:22:35 2024
;; MSG SIZE  rcvd: 52

My current DNS server's are:

$ sudo resolvectl status 
Global
           Protocols: -LLMNR -mDNS DNSOverTLS=opportunistic
                      DNSSEC=no/unsupported
    resolv.conf mode: stub
  Current DNS Server: 10.0.1.1#pfsense.<domain>.com
         DNS Servers: 10.0.1.1#pfsense.<domain>.com
Fallback DNS Servers: 127.0.0.1 10.0.1.1 9.9.9.9 1.1.1.2 1.1.1.3
          DNS Domain: <domain>.com ~.

Link 2 (eth0)
    Current Scopes: DNS
         Protocols: +DefaultRoute -LLMNR -mDNS DNSOverTLS=opportunistic
                    DNSSEC=no/unsupported
Current DNS Server: 10.0.1.1
       DNS Servers: 10.0.1.1
        DNS Domain: <domain>.com

DNS lookups however are not being blocked at the firewall level as I allow port 53 requests to pass if a DNS is specified such as with the following:

 $ drill disco.<domain>.com @1.1.1.1 
;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 54141
;; flags: qr rd ra ; QUERY: 1, ANSWER: 7, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;; disco.<domain>.com.	IN	A

;; ANSWER SECTION:
disco.<domain>.com.	300	IN	A	104.21.16.1
disco.<domain>.com.	300	IN	A	104.21.32.1
disco.<domain>.com.	300	IN	A	104.21.48.1
disco.<domain>.com.	300	IN	A	104.21.64.1
disco.<domain>.com.	300	IN	A	104.21.80.1
disco.<domain>.com.	300	IN	A	104.21.96.1
disco.<domain>.com.	300	IN	A	104.21.112.1

;; AUTHORITY SECTION:

;; ADDITIONAL SECTION:

;; Query time: 15 msec
;; SERVER: 1.1.1.1
;; WHEN: Wed Dec 18 01:35:42 2024
;; MSG SIZE  rcvd: 148

I'm not exactly sure how certbot is written but I've consulted the documentation and I don't see what I'm looking for in terms of trying to specify a DNS resolver. In comparison traefik using's LEGO's implementation for acme letsencrypt which allows the user to specify the dns resolver to use. For example:

certificatesResolvers:
  le:
    acme:
      email: <user>@<domain>.com
      #Staging Server
#      caServer: https://acme-staging-v02.api.letsencrypt.org/directory
      #Production Server
      caServer: https://acme-v02.api.letsencrypt.org/directory
      storage: /etc/letsencrypt/acme.json
      keyType: 'EC384'
      preferredChain: 'ISRG Root X1'
      dnsChallenge:
        provider: cloudflare
        delayBeforeCheck: 0
        resolvers:
          - "1.1.1.1:53"
          - "9.9.9.9:53"

Does certbot have an equivalent? I'm aware I could change the DNS resolver server at the host level, however it's kind of screwy to do this since I really only need the external DNS servers such as 1.1.1.1 or 9.9.9.9 with DNS challenge and the process of obtaining certificates.

I hope this post made some sense.

Certbot doesn't do any DNS resolving of your domains. Error states that Let's Encrypt's own resolvers cannot resolve your DNS record. As for why that happens: could be that your authoritative servers are not fully synced (LE resolver only ever try resolving ONCE using a random authoritative server of the domain that requires authorization) or something else.

But since you didn't share your domain name—help from this forum will be limited.

4 Likes

That resolvers setting does NOT modify at which nameservers the challenge will be performed in any way.

That said, the Traefik documentation for resolvers is pretty sparse and does not really say what it really does.. :roll_eyes:

3 Likes

Traefik does "preflight" checks where it looks for the TXT record before it triggers validation at the CA. Thus one can set the DNS servers it uses for this pre-validation step.

Certbot doesn't have such a preflight check by default and as such doesn't have a use case for such an option.

6 Likes

To perhaps add some clarity: Let's Encrypt will always and only query (at least one of) the authoritative nameservers for your domain. They don't use Cloudflare, or Google, or Quad9, or any other public DNS host; they step through the DNS tree to find the servers that are authoritative for your domain, and then query one or more of them (it's therefore essential that they all respond in the same way). There is no option to change this behavior, and I don't believe the CA/B requirements would permit such an option.

5 Likes

What is the TTL of the TXT record?

How long are you waiting before you trigger the challenge?

Your DNS may not have propagated yet.

For example If your DNS Provider was Cloud Flare the minimum TTL would be 1 Minute for non-enterprise customers, which means if you trigger the challenge before 1 minute there is a chance that DNS record may not be found.

Hint: The Certificate Authority failed to verify the DNS TXT records created by --dns-cloudflare. Ensure the above domains are hosted by this DNS provider, or try increasing --dns-cloudflare-propagation-seconds (currently 10 seconds).

Change it to 60 seconds (or 30 if you are an enterprise customer)

The TTL has no bearing to the Let's Encrypt authorization check since it queries the authoritative DNS servers directly.

Those authoritative DNS servers must sync world-wide after adding the TXT record. LE will check from various global points. And, this sync can take time. As you note, those are the Cloudflare suggestions for this. It just isn't TTL related.

Other DNS providers may take longer to sync. Some even have a way to check if the sync is complete (like Route53).

4 Likes

I appreciate everyone responding to this post, however what I'm gathering is a lot of explanation without a specific solution. I learn a lot about the underpinnings of DNS but still it doesn't exactly help me in this situation since the tool I'm using (certbot by virtue of the swag implementation) doesn't seem to give me a lot of flexibility.

Here are my authoritative name servers for my domain as seen by dig:

# dig SOA +trace disco.<domain>.com

; <<>> DiG 9.18.28-0ubuntu0.24.04.1-Ubuntu <<>> SOA +trace disco.<domain>.com
;; global options: +cmd
.			7033	IN	NS	c.root-servers.net.
.			7033	IN	NS	i.root-servers.net.
.			7033	IN	NS	m.root-servers.net.
.			7033	IN	NS	a.root-servers.net.
.			7033	IN	NS	g.root-servers.net.
.			7033	IN	NS	b.root-servers.net.
.			7033	IN	NS	l.root-servers.net.
.			7033	IN	NS	f.root-servers.net.
.			7033	IN	NS	h.root-servers.net.
.			7033	IN	NS	e.root-servers.net.
.			7033	IN	NS	k.root-servers.net.
.			7033	IN	NS	j.root-servers.net.
.			7033	IN	NS	d.root-servers.net.
;; Received 239 bytes from 127.0.0.53#53(127.0.0.53) in 0 ms

com.			172800	IN	NS	a.gtld-servers.net.
com.			172800	IN	NS	b.gtld-servers.net.
com.			172800	IN	NS	c.gtld-servers.net.
com.			172800	IN	NS	d.gtld-servers.net.
com.			172800	IN	NS	e.gtld-servers.net.
com.			172800	IN	NS	f.gtld-servers.net.
com.			172800	IN	NS	g.gtld-servers.net.
com.			172800	IN	NS	h.gtld-servers.net.
com.			172800	IN	NS	i.gtld-servers.net.
com.			172800	IN	NS	j.gtld-servers.net.
com.			172800	IN	NS	k.gtld-servers.net.
com.			172800	IN	NS	l.gtld-servers.net.
com.			172800	IN	NS	m.gtld-servers.net.
com.			86400	IN	DS	19718 13 2 8ACBB0CD28F41250A80A491389424D341522D946B0DA0C0291F2D3D7 71D7805A
com.			86400	IN	RRSIG	DS 8 1 86400 20241231140000 20241218130000 61050 . RzC258SXEnbXxArQSrsfWrGSMsyxDMM6K15scNr8NNBf3M7pLzXy5v2a 2PlkPnx1sygnzma2cGR5e7XJMo43hhxTqjdSRTBRCuFQxfp8K8Sx1Mk+ 9Hv6u1OOwD8ngUFWqPGb4UY4lMDzhVFjJEh9ISEuwqSuhkgF+DcMrZS6 tUMvqjclBx9cPKO4cBNVSwJe5vpPiCD++tQJS8wJ1qHvt/V6U0l27Bgz MXX+jno+LNH8cA8PdID0DHZpwt2Sf2ndRnglW9XNqncuTBfPneBSkhed tl4mrzdGyQL0fTSfNqEL/f+fJVBnUAkGdeXBnsnNFBu9pZSpU1bbwn+m yR/Jqg==
;; Received 1178 bytes from 199.7.83.42#53(l.root-servers.net) in 39 ms

<domain>.com.		172800	IN	NS	jerry.ns.cloudflare.com.
<domain>.com.		172800	IN	NS	connie.ns.cloudflare.com.
<domain>.com.		86400	IN	DS	2371 13 2 A98FB5EF85CE7C7E71B0A096E831C3607D65F69503735E162B09C744 3EABDCB4
<domain>.com.		86400	IN	RRSIG	DS 13 2 86400 20241223020540 20241216005540 29942 com. 4vEBehA+WIsfRrR8FogUPXypekIzv2EKy1a9W6GDSxVWuj2CPRQGwOBH h6SW8Q6Ifc1vdrgwWe80RP3P+ZM9bg==
;; Received 513 bytes from 192.35.51.30#53(f.gtld-servers.net) in 30 ms

<domain>.com.		1800	IN	SOA	connie.ns.cloudflare.com. dns.cloudflare.com. 2359994439 10000 2400 604800 1800
disco.<domain>.com.	1800	IN	NSEC	\000.disco.<domain>.com. A HINFO MX TXT AAAA LOC SRV NAPTR CERT SSHFP RRSIG NSEC TLSA SMIMEA HIP OPENPGPKEY SVCB HTTPS URI CAA
<domain>.com.		1800	IN	RRSIG	SOA 13 2 1800 20241219162756 20241217142756 34505 <domain>.com. Bae17e08ril5LJ9U2OR17z3kj1qhWxxHQHwljG/2vTFJQ//gerfiyWbV e7GsgJeHr3FPGcuk3MzUKC/sfmOmZQ==
disco.<domain>.com.	1800	IN	RRSIG	NSEC 13 3 1800 20241219162756 20241217142756 34505 <domain>.com. nWQ2ZauN1EIIqo3/1cbTHfNfWyGKXlIFNyijgmzpzwbpVjKxyvQrU4Rt wdjVyyylZrw7DBqrxPd1J4r2ITQNfA==
;; Received 372 bytes from 108.162.193.182#53(jerry.ns.cloudflare.com) in 7 ms

I remembering corresponding with @ldez who is maintainer of the LEGO implementation about similar issue with the lego variant in the past and he's walked me down the path of troubleshooting and querying SOA records. In this particular incident however the authoritative name servers seem to be found rather easily. Other than use a different tool I'm not exactly sure here what to do on my end to make this go.

Did you try increasing that? Try 60 seconds (at least)

2 Likes

I'm going to have to see how I can do this with the swag container as I don't run certbot directly from CLI. I'll reach out to the linuxserver.io people.

***Edit -- actually that was pretty easy to do myself making the changes within cloudflare.ini file

1 Like

You can add it to the config file that includes your credentials.

https://certbot-dns-cloudflare.readthedocs.io/en/stable/

3 Likes

We cross-posted :slight_smile: Did that resolve the problem?

3 Likes

Can you explain what you are trying to do with a use-case?

I noticed this bit, showing an IP in a reserved netblock.

As @danb35 wrote, LetsEncrypt will use the authoritative name servers; additionally, the DNS servers they point to need to be publicly accessible.

The only use cases I can imagine this being for are:

  • A preflight test, as suggested by @Nummer378
  • An internal routing requirement, wherein you need to update the DNS servers via a local IP (knowing they will be interrogated by the public IP)
5 Likes

My suspicion is that is interned to be treated as redacted. :man_shrugging:

2 Likes

I not really trying anything to complex. I have all my DNS requests on my LAN funneled to my pfSense DNS resolver. This setup breaks however when requests needed external DNS resolvers, so I've selectedly punched holes in the firewall to allow DNS requests out from servers (IP addresses) that need to obtain LE certs -- however all the LE certs are used on internal infrastructure -- so hence the need to used pfsense unbound as the resolver where I manually list the DNS overrrides. I don't think this setup seems very complex or difficult

@MikeMcQ -- will try since I reached the LE limits for the day. I could have switched to ZeroSSL or some other provider, however I just got caught up with other things.

I usually use traefik for my Reverse proxy which utilizes lego, however in this particularly instance I was dealing with some specific headers that needed to be in the request and the examples I was working with were using nginx. I suppose I could rework things and rather than using swag/certbot, I could just replace with straight nginx and obtain certs through either traefik/certs dumper, or just use acme.sh or something similar. Dealing with custom headers with traefik is sometimes really problematic as I'm not versed with troubleshooting these issues that well, unfortunately many header modifications just give nginx examples and so I was just trying to cut down on the amount of work.

1 Like

I just realized I am confusing internal propagation with TTL

Some even have a way to check if the sync is complete (like Route53).

Going to have to learn this shortly, probably.


Why did it suggest increasing --dns-cloudflare-propagation-seconds

The Cloudflare documentation says this should be 60 seconds unless you are an enterprise customer, then it should be 30 seconds.

10 seconds is definitely too low.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.