DNS Servers used by LetsEncrypt for Challenges

I am starting a small SaaS company that uses a check for DNS record before creating service. Letsencrypt cert happens during service creation. It would be best to ensure that if our check succeeds, letsencrypt will succeed in finding this domain record as well. This should be easily solvable if letsencrypt uses a specific primary secondary nameserver. If we use the same servers, I can directly query this server for the record.

Which DNS IP does letsencrypt use for http challenge?

Google DNS Servers are used.

You can check manually here: https://dns.google.com/

You can also specify the server to use with dig http://droptips.com/using-dig-to-query-a-specific-dns-server-name-server-directly-linux-bsd-osx

Andrei

Google’s DNS is evil, not using that for my servers, but I will have it coded on our site to dig for it.

Thanks for the reply.

Are you sure? I thought Let's Encrypt always queried the authoritative servers for the domain in question.

3 Likes

I kept seeing this in relation to the DNS challenge, not http. Will wait for other parties to clarify.

I agree with @danb35’s answer: Let’s Encrypt doesn’t use anyone else’s recursive resolver but goes right to the source (making its own queries to the authoritative nameservers for the particular domain in question).

3 Likes

makes sense - apologies for confusing people

@schoen if there are multuple name servers are all checked by LetsEncrypt or does is it just the first name server?

Would love to get verification as to what DNS server it connects to in order to verify the TXT record.
I am getting this error:

Failed authorization procedure. 45woodburn.faure.ca (dns-01): urn:acme:error:connection :: The server could not connect to the client to verify the domain :: DNS problem: NXDOMAIN looking up TXT for _acme-challenge.45woodburn.faure.ca, cloud.45woodburn.faure.ca (dns-01): urn:acme:error:connection :: The server could not connect to the client to verify the domain :: DNS problem: NXDOMAIN looking up TXT for _acme-challenge.cloud.45woodburn.faure.ca

Partly looks like an error "connecting" to the DNS erver, partly looks like a missing DNS entry. Very confusing that it says "The server could not connect to the client to verify the domain". I sincerely hope it does not try to connect to the client running the script (instead of the authoritative DNS server), which would be a big security hole.

My DNS entry is valid and available:

dig TXT _acme-challenge.45woodburn.faure.ca @ns.certainkey.ca

_acme-challenge.45woodburn.faure.ca. 900 IN TXT "gZ-Aj6S3XRB73AIBEmaTdzhtBjWdihMDbSiAUSoU-9g"

certbot and my hook scripts are running on a different box than my primary DNS server.
Another issue may be a propagation one... i have 6 NS servers, but only one of them will be updated within seconds. Any way I can specify which of the 6 servers listed in the "whois record" that certbot should use?

Thanks for your help.
Paul

It sends a query to one of your authoritative DNS servers, choosing in some typical DNS way; i don't know which one. (Arbitrary, randomly, or possibly something influenced by latency or reliability.)

Nah, it's just a confusing error message. I would guess that "The server could not connect to the client to verify the domain" is just a boilerplate error message that dates to before Let's Encrypt supported DNS-01 challenges. It's not actually trying to connect to your client. Ignore that part and focus on "DNS problem: NXDOMAIN looking up TXT for _acme-challenge.cloud.45woodburn.faure.ca".

That's absolutely a problem. It means Let's Encrypt has about a 5/6 chance of getting an NXDOMAIN and failing!

Indeed, let alone a few seconds, some or all of your other 5 DNS servers still aren't serving that TXT record. They're also showing a different SOA serial number:

$ digr +short faure.ca soa @ns.certainkey.ca
ns.certainkey.ca. hostmaster.faure.ca. 2017041609 86400 3600 604800 3600
$ digr +short faure.ca soa @ns1.linode.com
ns.certainkey.ca. hostmaster.faure.ca. 2016102567 86400 3600 604800 3600

(Though there may be a degree of caching.)

It should be possible to update Linode's nameservers within a couple seconds. (Though, again, there may be a degree of caching.) Is ns.certainkey.ca failing to NOTIFY axfr1.linode.com -
axfr5.linode.com?

Through standard DNS mechanisms, yes. You could make _acme-challenge.cloud.45woodburn.faure.ca. a separate zone delegated only to ns.certainkey.ca.. You could make a separate zone with a different name, e.g. acme-challenges.faure.ca., make _acme-challenge.cloud.45woodburn.faure.ca. a CNAME record to cloud.45woodburn.acme-challenges.faure.ca. or somesuch and update that record.

The current setup should be fine if you're able to ensure that all 6 nameservers are updated before Let's Encrypt attempts to validate it, though.

Edit: Typo, "challegnes".

One is picked randomly from the set of authoritative servers. Edit: updated to clarify, we pick a recursive resolver from a set that we run in our environment with a low max TTL (60s presently) at random. This is what the code I linked is responsible for.

The recursive resolver (Unbound in this case) picks an authoritative nameserver based on the delegations in place in the relevant zones. I believe this determination is made at random initially and then later by latency/reliability but I'm not positive.

1 Like

Is there any technical foundation for this assertion about Google DNS?

Yes, there is. Google gets to see your queries in plaintext. That’s it.

Well, I tried a few options to force a sync of the changes with no success. I have attempted the final suggestion to setup a separate zone, but I have to wait 24 hours for linode to update to the latest serial.

I am having troubles getting it going. The error that I am getting is:

"Failed authorization procedure. cloud.45woodburn.faure.ca (dns-01): urn:acme:error:connection :: The > server could not connect to the client to verify the domain :: DNS problem: NXDOMAIN looking up TXT > for _acme-challenge.cloud.45woodburn.faure.ca"

I set it up like mnordhoff sugested, separate zone, removed slave DNS and set the cacheing to 0.
The new zone looks like this:

$ORIGIN .
$TTL 300 ; 5 minutes
acme-challenges.faure.ca IN SOA ns.certainkey.ca. hostmaster.faure.ca. (
2017042016 ; serial
300 ; refresh (5 minutes)
30 ; retry (30 seconds)
86400 ; expire (1 day)
30 ; minimum (30 seconds)
)
NS ns.certainkey.ca.
A 127.0.0.1
$ORIGIN acme-challenges.faure.ca.
$TTL 0 ; 0 seconds
45woodburn TXT "a6pg7ndeq5uztyHOan6lE6gxz-J_lYzCLn45CVPppOg"
$ORIGIN 45woodburn.acme-challenges.faure.ca.
cloud TXT "ZShpLr0g4osQ0B0AlJMkN_2L73OLu92NWhdbn7P9G1g"

And the original zone with the CNAME looks like this:

$ORIGIN 45woodburn.faure.ca.
$TTL 0 ; 0 minutes
_acme-challenge CNAME 45woodburn.acme-challenges.faure.ca.
$ORIGIN cloud.45woodburn.faure.ca.
_acme-challenge CNAME cloud.45woodburn.acme-challenges.faure.ca.

I can resolve the TXT record no problem:

dig TXT _acme-challenge.45woodburn.faure.ca
;; ANSWER SECTION:
_acme-challenge.45woodburn.faure.ca. 0 IN CNAME 45woodburn.acme-challenges.faure.ca.
45woodburn.acme-challenges.faure.ca. 0 IN TXT "a6pg7ndeq5uztyHOan6lE6gxz-J_lYzCLn45CVPppOg"

Am I missing something? Is there still a cache issue?
I am looking at my query.log entries and I can't see anything coming in from letsencrypt.
How can I debug this further?

The Linode nameservers don’t have the acme-challenges.faure.ca delegation, so 5/6 of your nameservers will result in NXDOMAIN.

$ digr acme-challenges.faure.ca @ns1.linode.com

; <<>> DiG 9.10.3-P4-Ubuntu <<>> +norecurse acme-challenges.faure.ca @ns1.linode.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 4325
;; flags: qr aa; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;acme-challenges.faure.ca.      IN      A

;; AUTHORITY SECTION:
faure.ca.               3600    IN      SOA     ns.certainkey.ca. hostmaster.faure.ca. 2017042001 86400 3600 604800 3600

;; Query time: 1 msec
;; SERVER: 2400:cb00:2049:1::a29f:1a63#53(2400:cb00:2049:1::a29f:1a63)
;; WHEN: Sat Apr 22 05:04:47 UTC 2017
;; MSG SIZE  rcvd: 114

The Linode nameservers also need an NS record for acme-challenges in the faure.ca zone so resolvers will reliably be able to find the zone.

1 Like

I do know that when updating a domain via Linode's control panel, it waits until the next quarter hour to move the changes to the published info (essentially, to ns1.linode.com), and my experience is that it usually takes a couple of minutes before that propagates to ns2-ns5. I'm not sure if it's any different when updating via the API, but I doubt it.

If Linode's running as secondary servers, they pull changes within a second or two of receiving a NOTIFY. It's not subject to the quarter-hour delay.

That did it :slight_smile: Great stuff mnordohoff. Thanks for all your help.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.