DNS-01 challenge weird behaviour

I have developed a custom DNS server using coreDNS+GRPC backend and Postgresql as the database.

Also, I've written a webhook using the webhook-example and my present section looks like this:

func (c *customDNSProviderSolver) Present(ch *v1alpha1.ChallengeRequest) error {
	cfg, err := loadConfig(ch.Config)
	if err != nil {
		return err
	}
	pattern := `(?:\*\.)?(.*)`
	re := regexp.MustCompile(pattern)
	matchDomain := re.FindStringSubmatch(ch.DNSName)
	if len(matchDomain) <= 1 {
		return errors.New("Invalid DNS name.")
	}
	domain := matchDomain[1]

	domainRecord, err := dnsDomainRepository.FindByName(domain)
	if err != nil {
		return err
	}
	if domainRecord == nil {
		return errors.New("The domain not found.")
	}
	if domainRecord.Status != "ACTIVE" {
		return errors.New("The domain is not active")
	}

	dnsRecordRepository.Create(&repositories.Record{
		DomainId: domainRecord.ID,
		Name:     util.UnFqdn(ch.ResolvedFQDN),
		Content:  ch.Key,
		Type:     "TXT",
		TTL:      60,
	})
	return nil
}

It inserts the TXT record into my database.
And dig TXT _acme-challenge.example.org shows a correct TXT record.
But the CoreDNS log shows the dns queries are trying for the CNAME _acme-challenge, not the TXT.

Why?

My Database Log:

SELECT "record"."id", "record"."name", "record"."type", "record"."content", "record"."ttl", "record"."createdAt", "record"."updatedAt", "record"."domainId", "domain"."id" AS "domain.id", "domain"."name" AS "domain.name", "domain"."namespace" AS "domain.namespace", "domain"."status" AS "domain.status" FROM "records" AS "record" INNER JOIN "domains" AS "domain" ON "record"."domainId" = "domain"."id" AND "domain"."status" = 'ACTIVE' WHERE ("record"."name" = '_acme-challenge.example.org' OR "record"."name" = '*.example.org ') AND "record"."type" = 'CNAME'

I'm not familiar with the specific technologies you're using, but I know there are a lot of nuances to implementing a custom DNS server, not just around the positive success cases but also in returning the correct error (or success-but-no-records) when a record isn't found. And in general some system somewhere needs to make sure that the record isn't being redirected via a CNAME instead of being a normal response, though I don't know where in your DNS server software that check would take place.

Is your system running somewhere on a real domain name, so that people can try their own digging or use tools like DNSViz, ISC EDNS Compliance, and Unboundtest on it?

5 Likes

Also, I don't know if this was just a problem with trying to redact real data, but "example" is spelled wrong here, and the space between the name and the end-of-string ' seems a little weird too.

3 Likes

It's just an unrelated typo.
The (Automatic) Challenge doesn't query for a TXT record, It looks for a CNAME record. This is my problem.

Can you clarify why looking for a CNAME would be a problem? Shouldn't your system just report that there aren't any CNAME records, just like if it got a query for any other record type that the name doesn't have?

4 Likes

The dns01 challenge documentation says _acme-challenge is a TXT record.

It also says this
β€œ Since Let’s Encrypt follows the DNS standards when looking up TXT records for DNS-01 validation, you can use CNAME records or NS records to delegate answering the challenge to other DNS zones. This can be used to delegate the _acme-challengesubdomain to a validation-specific server or zone. It can also be used if your DNS provider is slow to update, and you want to delegate to a quicker-updating server.”

3 Likes

Yes, But I've inserted a TXT record there.

Also I haven't found the content of the CNAME record in cert-manager documentation.
All other hooks only uses TXT records. e.g. godaddy webhook

The actual Domain Name would helpfully so we can check its DNS.

2 Likes
dig TXT _acme-challenge.gomak.ir
2 Likes

So a DNS server, which has a TXT record and no other resource records for a name, would respond NOERROR (and no records) when queried for other resource record types, and with the TXT record when queried for TXT.

I feel like there's a disconnect between whatever logging or code you're looking at, and the way the DNS protocol expects things to work.

4 Likes

Here Hardenize Report: gomak.ir is showing DNS Zone errors

And here https://dnsspy.io/scan/gomak.ir

2 Likes

Yeah, you're going to want to fix your delegation and glue records, and make sure the DNS servers respond over TCP, before trying to debug anything else relating to the domain name.

https://dnsviz.net/d/_acme-challenge.gomak.ir/dnssec/

4 Likes

Yes, it's a dev environment, But it responds TXT records.

Shouldn't the Challenge Solver query for TXT records?
The cert-manager querying my DNS Server only with CNAME records.

[INFO] 10.42.0.1:5330 - 14616 "CNAME IN _acme-challenge.gomak.ir. udp 53 true 1400" NOERROR qr,aa 42 0.01282394s
[INFO] 10.42.0.1:39903 - 51947 "CNAME IN _acme-challenge.gomak.ir. udp 53 true 1400" NOERROR qr,aa 42 0.053124772s
[INFO] 10.42.0.1:48190 - 21117 "CNAME IN _acme-challenge.gomak.ir. udp 53 true 1400" NOERROR qr,aa 42 0.027537558s
[INFO] 10.42.0.1:1828 - 58442 "CNAME IN _acme-challenge.gomak.ir. udp 53 true 1400" NOERROR qr,aa 42 0.138318724s
[INFO] 10.42.0.1:37200 - 49764 "CNAME IN _acme-challenge.gomak.ir. udp 53 true 1400" NOERROR qr,aa 42 0.015863975s

Maybe the coreDNS community could be of assistance as well.

2 Likes

Regardless, you are requesting for a production cert and must abide by all those requirements.

Yes, but only locally AND only via UDP.
It should also respond globally and via TCP.

4 Likes

But it's working using certbot

sudo certbot -d *.gomak.ir -m vahid.lid@live.com --manual-public-ip-logging-ok --agree-tos --manual --preferred-challenges dns certonly
Use of --manual-public-ip-logging-ok is deprecated.
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Requesting a certificate for *.gomak.ir

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Please deploy a DNS TXT record under the name:

_acme-challenge.gomak.ir.

with the following value:

DjalLDFRs1CHAl7pYHeF7MCOvLDwRuyXsCMhPq2-Bv0

Before continuing, verify the TXT record has been deployed. Depending on the DNS
provider, this may take some time, from a few seconds to multiple minutes. You can
check if it has finished deploying with aid of online tools, such as the Google
Admin Toolbox: https://toolbox.googleapps.com/apps/dig/#TXT/_acme-challenge.gomak.ir.
Look for one or more bolded line(s) below the line ';ANSWER'. It should show the
value(s) you've just added.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Press Enter to Continue

Successfully received certificate.
Certificate is saved at: /etc/letsencrypt/live/gomak.ir/fullchain.pem
Key is saved at:         /etc/letsencrypt/live/gomak.ir/privkey.pem
This certificate expires on 2024-11-03.
These files will be updated when the certificate renews.

NEXT STEPS:
- This certificate will not be renewed automatically. Autorenewal of --manual certificates requires the use of an authentication hook script (--manual-auth-hook) but one was not provided. To renew this certificate, repeat this same certbot command before the certificate's expiry date.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
If you like Certbot, please consider supporting our work by:
 * Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
 * Donating to EFF:                    https://eff.org/donate-le
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

It seems there's a mistake or overlook in cert-manager

Some ACME clients do a check of their own, to help diagnose issues, before asking the CA's servers to validate. It may be that cert-manager is doing its own checks for a CNAME, and that's what you're seeing in your logs. If that's the case, you can probably configure cert-manager not to do so, but I suspect that if your DNS server follows the DNS specs and responds correctly for the query then it would work fine as is.

You haven't really said anything "broken" yet, you just seem confused about why you're seeing CNAME queries in your logs.

6 Likes

I concur with @petercooperjr and can confirm from my own experience that cert-manager (locally) performs its own prechecks.

6 Likes

I have fixed my DNS server problems and all are green.
The issue is related to the cert-manager.
Thanks for your reply. ( Most of the others were unrelated!)

1 Like