Persistent NXDOMAIN on dns-01 challenge despite verified, consistent authoritative answers

Help
1

I'm trying to issue a certificate for fahrschuleo.at with SANs including
*.pdf.fahrschuleo.at (delegated subdomain, own NS records) via
certbot --manual with a custom DNS-01 hook (staging environment for
testing). The challenge for pdf.fahrschuleo.at consistently fails with:

Domain: pdf.fahrschuleo.at
Type: dns
Detail: DNS problem: NXDOMAIN looking up TXT for
_acme-challenge.pdf.fahrschuleo.at - check that a DNS record exists
for this domain

What I've already verified before every attempt:

  1. Authoritative NS for pdf.fahrschuleo.at (both from parent zone and
    directly): ns2.datamagic.at and nspdf.fahrschuleo.at - consistent
    everywhere I checked (dig +trace, dig NS from multiple vantage
    points).

  2. Before calling certbot, my hook confirms via direct dig queries to
    BOTH authoritative NS (not a caching recursive resolver) that the
    TXT record is present and correct on both servers - this check
    passes every time before certbot proceeds to validation.

  3. I watched the record on both NS live (side-by-side, 1-second
    polling) throughout an entire failed attempt - both servers were
    consistently in sync the whole time, never diverged, appeared and
    disappeared at the same moment.

  4. tcpdump on BOTH authoritative NS simultaneously, filtered for port
    53 traffic related to pdf.fahrschuleo.at, during a full failed
    attempt: the ONLY TXT queries for _acme-challenge.pdf.fahrschuleo.at
    visible in either capture came from my own server (ns2, running the
    hook script itself) - zero queries from any external IP address
    during the entire challenge window.

  5. Confirmed nspdf.fahrschuleo.at (80.109.128.50) is reachable from an
    independent external server (different datacenter/network) on port
    53 and returns correct authoritative NXDOMAIN answers with the aa
    flag set for non-existent records.

  6. DNSSEC/CAA checked via letsdebug.net (dns-01 method): "All OK, no
    issues found".

  7. Ruled out: firewall (ufw explicitly allows port 53 from anywhere on
    both NS, IPv4 and IPv6), zone-transfer sync lag between the two NS,
    and a previous unrelated bug in my own DNS-record API that has since
    been fixed and confirmed working correctly (verified via repeated
    manual POST/DELETE + immediate dig checks, 10/10 consistent).

Given that my own captures show Let's Encrypt's validator never even
sent a query to either authoritative NS during the failed attempts,
I suspect the issue may be on Let's Encrypt's side (or somewhere in
network path specific to reaching these validators), rather than in my
DNS setup - but I have no way to confirm what IP/vantage point your
validators actually used, or whether they attempted a query at all.

Could someone please check the validation logs for a recent attempt
against pdf.fahrschuleo.at (or fahrschuleo.at, staging environment) and
let me know what your validators actually observed? Happy to provide
exact timestamps of failed attempts if useful.

Domain: fahrschuleo.at / pdf.fahrschuleo.at
Environment: staging (acme-staging-v02.api.letsencrypt.org)
Client: certbot 1.21.0, --manual with custom auth/cleanup hooks

2

Thanks for the detailed report. I don't have the time right now to do a detailed analysis of your DNS, but DNSViz reports two issues for fahrschuleo.at:

  1. ns3.datamagic.at (2a01:5740:1000:3300::52) doesn't seem to be responding at all.
  2. ns4.datamagic.at is in the authoritative NS RRset, but not in the delegation NS RRset.

I'm not sure if those might be unrelated to your problem entirely, but I'd first recommend double-checking the delegation records all the way from the root up, making sure that each step has consistent and correct NS records.

3

Definitely check out the DNSViz report @petercooperjr showed and the two key issues reported there. Some additional info:

You said several times you checked "both" authoritative servers but there are 5 defined (although not fully correct as DNSViz reports).

The reason I mention this is because Let's Encrypt will do a random walk down the authoritative tree. Any path it chooses must provide a correct response. It will not hunt around looking for a single path with a correct response. That is contrary to the security requirements of checking from multiple validation sites.

So, only checking two of your DNS servers may not show LE attempts.

The error is saying NXDOMAIN. That is an explicit reply from your DNS servers. It is not something that results from a timeout trying to reach them. We know from this that LE sent you validation requests.

The current version of Certbot is 5.7.0. You should review your install method to stay current. I'm guessing you installed from an package manager with a slightly dated o/s. You should look at the EFF's site for instructions for snap or pip installs. https://certbot.eff.org

This isn't causing this problem but a newer version of Certbot has considerable advantages including support for ARI and Let's Encrypt Profiles. ARI especially is very important.

4 
nslookup -q=ns fahrschuleo.at
fahrschuleo.at nameserver = ns4.datamagic.at
fahrschuleo.at nameserver = ns3.datamagic.at
fahrschuleo.at nameserver = ns2.datamagic.at
fahrschuleo.at nameserver = ns1.datamagic.at

nslookup -q=ns pdf.fahrschuleo.at
pdf.fahrschuleo.at nameserver = nspdf.fahrschuleo.at
pdf.fahrschuleo.at nameserver = ns2.datamagic.at

nslookup -q=ns _acme-challenge.pdf.fahrschuleo.at ns1.datamagic.at
Server:  ns1.datamagic.at
Address: 45.90.24.24
*** ns1.datamagic.at can't find _acme-challenge.pdf.fahrschuleo.at: Non-existent domain

^ I'd expect a nameserver reply of:
pdf.fahrschuleo.at nameserver = nspdf.fahrschuleo.at
pdf.fahrschuleo.at nameserver = ns2.datamagic.at

Like for this subdomain:

nslookup -q=ns pdf.fahrschuleo.at ns1.datamagic.at
Server:  ns1.datamagic.at
Address: 45.90.24.24
pdf.fahrschuleo.at nameserver = ns2.datamagic.at
pdf.fahrschuleo.at nameserver = nspdf.fahrschuleo.at
5

to add to what rg305 said :

nmap shows 80.109.128.50:53 open on udp/tcp

ns1.datamagic.at DNS is down, host is up

you have no root authority server for the root domain fahrschuleo.at, yet youre setting up subdomains

dig fahrschuleo.at A +nsid | grep AUTH
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

ns2-4.datamagic.at say no answer for pdf.fahrschuleo.at, and 83.218.161.136 for fahrschuleo.at

rdns for 83.218.161.136 points to www.internetkonzepte.at

you do have a root authority server for pdf.fahrschuleo.at, yet it wont answer for its own domain

dig pdf.fahrschuleo.at A +nsid |grep -A1 'AUTHORITY SECTION'
;; AUTHORITY SECTION:
pdf.fahrschuleo.at. 2252 IN SOA nspdf.fahrschuleo.at. root.nspdf.fahrschuleo.at. 2026082315 10800 3600 2160000 3600

your own dns server nspdf.fahrschuleo.at @80.109.128.50 doesnt resolve its own domain or subdomains

nslookup -type=a pdf.fahrschuleo.at ns4.datamagic.at
Server: ns4.datamagic.at
Address: 5.75.230.121#53

*** Can't find pdf.fahrschuleo.at: No answer

nslookup -type=a pdf.fahrschuleo.at nspdf.fahrschuleo.at
Server: nspdf.fahrschuleo.at
Address: 80.109.128.50#53

*** Can't find pdf.fahrschuleo.at: No answer

nslookup -type=a nspdf.fahrschuleo.at nspdf.fahrschuleo.at
Server: nspdf.fahrschuleo.at
Address: 80.109.128.50#53

** server can't find nspdf.fahrschuleo.at: REFUSED

nslookup -type=a fahrschuleo.at nspdf.fahrschuleo.at
Server: nspdf.fahrschuleo.at
Address: 80.109.128.50#53

** server can't find fahrschuleo.at: REFUSED

your authoritive ns not answering for its own domains/subdomains is what id fix first.
and its failing because ns2-4.datamagic.at and nspdf.fahrschuleo.at have no A record for pdf.fahrschuleo.at or refuse to answer. and also doublecheck what petercooperjr said about checkign the delegation records all the way to root.

6

Thanks for the reply, after I added ns1, ns3 and ns4 to the pdf.fahrschuleo.at zone file it worked perfectly.

Successfully received certificate.
Certificate is saved at: /etc/letsencrypt/live/fahrschuleo.at/fullchain.pem
Key is saved at: /etc/letsencrypt/live/fahrschuleo.at/privkey.pem
This certificate expires on 2026-11-22.