I've successfully set up a custom certbot configuration and due to specifics of my certificate (several domains, some of which also should contain wildcards) I have to use DNS authentication since it is the only available option.
And here s the caveat: I create the needed domain through my server's, API, and I check that domain is accessible through some DNS server, but certbot anyway shows an error "DNS resolution error" or "wrong challenge domain".
I think it's because of the DNS latency. I set TTL in my challenge record to the smallest available value 30 seconds, and introduced a delay of 15 minutes between creation of a domain and exit of the authenticator script, and anyway still often get those errors.
So my question is: what should be my strategy to ensure DNS record is already propagated to the DNS server Letsencrypt use to resolve challenge domains?
Should wait for (documentation suggest an hour!) in the authenticator script, or it is better to run the script once without the delay, and then re-run it once again after an hour passed?
You will probably need to wait for more of a DNS expert than me.
But, when checking the TXT record for a DNS challenge the Let's Encrypt servers check the authoritative name servers. Typically the delay is short (often just seconds) for the DNS provider to sync those. You can use the unboundtest.com test site which mimics what LE servers do to lookup TXT records.
Knowing your DNS provider and an example domain name is very helpful when trying to assist with such questions. And, are you using one of the DNS challenge plug-ins or is that part of your custom scripting too?
Several providers are notorious for using read-through caches on their public servers, but not implementing a write-through cache on their internal API. That means edits you make on their systems can usually take anywhere from 301 seconds (5minutes + 1 second) to 3601 seconds (60 minutes + 1 second) to clear [longer and shorter are possible, those are just the most typical settings]. These are application/infrastructure caches, and not related to TTL settings – so customers have no control over this behavior aside from switching providers.
SO, I can see I can use GET request with unboundtest.com and it gives me the results in a text format (is it like a dig?)
So is the format documented, should I parse it somehow or there are some more reliable way to get the content of TXT records?
So AFAICS the problem is that two Yandex DNS servers are out of sync? So can I for example query both of them and do not exit the script until both servers answer the correct record for, say, three times in a row with some delay?
You can try such ideas but I was getting the wrong response from their authoritative dns servers even after a long delay. The dns1 was wrong about 1/3 of the time and the other only about 1/10.
After such a wait you might still get a failure when Let's Encrypt checks them. So, just start renewing 30 days before expiry and try several times a day. Many attempts may fail but you only need one to work in the 30 day period.
A certbot renew does very little for "fresh enough" certs. Certbot just looks at your local cert and if it is fresh it does nothing else. Just be sure not to use "force" option - that won't help and is actually worse.
You could try switching DNS providers too. Ideally to one with a support API for DNS challenge
In my book, any reason is good enough to switch.
DNS is extremely simple - its's either done right or it isn't.
At least consider adding more DNS servers into the equation.
[more hits and less misses on any given test]