Impossible to know when TXT records have propagated

Help
1

My domain is:
vpn.valmarc.com
This is a host that runs our internal corporate VPN. There is no HTTP web server there. Port 80 is completely inaccessible from the outside world.

I ran this command:
certbot certonly --dry-run -d vpn.valmarc.com --manual --preferred-challenges dns

It produced this output:
Please deploy a DNS TXT record under the name:
_acme-challenge.vpn.valmarc.com.
with the following value:
z3qxT8CYdPbrR2aiIhOSrVy6G2xirFOQxMIBcoruDdY
Press Enter to Continue

My web server is (include version):
N/A

The operating system my web server runs on is (include version):
Ubuntu 24.04

My hosting provider, if applicable, is:
There is no HTTP web site. I run my own bind9 DNS servers for the domain on Ubuntu 24.04

I can login to a root shell on my machine (yes or no, or I don't know):
Yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):
No

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):
1.21.0

I run my own bind9 DNS servers on Ubuntu. I add the DNS record as follows:
_acme-challenge.vpn.valmarc.com. IN TXT "z3qxT8CYdPbrR2aiIhOSrVy6G2xirFOQxMIBcoruDdY"

I also update the SOA serial number, and then run "rndc reload" to push the changes. The advice from the certbot application is to check the following URL to wait for propagation of the TXT record:
Dig (DNS lookup).

I open the above URL in a web browser and start clicking "refresh" every 10 seconds or so. Eventually I see the TXT record show up. However, if I refresh again, the TXT record shows an old value from earlier today. Repeatedly clicking refresh in my browser makes the TXT record alternate between the old value (from an hour ago) and the new value.

If I go back to the certbot command-line window and press to continue, then it may or may not work. Sometimes it works because the TXT record is updated as shown in the googleapps.com site. Sometimes it fails because the TXT record still has an old value from an hour ago.

It is impossible to know when the TXT record is fully propagated. It seems like it is "partially" propagated and there are some servers that have the new value and some servers that still have to old value. There seems to be no way to know when all the servers at Google have the new value so I can guarantee success. I suppose I could wait numerous hours, but this seems silly.

I eventually want to automate this in a script using certbot's --manual-auth-hook and --manual-cleanup-hook. But there's just no way to know how long it will take to propagate. Is there a way to get the letsencrypt.com servers to do an authoritative DNS query for the TXT record, so that it goes directly to my bind9 DNS servers to fetch the value?

2

@idcipher, welcome to the community! :slightly_smiling_face:

I see that the serial does not match on the two servers. Check the log for the DNS zone propagation from the master to the slave DNS server. If properly implemented with DNSnotify, it should be instantaneous.

4 Likes
3

Let's Encrypt uses Unbound and checks the authoritative server. They do not use a public resolver.

Once all your servers are in sync you can immediately pass the challenge

5 Likes
4

If you're using bind and certbot, you may want to explore using the rfc2136 plugin, which is the closest thing there is to a standard for having automated updates to your DNS record.

But regardless, you'd need to have your bind servers configured to distribute those updates across all your authoritative servers. Thinking about "propagation" across public servers with different caching policies can overcomplicate things, it's just that you need to make sure that no matter which of your domain's authoritative servers gets asked, the correct response is returned.

3 Likes
5

Thanks everyone for the replies-- that was indeed the problem. It looks like I never configured the Bind9 server with an "also-notify" option. So the master would take its sweet old time allowing the slave to update itself. Once I added the also-notify option in named.conf.options, then it worked perfectly.

Now that I've generated a certificate with certbot, it claims the following in the output:

Certbot has set up a scheduled task to automatically renew this certificate in the background.

What exactly is it going to do though? When it tries to renew, is it going to re-run my manual hook scripts that I specified with the --manual-auth-hook and --manual-cleanup-hook options (which modify the Bind9 configuration and run 'rndc reload')?

What if I want to move the hook scripts to a different location? When I generated the certificate, I just had the scripts in a test location, but I want to move them to /usr/local/bin. If I do that, I assume this 'automatic renewal' process is not going to find my scripts any more?

6

Glad to hear you got it working!

Yes, it has saved the configuration that was used to get the certificate, and will use it for renewals. By default it checks a couple times a day to see if the certificate needs to be renewed. (In addition to when the certificate is close to expiring, it might also need to be renewed if the CA messed something up and needs to have you replace the certificate early, so that's why it's helpful for it to regularly check so often.)

You can change the saved configuration for certbot to use to get certificates with the certbot reconfigure command.

3 Likes