Automate renew using certbot with dns-01 for firewalled host

Help
1

Hi,

TL;DR; is it possible to use certbot with dns-01 authentication on one computer (a Jenkins slave) to renew a certificate for a publicly available name, where the certificate will be copied to another computer (an ESXi server) that isn’t able to run certbot and doesn’t have any inbound or outbound internet connectivity?

I’m attempting to automate the process of renewing my certificate:

  • certbot runs in manual mode with various flags to stop interactive behaviour.
  • certbot calls an auth-hook to generate the necessary TXT record with my DNS provider (CloudFlare)
  • certbot performs its magic
  • certbot calls a cleanup hook that will remove any temporary files and the TXT record from DNS.
  • cerbot does all this inside a docker container because magical unicorns.

I’m using a dns-based challenge because the host that will receive the SSL certificate is firewalled. Further, the machine in question can’t run the certbot client directly (or the others) because it has a limited/stripped operating system (ESXi).

docker run -v $(pwd):/letsencrypt-scripts certbot/certbot certonly \
  -n \
  --email me@mydomain.net \
  --agree-tos \
  --manual-public-ip-logging-ok \
  --manual \
  --preferred-challenges=dns \
  --manual-auth-hook /letsencrypt-scripts/letsencrypt-auth.sh \
  --manual-cleanup-hook /letsencrypt-scripts/letsencrypt-cleanup.sh \
  -d hostname.mydomain.net

I get the following error though:

Failed authorization procedure. liquistate.net (dns-01): urn:acme:error:connection :: The server could not connect to the client to verify the domain :: DNS problem: NXDOMAIN looking up TXT for _acme-challenge.hostname.mydomain.net

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: hostname.mydomain.net
   Type:   connection
   Detail: DNS problem: NXDOMAIN looking up TXT for
   _acme-challenge.hostname.mydomain.net

Can anyone elaborate on that error - server could not connect to the client to verify the domain - who is the server and client in this scenario? And what type of connection is it talking about? Ping? HTTP?

There is some useful advice given:

  • make sure that your domain name was entered correctly and the DNS A/AAAA record(s) for that domain contain(s) the right IP address. - its the right name, and has the correct (firewalled) IP address.
  • check that your computer has a publicly routable IP address - which computer? The computer I’m using to make the request is NAT’d, so has outbound internet connectivity, but not inbound. The machine that will be using the certificate has no outbound or inbound internet connectivity (hence why I want to use dns-01 auth and not web-based).

Any and all help warmly received!

2

It's a misleading error message category that was written before Let's Encrypt implemented DNS-01. The critical part is the error itself:

Let's Encrypt queries your domain's DNS servers, and the necessary TXT record doesn't exist.

You're right. It's general advice that doesn't really apply when using DNS-01.

Could you provide more information? Are you sure the TXT record is being created with the right name? How does the script work?

Does it wait after the API call to create the DNS record? Cloudflare usually deploys DNS changes in like a second, but not always. Maybe everything's right, but it's going too fast, and asking Let's Encrypt to check the challenge before Cloudflare is ready.

Maybe the domain is actually using a different DNS provider! (letsencrypt.log will confirm that, actually.)

3

I am seeing the same issue. I am nearly convinced that this is a certbot issue. Have a look at this post.

In my case it’s working for the root domain, but not subdomains.

Can you try it for “mydomain.net”? If it is the same problem as mine I think it will work fine.

4

you should really read the full error message

NXDOMAIN looking up TXT for _acme-challenge.hostname.mydomain.net

Note: currently certbot does not implement the TXT records automatically

I see you are running two scripts are these for implementing a DNS TXT record addition?

--manual-auth-hook /letsencrypt-scripts/letsencrypt-auth.sh
--manual-cleanup-hook /letsencrypt-scripts/letsencrypt-cleanup.sh \

If so please provide a copy of the script (without credentials)

Please also provide your Domain Name - because it's easier to help you with it

Andrei

5

Hi,

Thanks to everyone for the help; particularly @mnordhoff

I think I’ve narrowed my problem down to some unreliable DNS caching “somewhere” - most likely my ISP because manually looking up using Google’s DNS server (8.8.8.8) in another terminal while my hook script is in progress returns the correct result.

To answer a few queries for future visitors:

  • I’m using the CloudFlare example hook scripts provided in the certbot documentation. I had to tinker with them a little bit to make them work, but I’ve confirmed that they are creating the correct TXT record in CloudFlare okay.
  • I did hit the issue that @MrFollies mentioned, whereby the TXT record would get created with portions of my domain name. I set the correct name manually in the hook script to get past this.

So, rather than go to too much heroics trying to solve this DNS caching issue, I’ll take some time to identify a way of getting HTTP based auth to work for my scenario.

Everything is a Freaking DNS Problem right? :wink:

6

In my case it turned out to be the API of the domain vendor inserting a space in the TXT record when requesting it via the API. Frustrating, but I found it n the end.

7

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.