Unable to determine managed zone with Google DNS

Hi folks - I previously had Let's Encrypt working but made the mistake of updating about thirty packages at once, so now things are broken and I'm not sure what to roll back. I only figured this out because of a very helpful "your cert is expiring" reminder email from LE.

My base domain (66c.dev) is hosted on Google Domains. The _acme-challenge subdomain is CNAMED to _acme-challenge.acme.66c.dev; the entire acme.66c.dev subdomain is managed by Google Cloud DNS (and this is where certbot used to add / remove challenge records).

I distinctly remember that I needed to patch a line to make this work last time, which I've done per this thread: DNS plugins don't work if _acme-challenge is a separate zone · Issue #7701 · certbot/certbot · GitHub

But I'm still getting this mystery error:

Unable to determine managed zone for 66c.dev using zone names: ['_acme_challenge.66c.dev', '66c.dev', 'dev'].

This surprises me because _acme_challenge.66c.dev does exist. What am I missing?

The Google specific code is here.

My domain is:

*.66c.dev

I ran this command:

I'm using the Let's Encrypt Home Assistant plugin.

It produced this output:

[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] file-structure.sh: executing... 
[cont-init.d] file-structure.sh: exited 0.
[cont-init.d] done.
[services.d] starting services
[services.d] done.
[22:31:49] INFO: Selected DNS Provider: dns-google
[22:31:49] INFO: Use propagation seconds: 60
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Requesting a certificate for *.66c.dev
Error finding zone. Skipping cleanup.
Unable to determine managed zone for 66c.dev using zone names: ['_acme_challenge.66c.dev', '66c.dev', 'dev'].
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.
[cont-finish.d] executing container finish scripts...
[cont-finish.d] done.
[s6-finish] waiting for services.
[s6-finish] sending all processes the TERM signal.
[s6-finish] sending all processes the KILL signal and exiting.

My web server is (include version):

Using DNS challenge.

The operating system my web server runs on is (include version):

Ubuntu latest.

My hosting provider, if applicable, is:

N/A

I can login to a root shell on my machine (yes or no, or I don't know):

Yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):

Yes, Home Assistant

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):

Let's Encrypt
Current version: 4.12.2

That should have been:
_acme-challenge.66c.dev
[one underscore and one hyphen]

Why/Where it happened?
Beats me!
I'd have to guess that something within Home Assistant has that TYPO.
I would try uninstalling the LE portion and then reinstalling it [if possible].
Otherwise, try checking on their support channels for anything related.

2 Likes

Great catch - thank you! I fixed the typo but unfortunately it's still having trouble finding the zone:

[09:17:23] INFO: Selected DNS Provider: dns-google
[09:17:23] INFO: Use propagation seconds: 60
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Requesting a certificate for *.66c.dev
Error finding zone. Skipping cleanup.
Unable to determine managed zone for 66c.dev using zone names: ['_acme-challenge.66c.dev', '66c.dev', 'dev'].
1 Like

A little progress:

I'm using some very sophisticated instrumentation technology (lots of print statements) and it looks like the google API is returning 0 managed zones in the response for _acme-challenge.66c.dev. Here's the GET (from certbot):

GET https://dns.googleapis.com/dns/v1/projects/home-assistant-288014/managedZones?dnsName=_acme-challenge.66c.dev.&alt=json

Google has an API explorer that lets you handcraft calls in the browser (and then it handles auth for you). And I can see from there the response actually is empty:

So I'm going to have a sniff around in my account settings / see if there have been any changes to this API.

2 Likes

Okay - working again! Nothing wrong with Let's Encrypt or certbot, I'd just forgotten the extent of the modifications you need to make. Leaving a little summary here for myself (and hopefully others).

My setup is: domain name registered with Google Domains; zone managed with Google Cloud DNS. You need to do this because Google Domains doesn't support programattic zone editing (required for ongoing proof-of-ownership through acme verification) and Cloud DNS doesn't support domain purchase / renewal. But the two can work together! This connects with the Home Assistant version of Lets Encrypt.

The Google Domains setup is mostly vanilla:

  • A dynamic DNS A record, so www.66c.dev always points to my home / server IP.
  • A domain forward from 66c.dev to http://www.66c.dev

But you need some changes for acme purposes:

  • An NS record from acme.66c.dev pointing to Google Cloud DNS (ns-cloud-d1.googledomains.com.). This lets you make programmatic changes to your DNS records - but only on the acme.main.domain.
  • A CNAME from _acme-challenge.66c.dev to _acme-challenge.acme.66c.dev. This means when the validator comes looking, we bounce it from the "real" _acme-challenge. subdomain (managed on Google Domains) to the _acme.-challenge.acme subdomain (managed under Cloud DNS).

DNSSEC is turned off (here and in Cloud DNS).

Cloud DNS is pretty simple - I just have a single public zone, called lets-encrypt-challenge-handler with DNS acme.66c.dev.

Home Assistant just needs the Let's Encrypt add on installed. I'm targeting the domain *.66c.dev with privkey.pem / fullchain.pem in their standard locations, challenge type DNS, config provider: dns-google and google_creds: google.json.

Let's Encrypt needs two modifications, which is what I'd forgotten. Both of them are in dns_google.py; there's probably a better way of finding it (docker shuttles it around the filesystem) but I just use sudo find / -name dns_google.py.

First, you need to change the find_managed_zone_id logic so it looks for acme. I did this like so:

zone_dns_name_guesses = dns_common.base_domain_name_guesses('acme.' + domain)

Second, you need to change the Authenticator class (the first thing in the file). By default, it's going to try and add / delete the validation text records against _acme-challenge.66c.dev - which the Cloud DNS API won't accept, because that's not within its acme.66c.dev purview. You could probably be more elegant about this, but I just hardcode the correct values over validation_name in _perform and _cleanup. E.g, mine now looks like this:

    def _perform(self, domain, validation_name, validation):
        self._get_google_client().add_txt_record(domain, "_acme-challenge.acme.66c.dev", validation, self.ttl)

    def _cleanup(self, domain, validation_name, validation):
        self._get_google_client().del_txt_record(domain, "_acme-challenge.acme.66c.dev", validation, self.ttl)

This works, including automatic self renewals - but will presumably break whenever you update LE. I'm tagging @patrakov just in case he has some ideas of how to do this more elegantly - I think he has been through a similar journey with another DNS provider. But for now, the hacky approach works for me and ticks all my boxes :slight_smile:

1 Like

@samuelalexmclean not really, my problem was different. You have a CNAME pointing into a different zone, i.e. a challenge alias. I don't have that, but I maintain the _acme-challenge record as a separate zone for security.

Your setup: zone 66c.dev, zone acme.66c.dev, TXT record at _acme-challenge.acme.66c.dev, CNAME record at _acme-challenge.66c.dev

My setup: zone undisclosed.com, zone _acme-challenge.undisclosed.com, TXT record at _acme-challenge.undisclosed.com (i.e. at the zone root).

Unfortunately I didn't succeed in making this official. The de-facto position seems to be that no fixes in DNS plugins are accepted, because of wildly exceeded manpower.

What I did was to switch to the RFC2136 plugin which worked well enough for our purposes. Nowadays for complex cases such as yours I recommend skipping the existing DNS plugins and using a hook-based approach that could e.g. call Lexicon.

1 Like

That's really helpful - thank you, I didn't get the distinction before.

Changing plugins seems like the only fix if there's not enough manpower to review a PR for such a minor issue (which I totally get - this is a pretty esoteric problem). Next time mine breaks I'll go down that path.

Appreciate the reply!

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.