Certbot 1.32 dns challenge with CNAME to acme/dynamic zone allowing updates

Help
1

I've been using the certbot-dns-rfc2136 plugin with updates being performed on the main bind zone for rfsmiths.com . This has all worked fine except for the issue of the zone file being rewritten after dynamic updates and no longer user friendly after the dynamic zone updates by the plugin.

I'm trying to implement the CNAME method so the primary zone file (rfsmiths.com) remains untouched, and the dynamic updates are added to a dedicated zone acme.raysee.net zone. I'm attempting to use the DNS challenge to generate certificates for mail.rfsmiths.com and smtp.rfsmiths.com with the following command:

certbot certonly --dry-run --dns-rfc2136 --dns-rfc2136-credentials ~/certbot-dns-rfc2136.ini --dns-rfc2136-propagation-seconds 10 --cert-name mail.rfsmiths.com -d mail.rfsmiths.com,smtp.rfsmiths.com -vv

however, it never arrives at the actual waiting for the record to propagate as it is attempting to run the update against the rfsmiths.com zone, instead of the desired acme.raysee.net zone per the log shown at the bottom of this message.

All that I have found indicates that I could get a cert for a subdomain on one top level domain, by pointing a CNAME to the _acme-challenge subdomain on another top level domain.

What am I doing wrong? the "REFUSED" DNS server response is expected as I don't want it to be dynamically changing rfsmiths.com and thus there are no permissions there. The permissions are granted for updating _acme-challenge.mail.acme.raysee.net and _acme-challenge.smtp.acme.raysee.net

I'm using certbot 1.32, the EL 9 package. The snap package doesn't work with EL 9 due to a squashfs issue (I was using it on EL 8 without issue).

I saw some references here about support for CNAME but they are all a few years old:

I'll add a few more links when I find the other links that I've since closed.

Performing the following challenges:
dns-01 challenge for mail.rfsmiths.com
dns-01 challenge for smtp.rfsmiths.com
No authoritative SOA record found for _acme-challenge.mail.rfsmiths.com
No authoritative SOA record found for mail.rfsmiths.com
Received authoritative SOA response for rfsmiths.com
Encountered exception:
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/certbot/_internal/auth_handler.py", line 86, in handle_authorizations
    resps = self.auth.perform(achalls)
  File "/usr/lib/python3.9/site-packages/certbot/plugins/dns_common.py", line 76, in perform
    self._perform(domain, validation_domain_name, validation)
  File "/usr/lib/python3.9/site-packages/certbot_dns_rfc2136/_internal/dns_rfc2136.py", line 84, in _perform
    self._get_rfc2136_client().add_txt_record(validation_name, validation, self.ttl)
  File "/usr/lib/python3.9/site-packages/certbot_dns_rfc2136/_internal/dns_rfc2136.py", line 146, in add_txt_record
    raise errors.PluginError('Received response from server: {0}'
certbot.errors.PluginError: Received response from server: REFUSED

Calling registered functions
Cleaning up challenges
No authoritative SOA record found for _acme-challenge.mail.rfsmiths.com
No authoritative SOA record found for mail.rfsmiths.com
Received authoritative SOA response for rfsmiths.com
Encountered exception during recovery: certbot.errors.PluginError: Received response from server: REFUSED
Exiting abnormally:
Traceback (most recent call last):
  File "/bin/certbot", line 8, in <module>
    sys.exit(main())
  File "/usr/lib/python3.9/site-packages/certbot/main.py", line 19, in main
    return internal_main.main(cli_args)
  File "/usr/lib/python3.9/site-packages/certbot/_internal/main.py", line 1744, in main
    return config.func(config, plugins)
  File "/usr/lib/python3.9/site-packages/certbot/_internal/main.py", line 1591, in certonly
    lineage = _get_and_save_cert(le_client, config, domains, certname, lineage)
  File "/usr/lib/python3.9/site-packages/certbot/_internal/main.py", line 141, in _get_and_save_cert
    lineage = le_client.obtain_and_enroll_certificate(domains, certname)
  File "/usr/lib/python3.9/site-packages/certbot/_internal/client.py", line 530, in obtain_and_enroll_certificate
    cert, chain, key, _ = self.obtain_certificate(domains)
  File "/usr/lib/python3.9/site-packages/certbot/_internal/client.py", line 442, in obtain_certificate
    orderr = self._get_order_and_authorizations(csr.data, self.config.allow_subset_of_names)
  File "/usr/lib/python3.9/site-packages/certbot/_internal/client.py", line 510, in _get_order_and_authorizations
    authzr = self.auth_handler.handle_authorizations(orderr, self.config, best_effort)
  File "/usr/lib/python3.9/site-packages/certbot/_internal/auth_handler.py", line 86, in handle_authorizations
    resps = self.auth.perform(achalls)
  File "/usr/lib/python3.9/site-packages/certbot/plugins/dns_common.py", line 76, in perform
    self._perform(domain, validation_domain_name, validation)
  File "/usr/lib/python3.9/site-packages/certbot_dns_rfc2136/_internal/dns_rfc2136.py", line 84, in _perform
    self._get_rfc2136_client().add_txt_record(validation_name, validation, self.ttl)
  File "/usr/lib/python3.9/site-packages/certbot_dns_rfc2136/_internal/dns_rfc2136.py", line 146, in add_txt_record
    raise errors.PluginError('Received response from server: {0}'
certbot.errors.PluginError: Received response from server: REFUSED
Received response from server: REFUSED
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.
2

Yeah, I don't think Certbot supports this "DNS alias" mode, where you update one zone even though your domain is in another. You'll need to find a better ACME client to get that feature.

7 Likes
3

What CNAME(s) have you created?

6 Likes
4

ack! I was under the impression that it was in fact working but that might explain why I'm not seeing recent updates to this issue.

1 Like
5

Sorry for not including that...I knew I was forgetting something!

_acme-challenge.mail    CNAME acme.raysee.net.
_acme-challenge.smtp    CNAME acme.raysee.net.
1 Like
6

Did it ever work?

3 Likes
7

My original configuration without CNAMEs did work, but that was updating the rfsmiths.com zone directly, which makes a mess of the zone file.

It has not worked in my attempts to complete the challenge using CNAMEs

2 Likes
8

Have you tried?:
_acme-challenge.mail CNAME _acme-challenge.acme.raysee.net.

7 Likes
9

I have, but I will try again. The problem is that I can see in the logs that it is trying to update the rfsmiths.com zone directly, rather than following the CNAME, then making the changes there.

10

Those need different CNAMEs than the ones used.
_acme-challenge.mail CNAME _acme-challenge.mail.acme.raysee.net
_acme-challenge.smtp CNAME _acme-challenge.smtp.acme.raysee.net

7 Likes
11

That is a problem...

Which you might be able to workaround with some DNS tricks ...
Let me check my bag...

7 Likes
12

OK if you lie to certbot (via DNS)
And tell it that the authoritative DNS servers for domain #1 are the ones for domain #2 ...
We might stand a chance!

7 Likes
13

hmm, not sure I understand. My interpretation was that the CNAME should indicate to the challenge script where to do the zone updates. I have this in the rfsmiths.com zone:

_acme-challenge.mail    CNAME _acme-challenge.mail.acme.raysee.net.
_acme-challenge.smtp    CNAME _acme-challenge.smtp.acme.raysee.net.

and here's the acme.raysee.net zone with the grants

    zone "acme.raysee.net." {
        type master;
        file "external.acme.raysee.net.zone";

        update-policy {
            grant certbot. name _acme-challenge.mail.acme.raysee.net. txt;
            grant certbot. name _acme-challenge.smtp.acme.raysee.net. txt;
        };
        allow-query { any; };
    };
1 Like
14

Which means you will have to CNAME from domain #1 to domain #2
Lie to certbot and make the changes in domain #2
But even that would need some internal DNS trickery too
So that the Internet requests to domain #2 are responded with the entries updated into the fake domain #1 in DNS server for domain #2.

6 Likes
15

ah, okay...it was my understanding that I was making mistakes in a method that is known to work and supported. It sounds like the alternative client or some other manual intervention is required. I can also do it other ways using the web server that already exists on this VM...however, I was hoping it could be a bit more generic in case I move the mail server away from the web server in the future.

16

Seems like alot to do for something that should just work "out of the box" though.

A "simpler" (already tested/proven) ACME DNS client may be a better choice.

7 Likes
17

maybe still a lesser of two evils...now that I know it isn't an out of the box request, my efforts may be better invested elsewhere.

If you have any tips or tricks, I'm happy to try them out!

2 Likes
18

What happens now with the new CNAMEs?
use -vv or more detail

6 Likes
19

it's the same problem here...before this is the nonce/key exchange with the acme servers which doesn't show any issues.

Performing the following challenges:
dns-01 challenge for mail.rfsmiths.com
dns-01 challenge for smtp.rfsmiths.com
No authoritative SOA record found for _acme-challenge.mail.rfsmiths.com
No authoritative SOA record found for mail.rfsmiths.com
Received authoritative SOA response for rfsmiths.com
Encountered exception:
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/certbot/_internal/auth_handler.py", line 86, in handle_authorizations
    resps = self.auth.perform(achalls)
  File "/usr/lib/python3.9/site-packages/certbot/plugins/dns_common.py", line 76, in perform
    self._perform(domain, validation_domain_name, validation)
  File "/usr/lib/python3.9/site-packages/certbot_dns_rfc2136/_internal/dns_rfc2136.py", line 84, in _perform
    self._get_rfc2136_client().add_txt_record(validation_name, validation, self.ttl)
  File "/usr/lib/python3.9/site-packages/certbot_dns_rfc2136/_internal/dns_rfc2136.py", line 146, in add_txt_record
    raise errors.PluginError('Received response from server: {0}'
certbot.errors.PluginError: Received response from server: REFUSED

Calling registered functions
Cleaning up challenges
No authoritative SOA record found for _acme-challenge.mail.rfsmiths.com
No authoritative SOA record found for mail.rfsmiths.com
Received authoritative SOA response for rfsmiths.com
Encountered exception during recovery: certbot.errors.PluginError: Received response from server: REFUSED
Exiting abnormally:
Traceback (most recent call last):
  File "/bin/certbot", line 8, in <module>
    sys.exit(main())
  File "/usr/lib/python3.9/site-packages/certbot/main.py", line 19, in main
    return internal_main.main(cli_args)
  File "/usr/lib/python3.9/site-packages/certbot/_internal/main.py", line 1744, in main
    return config.func(config, plugins)
  File "/usr/lib/python3.9/site-packages/certbot/_internal/main.py", line 1591, in certonly
    lineage = _get_and_save_cert(le_client, config, domains, certname, lineage)
  File "/usr/lib/python3.9/site-packages/certbot/_internal/main.py", line 141, in _get_and_save_cert
    lineage = le_client.obtain_and_enroll_certificate(domains, certname)
  File "/usr/lib/python3.9/site-packages/certbot/_internal/client.py", line 530, in obtain_and_enroll_certificate
    cert, chain, key, _ = self.obtain_certificate(domains)
  File "/usr/lib/python3.9/site-packages/certbot/_internal/client.py", line 442, in obtain_certificate
    orderr = self._get_order_and_authorizations(csr.data, self.config.allow_subset_of_names)
  File "/usr/lib/python3.9/site-packages/certbot/_internal/client.py", line 510, in _get_order_and_authorizations
    authzr = self.auth_handler.handle_authorizations(orderr, self.config, best_effort)
  File "/usr/lib/python3.9/site-packages/certbot/_internal/auth_handler.py", line 86, in handle_authorizations
    resps = self.auth.perform(achalls)
  File "/usr/lib/python3.9/site-packages/certbot/plugins/dns_common.py", line 76, in perform
    self._perform(domain, validation_domain_name, validation)
  File "/usr/lib/python3.9/site-packages/certbot_dns_rfc2136/_internal/dns_rfc2136.py", line 84, in _perform
    self._get_rfc2136_client().add_txt_record(validation_name, validation, self.ttl)
  File "/usr/lib/python3.9/site-packages/certbot_dns_rfc2136/_internal/dns_rfc2136.py", line 146, in add_txt_record
    raise errors.PluginError('Received response from server: {0}'
certbot.errors.PluginError: Received response from server: REFUSED
Received response from server: REFUSED

and I can see the refusal in the named logs, as it is trying to update rfsmiths.com instead of the acme.raysee.net zone.

3 Likes
20

on this topic, it seems like something like this could work, that does the nsupdate on the side, but that just points to using another client perhaps. For now, setting up another virtual host in apache is easy and solves my issue for now.

1 Like