Pre/post manual hooks missing environment variables

Command:sudo certbot renew --dry-run
Result: Failure to validate
Root SSH Access: Yes
Certbot version: 0.31.0

I initially generated a manual wildcard certificate on my domain, and everything went fine. The SSL certificate is up and running on my domain and all subdomains. I'm using Office 365 as the DNS controller so I can't use a DNS application for the verification (must be done manually). So I decided to write some pre/post shell scripts which will create the file used for HTTP verification. However the environment variables that are supposedly set and available in the pre/post hooks aren't. I've read other forum posts like this one which illustrate my problem, but the solutions don't work.

I know that the pre/post scripts are being executed as I can see the file being created/destroyed; it's just that the filename/contents are the "missing" defaults and not the expected environment variables. Here's my current setup:

/etc/letsencrypt/renewal/mydomain.com.conf

# renew_before_expiry = 30 days
version = 0.31.0
archive_dir = /etc/letsencrypt/archive/mydomain.com
cert = /etc/letsencrypt/live/mydomain.com/cert.pem
privkey = /etc/letsencrypt/live/mydomain.com/privkey.pem
chain = /etc/letsencrypt/live/mydomain.com/chain.pem
fullchain = /etc/letsencrypt/live/mydomain.com/fullchain.pem

# Options used in the renewal process
[renewalparams]
server = https://acme-v02.api.letsencrypt.org/directory
authenticator = manual
account = [redacted]
pref_challs = http-01,
manual_auth_hook = /usr/local/bin/certbot-manual-auth-hook.sh
manual_cleanup_hook = /usr/local/bin/certbot-manual-cleanup-hook.sh

/usr/local/bin/certbot-manual-auth-hook.sh

#!/bin/sh

TOKEN="${CERTBOT_TOKEN:-token_missing}"
VALIDATION="${CERTBOT_VALIDATION:-validation_missing}"
validation_file="/home/account/public_html/.well-known/acme-challenge/$TOKEN"
sudo -u web-server ssh root@mydomain.com "echo $VALIDATION > $validation_file"

/usr/local/bin/certbot-manual-cleanup-hook.sh

#!/bin/sh

TOKEN="${CERTBOT_TOKEN:-token_missing}"

sudo -u web-server ssh root@mydomain.com "rm -f /home/account/public_html/.well-known/acme-challenge/$TOKEN"
1 Like

To clarify, is this certificate a wildcard?

Wildcards can only be obtained using the DNS challenge. Trying to renew a wildcard certificate with the HTTP challenge should cause Certbot to produce an error that the challenge is not supported.

(With the caveat that with your older Certbot 0.31.0, that error might not crop up until a date closer to your next renewal).

When you say pre and post, do you mean --manual-auth-hook and --manual-cleanup-hook?

Or are you also running these scripts via --pre-hook and --post-hook separately? Those would not have those environment variables set, since they serve a different purpose.

1 Like

Yes, I do mean --manual-auth-hook and --manual-cleanup-hook (as per the config).

Actually, I initially did validate the wildcard certificate using a DNS entry simply because I had manual access to it and it seemed convenient. Only when I discovered the lack of DNS autorenewal support for Office 365 did I switch to the HTTP method. I guess this is my biggest issue.

I would really prefer to keep using the wildcard method as it keeps my setup simpler to manage. Do you know of a way of automatically updating Office 365 managed DNS?

1 Like

The manual hooks are explained in the section pre and post hooks in the certbot manual... Might add to the confusion...

2 Likes

If Office365 DNS doesn't provide an API, then you might be able to use something acme-dns.

That way, you would setup a CNAME record by hand one time in Office365, and after that you will be able to have automated wildcard renewals using the DNS challenge.

Using Certbot, you can follow https://github.com/joohoi/acme-dns-certbot-joohoi through the the process of setting up a certificate using this method.

:frowning:

1 Like

I think I might have to take Brad up on that PR to overhaul the user guide pretty soon...

1 Like

I have a split-horizon DNS situation going on. The official DNS server is with Office 365, and my local LAN DNS server gives out local IP records for arbitrary subdomains primarily for local network services.

I followed the instructions found here for configuring BIND9 with RFC2136. I was able to manually update the certificate using RFC/DNS protocol but the dry-run fails, giving the error:

Domain: mydomain.com
Type: unauthorized
Detail: No TXT record found at _acme-challenge.mydomain.com

I had to create a CNAME entry on Office 365 for _acme-challenge which points to my private network (static IP). My router has port 53 forwarded to my local bind DNS application. I know that the setup is working to the extent that the DNS server is giving a response (tested using online DNS tools pointing to _acme-challenge.mydomain.com). However it seems the TXT record updates aren't working for whatever reason.

1 Like

I don't think the Certbot RFC2136 plugin currently supports setups where _acme-challenge is a CNAME.

i.e. If your external DNS says:

_acme-challenge.example.com. IN CNAME acme.local.example.com.

local.example.com.           IN NS    ns.local.example.com.
ns.local.example.com.        IN A     192.0.2.1

then Certbot would need to write the TXT record to acme.local.example.com. rather than to _acme-challenge.example.com.. It won't do that, because it doesn't understand CNAMEs right now.

You could probably make it work by using a --manual-auth-hook + a nsupdate script.

acme-dns would do the trick as well, since it is specifically made to handle this CNAME situation.

Or even another ACME client which supports aliasing + RFC2136, e.g. https://github.com/acmesh-official/acme.sh/wiki/DNS-alias-mode

1 Like

My external DNS has a CNAME: _acme-challenge.example.com > local.example.com.

When running certbot I can publicly see the TXT record of _acme-challenge.example.com using this website wherein I use that domain for both fields (DNS Server and Domain Name). So the local DNS server is updating the TXT record that certbot is issuing locally. However this TXT record only applies to _acme-challenge.example.com, not example.com.

When certbot attempts to validate, I guess it isn't trying to retrieve the TXT record from _acme-challenge.example.com, but instead from example.com? That's the only way I can see it failing.

In that case, would it be reasonable to append my local nameserver to the registrar's list of nameservers as a fallback? Do TXT record searches work that way? Will a TXT request from example.com fall back to the _acme-challenge.example.com nameserver?

To make this work, you would also need to subdelegate local.example.com to your local nameserver as well, using a NS record (like in my last post).

Without a subdelegation, Let's Encrypt's recusors aren't going to end up finding your local BIND server, and the process will just result in NXDOMAIN.

But what I said about CNAMEs and certbot-dns-rfc2136 still applies. Certbot will create the record at the wrong label. It needs to create it at the CNAME target, and I suggested a couple of ways to make that work.

I'm unable to create NS records on Office 365. Would moving my DNS to namecheap and using ALIAS for _acme-challenge fix the issue? If not, at the very least I would also be able to create NS record you suggested.

https://www.namecheap.com/security/premiumdns/#features

Certbot will create the record at the wrong label.

Is that what's happening in my situation? Based on my results it seems to me that TXT requests to _acme-challenge.example.com is yielding results:

The [redacted] string was presumably created by certbot and updated on my local BIND server using rfc2136 under the _acme-challenge zone and is publicly accessible. Is the missing NS record the only culprit here?

EDIT: Or is the real issue that my wildcard cert for *.example.com requires the TXT record to be present on example.com and not _acme-challenge.example.com?

Yes, doing this at all requires the ability to create an NS record (delegate a subdomain to another nameserver). Or otherwise using a second domain name entirely.

If you are willing to move DNS hosts, you may as well choose one that supports automation and avoid this problem to begin with. Be aware that Namecheap only gives you API access if you spend a certain amount of money.

There's a list of the DNS hosts that Certbot has first-party support for which you might find helpful. Also DNS providers who easily integrate with Let's Encrypt DNS validation.

I personally recommend a Cloudflare account for free DNS hosting, it's fast, reliable and straightforward to use with Certbot. But there are other good choices as well.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.