The problem is that renewals on wildcards are failing on a custom renewal system when using these options
–manual
–manual-public-ip-logging-ok
–preferred-challenges=dns-01 –manual-auth-hook /home/user/letsencrypt/automate_hook_auth.py
Turns out that when a wildcarded domain is about to be renewed, automate_auth_hook.py is receiving
“example.com” instead of “*.example.com”
and
“subdomain.example.com” instead of “*.subdomain.example.com”
This custom system works as follows:
automate_hook_auth.py sends the CERTBOT_DOMAIN and CERTBOT_VALIDATION pair to a tiny custom DNS server, which caches that pair in a map.
When later the let’s encrypt validation server queries for the CERTBOT_VALIDATION of that CERTBOT_DOMAIN via the public DNS system, the server looks up the domain in the map and answers with the validation code.
The problem is that since the wildcard is omitted in the CERTBOT_DOMAIN environment variable passed to the script, this erroneous ‘example.com’ will overwrite the map entry which was supposed to be for the real ‘example.com’
So: The DNS server will receive one CERTBOT_VALIDATION for “example.com”, which is meant to be for “*.example.com”, followed by a CERTBOT_VALIDATION for the real “example.com”, which is supposed to go in the same certificate as the wildcard version of that domain. Since the DNS server receives two different CERTBOT_VALIDATION for the same “example.com”, the last one overwrites the first CERTBOT_VALIDATION. So when queried for “*.example.com”, which is actually a query for “example.com” because DNS doesn’t allow wildcards, the return CERTBOT_VALIDATION is the one for the real “example.com”, which doesn’t match the first one. So the certificate renovation will fail.
I’ve noted differences in capitalization arriving in the DNS server, like one time for “exAMPle.Org” and the second one “ExamPLE.oRG”. Is there a way to pass this capitalization to automate_hook_auth.py so that it can pass it to the DNS server and that one then can map a capitalization-sensitive version of the domain names and use that to distinguish which CERTBOT_VALIDATION is referred to?
Actually, you need to create two TXT records with the same name because _acme-challenge.example.com is used to validate both example.com and *.example.com. The incoming query will just be for _acme-challenge.example.com so you have no way of knowing which name it's trying to validate - you have to respond for both.
Have you considered using acme-dns rather than a custom DNS server? It's designed for exactly this purpose (and handles the mixed case queries correctly too).
I do realize that ideally I’d need two TXT records, but the thing is that the incoming DNS requests for “*.example.com” and “example.com” look alike (they only have capitalization differences).
Also, the hook only gets “example.com”, two times, because certbot-auto is apparently dropping the “*.” of the wildcard version of the domain, so I don’t even get the chance to differ between both variants.
I’m not allowed to use a non-custom DNS server (every line in the source code needs to be accounted for).
I’m currently thinking of an approach where I insert the CERTBOT_VALIDATION strings into a list for one CERTBOT_DOMAIN and hope that the validation server picks them up via FIFO, that would allow me to distinguish.
So automate_hook_auth.py would cause the DNS server to push the CERTBOT_VALIDATION to the domain map list, and when the public DNS requests from Let’s Encrypt come in it serves the CERTBOT_VALIDATION via FIFO.
Yes, but acme-dns it is a server consisting of thousands of LOC in dozens of files, while the custom one is a one-file python server consisting of less than 100 lines, where half of them are print statements.