Hello,
I'm here trying to suggest to improve the support for wildcard certificates expecially on the renewal side. Indeed, it appears that going ahead with the "manual" plugin - one of the few supported - breaks then any automation effort to renew them.
It is difficult to accept this situation.
Thanks, kind regards.
Daniele Bonini
Is this a suggestion for improving Certbot?
If so, Certbot offers several built-in plugins for DNS wildcard automation. And, there are several 3rd party plugins. These are described in the Certbot docs (link here).
Certbot also has this extension by one of their developers for further DNS support for wildcard certs (link here)
But, there are many ACME clients you can use to get certs from Let's Encrypt. One of them, acme.sh, has support for numerous DNS systems for automation. Search github for it or see this list for Let's Encrypt's list of other ACME clients
You should use the renewal hooks.
Automating a renewal with the dns-01 challenge requires interacting with your authoritative nameservers. And that can't always be automated.
Another alternative is using acme-dns, and delegating the dns challenge to a different nameserver, using a cname record.
When using the --manual plugin without hooks, there is nothing to improve. The CA is required to re-validate the hostname(s) and the CA is required to do so using one of the challanges allowed for wildcard certificates, in the case of what Let's Encrypt offers, the dns-01 challenge.
When combining the above facts in combination with the manual plugin without hooks, it's evidentely clear that automation is not possible in principe. Therefore, there is nothing to improve.
As my fellow volunteers have already said, there are possibilities to automate the dns-01 challenge, either using Certbot (multiple ways) or using one of the other ACME clients with more DNS support.
I think the problem is really anyone suggesting using the interactive-based (rather than script-based) manual plugin at all, other than for some specific debugging purpose in the staging environment, not really related to wildcards at all. But there are some terrible tutorials out there.
Hello everyone and thanks for your replies.
I'm not a "scolar" of the certicate studies, sorry for that, indeed I'm more on the programming side. However, yes I'm using and sticking on Certbot in its default setup. As I already mention in other posts, I'm not for spending too blood on certificates, expecially if the matter become to sacrify my own webserver conf or the security of my own whole settings.
I'm just watching my recent own scripts to automate the certificates renewal breaking on the wildcard certificates and resulting in splitting my business time emploied for that purpose (now I have to deal with wildcard certificates on a separate time window). Big sufferences, and I'm not that guy who dislike technical challanges.. but the dns hooks...
I'm mostly on my own doing all my stuff, and I could love staging practices.. but try to think I'm also usual to cut off on backend interfaces to do things in time fashion..
I francky would expect any expedient to be able to guess correctly wildcard certificates to renew with easie as they already been verified at emission time...... thats great concept!
Again, thanks and do not hide us any more hints on the wildcard subject!
Daniele Bonini
I have one very simple question:
Can you do any of that with any other CA?
If so, we may be able to copy what they have done.
If not, then it is likely not something that can be done by any CA.
challenging.. but inventions....
I'm not sure, but you might be suggesting that Let's Encrypt should/could be the first CA to "invent" something else with regard to your trouble automating issuance of wildcard certificates.
"Unfortunately" all publicly trusted CAs have to adhere to the CA/Browser Forum Baseline Requirements. And those requirements have very distinct restrictions with regard to what kind of challenge is allowed for issuance of wildcard certificates. Thus, even if LE would want to "invent" something else, the BR would not allow it.
rewording/expanding on that thought a bit...
Even if anyone could event some other "new way" to solve the problem you describe, LE could not implement it without first obtaining the approval of CA/B forum.
[which, at the very best, that would be many months, if not years, in the making]
Only if, frankly, you're doing things very poorly.
If you're using --manual without --manual-auth-hook, you're doing it wrong. Either work out how to automate issuance/renewal using certbot and your DNS host, change to a different client (like acme.sh) that supports more DNS hosts, consider using a different DNS host, or self-host your DNS for validation purposes using something like acme-dns.
Frankly.... thanks for this one and the clarifications contained in it.
Indeed, reading your replies I see the opportunity to automate the dns validation. i can say that I'm much gelouse of my own things, expecially my certbot installation and my dns records. I would prefer to do it simpler and maybe wrong but for me, as a single individual the "future" are some shell scripts as I don't own an important infrastructure with all the security features of the case and I don't want to have it. I neither know the implications in network security of having my host opening a socket to manage the dns records of interest. It could be safe, it could be not..
I also got, thanks for these expansions too that inventions in this school are not welcome but, I want to repeat that, at the time of the wildcard certificate emission Certbot ask to me to create a TXT record and a .wellknown file, I like them a lot and leave them there.... why the renewal process can't simply verify the existance of the same TXT record and file? It is not enough an historical reference for every wildcard certificate and maybe, if you let me joke about my own stuff, a PHP script to do that check?
Thank again for your time!
Daniele Bonini
The short answer is that the Certificate Authority/Browser forum won't allow this, at least after a certain amount of time--any other request for a cert requires a different value for the validation token.
And if you think about it a bit, that value's public. Anyone who had it, then, would be well-armed to get a cert for your domain--which kind of defeats the purpose of the cert.
The main hint we can give is "do you actually need a wildcard, or is a certificate for a big number (<100) of subdomains enough?"
Because if it is enough, automating the http-01 challenge is a lot easier.
Uhm, you are getting me on personal stuff....
I think to need wildcard certificates as I leave the user adding my domain as suffix to theirs, ie: facebook.com.could.work by rewriting the url. I have a certain number of services doing the same. And for the automation purpose one more or one less wildcard doesnt change anything. Just the fact that my shell script breaks on one is enough smoking.
Daniele Bonini
no, it can't. the * in the wildcard can only replace one label. in your example you'd have to get a wildcard for *.com.could.work -- you would need 1500 wildcards just to cover second level domains.
you can rewrite and normalize domains, tho, by removing the dots.
We get all type of requests, so understanding the audience for the Topic is helpful. We are not trying to judge, just understand. 
for that wildcard doesn't work, as wildcard only covers one level subdomain. *.foo.com will cover baa.foo.com but not www.baa.foo.com
Ah-ah, you get me on personal stuff but no one of you tried to launch it, uh?
This is not serious from your side..
Secret: It is able to cover by rewrites 4 labels..
if you are willing to rewrite you could just yourdomain/their-domain and skip wildcard at all