I have 5 domains all pointing to the same server and I have to use manual mode without any hooks. Manual mode with expand provides a DNS verification code once for each domain, so I have to update my DNS record and wait for it to propagate (which can take an hour or longer) before I get the verification code for the next domain. I have to repeat this 5 times which can take all day.
Is there a way to run certbot in manual mode with expand and get all the verification codes at one time? This will allow me to update my DNS records once and then wait for the propagation at the same time.
certbot certonly --manual -v --preferred-challenges dns --expand -d example.com,example2.com,example3.com,example4.com, example5.com
That sounds like an awful DNS provider if it takes an hour or more for the authoritive servers to sync. Usually that only takes a few seconds or even a couple minutes. The Let's Encrypt servers look directly at the authoritive DNS servers and don't need to wait for TTL expiry in resolvers.
If using this DNS provider is so difficult have you considered switching to a friendlier one?
Or CNAME _acme-challenge.example[1-5].com to another DNS hosting provider with a friendly API. I am doing this way.
Forget it. I forget certbot doesn't support following CNAME. Using CNAME requires changing to another ACME client.
(Because of historical reasons, I have to keep using certbot, so I actually patched certbot plugin to support CNAME. If I were to build my centralize certificate request server today, I would almost certainly used acme.sh.)
That's how Certbot currently works: it asks the user to put all the challenges in place and after everything is in place, only then it will notify the ACME server the challenges are ready to be validated with the "Before continuing, verify the TXT record has been deployed. (...)" instructions.
So you can in fact just go through the 5 challenges and wait after the last one.
Or maybe I don't understand your issue properly? Because as I see it, Certbot functions just the way you want it to function?
Are you sure you are checking the authoritative DNS servers directly?
[that sounds way too long]
If so, then you might want to speak with the DSP about their slow sync issues OR switch to a better DSP.
Thanks @Osiris. I hope that's true, but the instructions from Certbot say something different. Are you saying that I should just add the value to all domain DNS TXT records and not the one?
Here are the instructions from certbot. example1.com is only one of the five domains
Please deploy a DNS TXT record under the name:
_acme-challenge.example1.com.
with the following value:
pjsExXChpjsExXCpjsExXCpjsExXCpjsExXC
Before continuing, verify the TXT record has been deployed. Depending on the DNS
provider, this may take some time, from a few seconds to multiple minutes. You can
check if it has finished deploying with aid of online tools, such as the Google
Admin Toolbox: https://toolbox.googleapps.com/apps/dig/#TXT/_acme-challenge.glueynotes.com.
Look for one or more bolded line(s) below the line ';ANSWER'. It should show the
value(s) you've just added.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Press Enter to Continue
I'm using Google Domains. This is what they say regarding propagation: These DNS settings are active. Changes are published immediately, but may take time to propagate
Supposedly it can take up to 48 hours to propagate. I've found that it usually takes 30 min to 3 hours, though sometimes it's very fast and only takes a few minutes. I've also notice that sometimes if a change doesn't propogate within a few hours and a make the change again, it happens very quickly, like somehow it was stuck.
When I use a command line like you showed with multiple domain names I do not get the prompt that says "Before continuing" until after all 5 TXT values have been shown.
I do get a simple "Press Enter" after each one but I don't get the full warning about verifying the TXT records until I see all 5 (well, in my test just 2).
Try using unboundtest.com for the TXT check. Google domains shouldn't take all that long.
But, even better is what Osiris is suggesting for Certbot. Or, consider switching to acme.sh which also supports that DNS as a plug-in.
My test sequence for 2 domain names:
Please deploy a DNS TXT record under the name:
_acme-challenge.testu.example.website.
with the following value:
uPIArxy2lETpdq1RwIZ-KnCm7GFeegBd__9YiTulGmw
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Press Enter to Continue
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Please deploy a DNS TXT record under the name:
_acme-challenge.testu2.example.website.
with the following value:
KSHKyCTSaCFmefo-cEheHtxjssx3PhX5NvfZeuSeBNE
(This must be set up in addition to the previous challenges; do not remove,
replace, or undo the previous challenge tasks yet. Note that you might be
asked to create multiple distinct TXT records with the same name. This is
permitted by DNS standards.)
Before continuing, verify the TXT record has been deployed. Depending on the DNS
@Osiris Yes, that worked. Even though the certbot instructions only says to update the DNS TXT for one domain, you can update it for all the domains. Yay! That saves me a lot of time.
I really wish I could find a more automated way. My issue is that I'm running a Docker image on Google Compute Engine. Google requires their own OS called Container-Optimized OS. It doesn't have snap installed and there are a bunch of limitations, many packages can't run on it.
One solution is to use Google's load balancer which supports their own managed SSL, but that will add a bunch of additional costs and time that I rather avoid now. Eventually, I'll likely go that route.
Does it take time to propagate between all the authoritative DNS servers?
OR
Does it take time to propagate between all the caching DNS servers on the Internet?
If #1, then they should provide the expected worst case scenario time.
If #2, ignore that line; Because LE only uses authoritative DNS queries / replies.
And ask them how long for #1.
I don't think you really need to wait it for propagate, it probably only means DNS TTL. LE checks authoritative DNS server directly. (so there is no TTL issue)
Typically a google domains update will be complete across nameservers within 60 seconds.
The problem though, is that google domains works with one API token per zone, so if you are trying to get multiple domains (zones) in one cert I'm not sure how you can achieve that with certbot. Some other clients support lists of authorizations configs matched to domains. One strategy could be to get a cert for each domain (zone), then order the final cert with the combined domains as the authorizations should be cached by LE (I can't remember what the behaviour is across multiple orders though but I think it would work).
Note that the above Certbot plugin and the certbot-dns-multi plugin I mentioned earlier also provide (experimental) Docker images (as OP noted they're using Docker).
Authorizations are not coupled to orders. You should even be able to request 5 certs with purposely adding a failing (sub)domain in the request, authorizing the valid hostnames, but without actually issuing a cert, so you have less problems with rate limits (except for the failed order rate limit, but that one is quite high).
@Osiris Yes, that worked. Even though the certbot instructions only says to update the DNS TXT for one domain, you can update it for all the domains. Yay! That saves me a lot of time.
I'm still wondering why you seemingly are getting different results than Mike above and myself. For me (and Mike above), the DNS challenge instructions are very clear:
add challenge for hostname 1
add challenge for hostname 2
add challenge for hostname ...
add challenge for hostname n
after the last challenge, verify everything is set up correctly
profit
But for some reason you seem to have the notion that Certbot is instructing you otherwise? I'm very curious what output of Certbot you're getting..