Acme.sh with multiple DNS providers for same cert?

[gossamer]

I just started using acme.sh after having used "certbot --manual --preferred-challenges dns certonly" for many years. Most of my domains are with cloudns, but two are proxied/cached and managed by cloudflare. The two domains with cloudflare have webservers and email servers associated with the domain, while the other 10+ domains with cloudns only have postfix servers associated with them.

The problem I'm having is that a mail server may be responsible for receiving mail for one of the domains from cloudns as well as one of the domains managed by cloudflare. It used to be that I could manually add the _acme-challenge TXT record to the corresponding DNS provider and certbot would just query the TXT record and receive the response. Now I'm not so sure that is happening.

Using acme.sh with "--dns dns_cloudns" succeeds in producing a working certificate for the domains managed by cloudns, and using "--dns dns_cf" succeeds in producing a working certificate for the domains managed by cloudflare, but combining the two --dns options apparently causes it to go through the process of updating the cloudflare certificate, but ultimately it fails.

Just to be clear, I'm successfully using the cloudflare and cloudns APIs separately, so I don't think it's an authentication issue.

What could I possibly be doing wrong? There doesn't seem to be any single error message that I can point to to help troubleshoot this. Is this the expected behavior? acme.sh appears to produce a series of timeouts:

[Tue Sep  6 04:35:37 PM EDT 2022] d='www.dom1.org'
[Tue Sep  6 04:35:37 PM EDT 2022] txtdomain='_acme-challenge.www.dom1.org'
[Tue Sep  6 04:35:37 PM EDT 2022] aliasDomain='_acme-challenge.www.dom1.org'
[Tue Sep  6 04:35:37 PM EDT 2022] txt='hanFYdBvtIzJ1N2bp77PGswQGTylKsxU2PBZVY1IZmk'
[Tue Sep  6 04:35:37 PM EDT 2022] d_api='/etc/letsencrypt/acme.sh-master/dnsapi/dns_cloudns.sh'
[Tue Sep  6 04:35:37 PM EDT 2022] Checking www.dom1.org for _acme-challenge.www.dom1.org
[Tue Sep  6 04:35:37 PM EDT 2022] Already success, continue next one.
[Tue Sep  6 04:35:37 PM EDT 2022] d='www.dom2.org'
[Tue Sep  6 04:35:37 PM EDT 2022] txtdomain='_acme-challenge.www.dom2.org'
[Tue Sep  6 04:35:37 PM EDT 2022] aliasDomain='_acme-challenge.www.dom2.org'
[Tue Sep  6 04:35:37 PM EDT 2022] txt='PhN_Zflss56jiJmcRHep27I8YEZh8sN4V_Ghf_SM1yo'
[Tue Sep  6 04:35:37 PM EDT 2022] d_api='/etc/letsencrypt/acme.sh-master/dnsapi/dns_cloudns.sh'
[Tue Sep  6 04:35:37 PM EDT 2022] Checking www.dom2.org for _acme-challenge.www.dom2.org
[Tue Sep  6 04:35:37 PM EDT 2022] Already success, continue next one.
[Tue Sep  6 04:35:37 PM EDT 2022] Let's wait 10 seconds and check again.
[Tue Sep  6 04:35:48 PM EDT 2022] Timed out waiting for DNS.
[Tue Sep  6 04:35:48 PM EDT 2022] check dns error.
[Tue Sep  6 04:35:48 PM EDT 2022] _on_issue_err
[Tue Sep  6 04:35:48 PM EDT 2022] Please check log file for more details: /home/alex/.acme.sh/acme.sh.log

It looks like it doesn't have the intelligence to know which DNS provider to add the TXT record? Here's an example of acme.sh adding a TXT record for cloudns when it should have added it to cloudflare:

Tue Sep  6 05:04:16 PM EDT 2022] Adding txt value: klzv72XYWH0GoGHqpAONnFPnWfUvtk6-oCO1f5O80rc for domain:  _acme-challenge.portal.dom3.com
[Tue Sep  6 05:04:16 PM EDT 2022] Using cloudns
[Tue Sep  6 05:04:18 PM EDT 2022] Adding the TXT record for _acme-challenge.portal.dom3.com
[Tue Sep  6 05:04:18 PM EDT 2022] Added.
[Tue Sep  6 05:04:18 PM EDT 2022] The txt record is added: Success.

It later then proceeds to remove the record, apparently after a timeout or inability to resolve the record.

Ideas greatly appreciated

[Rip]

@gossamer

acme.sh

Hi @gossamer and welcome back!
I am intrigued with your issue but don't have the experience with multiple DNS providers or how to implement such a configuration with acme.sh.

Let's ping @Neilpang and see if he would be willing to help you out!

[Neilpang]

@Rip thanks for Ping me, yes, I'm always there to help.

acme.sh supports to use different dns providers for different domains in the same cert.

Here is the doc about the hybrid mode:

How to issue a cert · acmesh-official/acme.sh Wiki

Let me know if you have any more problems.

Thanks

[gossamer]

Thanks so much for your help. Before I do this, can I ask you to confirm that it appears the way to go about this is to specify each domain along with its corresponding DNS server/API method, correct?

[Neilpang]

Yes, specify one provider for each domain.

There is a shortcut, if you have 5 domains, but you only specify 3 providers, the first 3 domains will use the 3 providers, the last 2 domains will use the last one provider.

-d   a.com
-d  b.com
-d  c.com
-d  d.com
-d  e.com
--dns  dns_providerA
--dns  dns_providerB
--dns  dns_providerC

The domain a.com will use providerA, domain b.com will use providerB, the domain c.com, d.com and e.com will all use the providerC.

[Osiris]

Which command line options would one use if one would have domains A and B use DNS provider X and C and D with DNS provider Y?

Because if you'd use the format above, it would be using A, B and C with X and just D with Y?

Wouldn't it be possible to have something like the --webroot-path and -d options in Certbot? The --dns option before the -ds is valid for just those -d options and if you use another --dns option, that's going to be used for the next -d options?

[Neilpang]

Which command line options would one use if one would have domains A and B use DNS provider X and C and D with DNS provider Y?

-d A   --dns X
-d B  --dns X
-d C  --dns Y
-d D  --dns Y

or:

-d A   --dns X
-d B  --dns X
-d C  --dns Y
-d D 

Only the last provider is shared.

[Osiris]

Cool, good to know!

[rg305]

@Neilpang

Only the last provider is shared.

hmm...

What do these do?

-d A  
-d B  --dns X
-d C  --dns Y
-d D

-d A  --dns X 
-d B
-d C
-d D  --dns Y

[Neilpang]

Only the domain A will use the X, BCD will all use Y.

It doesn't matter where you write the parameters. It's just a list internally. like: A,B,C,D and X,Y.

[Osiris]

@Neilpang

It doesn't matter where you write the parameters. It's just a list internally. like: A,B,C,D and X,Y.

Hm, that sounds like it could be improved. Because if you want to combine a few (e.g. 2) DNS providers with a lot of hostnames, you'd need to use a -d and --dns combination for every hostname, otherwise you wouldn't have exact controle over it.

[Neilpang]

Yes, it looks not perfact, but it's a limitation of how acme.sh processes commandline parameters.
As this is a rare case, I would prefer to keep it as simple as is.

If there are a lot of user demands, maybe in future I will change it.

Thanks

[gossamer]

It worked - thanks so much for your awesome support and for developing such an incredible solution.

I just appended the corresponding --dns parameter to each of the hosts to avoid any confusion.

It's worth noting that cloudflare apparently won't allow you to create certs for hosts that it's already proxying and providing its own certs. The whole operation failed until I took out the few hosts that it's providing web certs.

[Rip]

@Neilpang

As this is a rare case,

True enough. This is a rare case and I haven't encountered it previously.
But there is room to grow @Neilpang .
I use acme.sh (3 years now) with no issues, but in the future, I may have multiple DNS providers that are not linked in any way.

ACME.SH is a great script. Well Done. There is some pushback regarding the default CA, and I understand that. But again I say Well Done.
And thank you for helping in this thread

[rg305]

The simple solution is to get individual certs.

So, this (to me) will only really affect those that:

• insist on using a single cert for all their FQDNs
  and
• use DNS-01 authentication
  and
• use multiple DSPs

Even if each of those categories has 20% of all issued certs, the resulting intersection (group) would be 20% of 20% of 20% or less than 1%.
[yeah, that is extremely quick mafs, but it shows how unlikely/uncommon this use case should be]