Get all challenge strings at once?

[jkugler]

Howdy! Because of our DNS setup* we have to do things manually. And it take several minutes for each update to even be visible at our public servers.

Why does certbot give me one challenge string to add to the DNS, and then, once I hit enter, ask me to enter another one?

Could it give me all the strings I need to add as _acme-challenge.gwscientific.com. TXT records so I can add them all at once?

* We have a hidden master which is slaved by our provider. Once we add the records to our server, it sends notifies to their servers. Even after the new record is visible in their web UI, it can take five to 10 minutes for it to be visible directly querying their dns servers. ns1.peer1.net, if you care to know. Thus, it would be nice to have all the challenge strings up front.

[Osiris]

@jkugler

Why does certbot give me one challenge string to add to the DNS, and then, once I hit enter, ask me to enter another one?

Does it try to validate the challenge after you hit enter and before it gets to the next challenge?

[jkugler]

Does it try to validate the challenge after you hit enter and before it gets to the next challenge?

I don't know. After I hit enter, it comes back really fast with the next string, so I don't know if it's querying or not.

[Osiris]

It should have told you so I guess.. It seems to wait until the end.

I'm wondering, I understand your point of getting a nice list of domains, but what is keeping you from pressing enter just a n amount of times? Of course you'd need to know when the last hostname is being presented and not hit enter at that time...

I guess the developers tried to make it as foolproof as possible, as certbot is used by many different users with different levels of skill. Not sure if they'd want to change that.

[jkugler]

Well, I suppose I could do that, but it doesn't say this string is for this domain, so I didn't know if I could press enter X times (where X is the number of host names), or if there was some arbitrary number of strings it would present to me. In my case, yes, I do two host names (my base domain, and *.domain.com), but there was never any indication it would be presenting me X strings because I had X host names. It seems like it might actually be clearer, instead of saying "add this one, don't delete the other one you just added" to print out something that says "Add all these TXT records to your DNS information."

[Osiris]

The hostname is """hidden""" in the challenge hostname, only to be prepended by the _acme-challenge label. But indeed, wildcard hostnames have an identical ACME challenge hostname as their base domain name.

In the end, if you know the number of hostnames, you know how many times to press enter to get all the challenges (${number_of_hostnames} - 1).

I'm curious: from your post I read you're running your own DNS server from which your DNS provider reads the zone info? If you own and manage the primary DNS server, isn't it possible to use a DNS plugin?

[jkugler]

It might be possible. I'll have to investigate. I do see https://certbot-dns-rfc2136.readthedocs.io/en/stable/

I assume when a zone file is updated in that manner, a notify is sent? If so, it is possible it could work. My propagation seconds would still have be something crazy high like 600, but it might be workable.

[jkugler]

I assume it will only remove the strings it added? I can see two hosts stepping on each other otherwise. What are the chances of two hosts generating the same strings?

[_az]

This is pretty inconvenient, but with Certbot >1.4.0 you can emulate "print everything at once" by using a manual auth hook.

Something like:

#!/usr/bin/env sh

echo "_acme-challenge.$CERTBOT_DOMAIN. 60 IN TXT $CERTBOT_VALIDATION" >> /tmp/all-challenges.txt

if [ "$CERTBOT_REMAINING_CHALLENGES" = "0" ]; then
  read -p "Press any key to continue" reply;
fi

then if you run:

sudo certbot certonly --manual  --manual-public-ip-logging-ok \
--manual-auth-hook "./manual.sh" --manual-cleanup-hook "/bin/true" \
--preferred-challenges dns -d foobaz.com -d www.foobaz.com

it will spit out:

$ cat /tmp/all-challenges.txt
_acme-challenge.foobaz.com. 60 IN TXT ypUZHBtoxQsmSZRVLfjpzp91-4apkWcnCHimEzKWy-I
_acme-challenge.www.foobaz.com. 60 IN TXT b7ShWNtW6f7-fZb0WcVL74ponCl-gG9_EnA9bTOTSvY

then you just press any key in the Certbot terminal when ready.

But instead of doing something like that, I would instead use something like acme-dns or DNS alias mode, which rely instead on permanent CNAME delegations, so you only have to setup DNS once.

[Osiris]

@jkugler

I assume when a zone file is updated in that manner, a notify is sent?

I don't know actually. I use the plugin myself with BIND, but that's just a single server system (not professionally ). My knowledge of the RFC 2136 is too little to answer that.

@jkugler

My propagation seconds would still have be something crazy high like 600 (…)

Nah, that isn't crazy high, I think I've seen numbers like 1200 which still is kinda normal

[jkugler]

@_az Ha! That would work, thanks!

And I'll certainly check out acme-dns and DNS alias mode. Sounds like that might be better.

Thanks both for the discussion and tips!

[Osiris]

@_az

This is pretty inconvenient, but with Certbot >1.4.0 you can emulate "print everything at once" by using a manual auth hook.

I thought about that and while the script is nice, somehow it comes across as "ugly workaround" to me

Personally, this should be able to automate, so I think that's where the focus should be

[rmbolger]

@jkugler

It might be possible. I'll have to investigate. I do see https://certbot-dns-rfc2136.readthedocs.io/en/stable/

I assume when a zone file is updated in that manner, a notify is sent?

Yes. Any DNS primary that accepts RFC2136 based updates should notify its secondaries just like if you had made a change manually.

[jvanasco]

@_az

But instead of doing something like that, I would instead use something like acme-dns or DNS alias mode, which rely instead on permanent CNAME delegations, so you only have to setup DNS once.

+1 on this recommendation. Unless you admin your own DNS system and it is rather simple, there are too many peculiarities that make switching records within the DNS-01 auth system difficult to implement. acme-dns is simple to setup and use.

@jkugler

• We have a hidden master which is slaved by our provider

These terms are generally considered outdated and problematic; many open source projects have transitioned away from them, and some consider their usage to be against "Code of Conduct" policies.

If this usage wasn't accidental -- it often is! -- I strongly suggest you begin to migrate to using more accepted and inclusive terms like "primary/replicant" or other popular alternates.