Problem with certbot manual and dns challenge

Hello.
I’m trying to generate a wildcard let’s encrypt certificate using the DNS challenge and manual method.
The command I use is the following:

certbot certonly -n
–manual
–preferred-challenges=dns
–manual-public-ip-logging-ok
-m my-email
–manual-auth-hook /root/cerbero/certbot/generate_new_cert_pre.sh
–manual-cleanup-hook /root/cerbero/certbot/generate_new_cert_post.sh
–server https://acme-staging-v02.api.letsencrypt.org/directory
-d my-domainname -d *.my-domainname

Inside the manual-auth-hook and manual-cleanup-hook I update via API my DNS, so that the challenge can be verified.
As far as I can see, certbot performs two challenges, since I can see two TXT records being added and then deleted in my DNS (I suppose because I request and *..)

The problem is that somehow randomly the challenges (or one of the two) fail. If I repeat the request for that domain it could work, but it’s not deterministic.

Can you help me to understand what’s happening? Could it be an issue related to the TXT record TTL? I also tried to lower it to 10s, but without success.

Thanks a lot.
Kind regards
Riccardo

Probably not related to the TTL, because, as far as I know, TTL is related to caching. But the Let’s Encrypt DNS client always asks the authorative DNS servers, which should deliver an uncached response.

It could however be due to propogation: the time between your script requesting the change via the API and the actual updating of the zone files in all the authorative DNS servers.

You could debug this by using --debug-challenges and while certbot is waiting for user input to continue after deploying the challenges, you can ask all the authorative DNS servers for the TXT records and check if they are there.

Hi @rbrunetti

your domain name is required to check if your name server configuration is buggy.

1 Like

Hello Osiris.
I made another test, without success. This is the command:

certbot certonly -n --manual --preferred-challenges=dns --manual-public-ip-logging-ok -m si@host.it --debug-challenges --manual-auth-hook /root/cerbero/certbot/generate_new_cert_pre.sh --manual-cleanup-hook /root/cerbero/certbot/generate_new_cert_post.sh --server https://acme-staging-v02.api.letsencrypt.org/directory -d corsodijoomla.it -d *.corsodijoomla.it

and this is the output:

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator manual, Installer None
Obtaining a new certificate
Performing the following challenges:
dns-01 challenge for corsodijoomla.it
dns-01 challenge for corsodijoomla.it
Waiting for verification…


Challenges loaded. Press continue to submit to CA. Pass “-v” for more info about
challenges.


Cleaning up challenges
Failed authorization procedure. corsodijoomla.it (dns-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Incorrect TXT record “PQPdK1a0K_p7jOKBTrQOZAgoymjIRtEztvo69XYlPCA” found at _acme-challenge.corsodijoomla.it

IMPORTANT NOTES:

  • The following errors were reported by the server:

    Domain: corsodijoomla.it
    Type: unauthorized
    Detail: Incorrect TXT record
    “PQPdK1a0K_p7jOKBTrQOZAgoymjIRtEztvo69XYlPCA” found at
    _acme-challenge.corsodijoomla.it

    To fix these errors, please make sure that your domain name was
    entered correctly and the DNS A/AAAA record(s) for that domain
    contain(s) the right IP address.

While the command was running, I checked our auth DNS servers and I found:

First:

dig +short @185.84.96.5 _acme-challenge.corsodijoomla.it TXT && dig +short @185.84.97.5 _acme-challenge.corsodijoomla.it TXT
“9FCIGew5jouDS29neRrnL0jSdKo4ISnrojV_FvCC4VA”
“9FCIGew5jouDS29neRrnL0jSdKo4ISnrojV_FvCC4VA”

And after a while:

dig +short @185.84.96.5 _acme-challenge.corsodijoomla.it TXT && dig +short @185.84.97.5 _acme-challenge.corsodijoomla.it TXT
“PQPdK1a0K_p7jOKBTrQOZAgoymjIRtEztvo69XYlPCA”
“PQPdK1a0K_p7jOKBTrQOZAgoymjIRtEztvo69XYlPCA”

So, it seems to me that the correct TXT records where inserted into the DNS.

Thanks fot your help.
Riccardo

Hi @rbrunetti

that looks wrong. There are two entries with the same value.

But that command -d corsodijoomla.it -d *.corsodijoomla.it requires two entries with the same name, but different values.

Checking your domain there is no TXT entry visible - https://check-your-website.server-daten.de/?q=corsodijoomla.it#txt

1 Like

Hello.
In the dig command I reported above, I checked both our two auth DNS servers: 185.84.96.5 and 185.84.97.5.
Thus it’s correct to have two answers (not entries) with the same value. This is exactly what I wanted to check.

In this moment the entries have been deleted from our servers, since inside the manual-clean-up hook I remove the TXT records.

Regards.
Riccardo

But then it’s the wrong result. Then you should see 2 answers with 4 entries (complete, 2 entries per answer).

1 Like

Hello.
So, if I understand correctly, inside the manual-auth-hook I should write both the challenges?

_acme-challenge.corsodijoomla.it TXT “9FCIGew5jouDS29neRrnL0jSdKo4ISnrojV_FvCC4VA”
_acme-challenge.corsodijoomla.it TXT “PQPdK1a0K_p7jOKBTrQOZAgoymjIRtEztvo69XYlPCA”

If the answer is yes: how can I retrieve both of them? At the moment I get the content of the variable: $CERTBOT_VALIDATION

Thanks
Riccardo

Yes, that’s correct and required because you want to validate two domain names - corsodijoomla.it and *.corsodijoomla.it. So the non-wildcard-part is the same, so both validations require _acme-challenge.corsodijoomla.it.

No idea, never tested :wink:

2 Likes

Ok, thank you very much for your help, JuergenAuer.
Does anybody can give me some hint on how to retrieve both the challenges that I need in order to put them into our DNS?

Thanks
Riccardo

According to my simple and quick test, the output of certbot suggests it runs the scripts as many as times as necessary. So if you’d put two domains to validate on the command line, it runs the auth and cleanup scripts twice:

Test with foo.example.com and bar.example.com:

Are you OK with your IP being logged?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(Y)es/(N)o: y
Running manual-auth-hook command: /tmp/auth.sh
Running manual-auth-hook command: /tmp/auth.sh
Waiting for verification...
Challenge failed for domain bar.example.com
Challenge failed for domain foo.example.com
http-01 challenge for bar.example.com
http-01 challenge for foo.example.com
Cleaning up challenges
Running manual-cleanup-hook command: /tmp/cleanup.sh
Running manual-cleanup-hook command: /tmp/cleanup.sh
Some challenges have failed.

My auth script just outputted the variables to a text file:

Domain: bar.example.com
Validation string: AbCdEfG0123456789.myaccount
Remain. chal.: 1
All domains: bar.example.com,foo.example.com

Domain: foo.example.com
Validation string: XyZ0987654321.myaccount
Remain. chal.: 0
All domains: bar.example.com,foo.example.com

Using the dns-01 challenge only the value of the validation string changes (the account key part is removed), but the rest stays the same.

Luckily no rocket science here!

1 Like

That is exactly right. The CERTBOT_VALIDATION variable will give you only one value, but your hook script is run once per challenge. So each time, it should add the record that was indicated to it that time. :slight_smile:

1 Like

Hello Schoen and thanks for your reply.
It seems that everything works if I do not delete the TXT record inside the manual-cleanup-hook.
In this case, the record content is simply written with the first challenge and then replaced with the second challenge.
The only side-effect is that the second TXT will remain inside the zone and I need to delete it at the very end.
I can’t really understand why this happens, but this is what I’m seeing “experimentally”.
Kind regards
Riccardo

That’s strange, because if you look at the certbot output, the cleanup hook is only ran after both the challanges have been performed by the validation server. Are you sure your auth-script isn’t just overwriting any previous existing TXT record? The idea is that an auth-script is adding the TXT record only, not overwriting it.

1 Like

Hello Osiris.
Now I modified the scripts in order to add both the challenges and then delete them inside the cleanup hook once for all.
It seems this solves the issue.
Thanks everybody for your help.
Riccardo

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.