Certbot Error 401 DNS Problem [Noob Question]

Hi folks. Please see the details as requested below. I'm quite sure that it is my fault. Unfortunately I'm quite inexperienced with Linux.

All I want to do is to generate a certificate which I want to use with Adguard Home on my Raspi. As DynDns-Service I'm using desec.io. Im using this manual for Certbot on Desec.io: TLS Certificate with Let’s Encrypt — deSEC DNS API documentation

I've installed Apache2 also and opening the default page (listening on port 81 because 80 is used by Adguard) works fine.

The credentials in .dedynauth should also be fine because I've copied them from my router settings (which should work as the url opens successfully)

Perhaps someone can give me a hint what I'm doing wrong :slight_smile:

My domain is:
maddino.dedyn.io

I ran this command:
sudo certbot --manual --manual-auth-hook ./hook.sh --manual-cleanup-hook ./hook.sh --preferred-challenges dns -d "maddino.dedyn.io:81" certonly

It produced this output:
Error output from manual-cleanup-hook command hook.sh:
curl: (22) The requested URL returned error: 401
curl: (22) The requested URL returned error: 401
curl: (22) The requested URL returned error: 401

Some challenges have failed.

IMPORTANT NOTES:

  • The following errors were reported by the server:

    Domain: maddino.dedyn.io
    Type: dns
    Detail: DNS problem: NXDOMAIN looking up TXT for
    _acme-challenge.maddino.dedyn.io - check that a DNS record exists
    for this domain

My web server is (include version):
Apache/2.4.38 (Raspbian)
The operating system my web server runs on is (include version):
PRETTY_NAME="Raspbian GNU/Linux 10 (buster)"
NAME="Raspbian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster

I can login to a root shell on my machine (yes or no, or I don't know):
Yes (for my Raspi)

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):
No
The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):
certbot 1.14.0

1 Like

Hi @maddino

a domain name has never a port, so that's always wrong.

Use

-d maddino.dedyn.io

instead.

If you use dns validation, you don't need a webserver.

But you have to create a TXT entry _acme-challenge.maddion.dedyn.io.

There - maddino.dedyn.io - Make your website better - DNS, redirects, mixed content, certificates - is no such TXT entry.

Compare it with https://check-your-website.server-daten.de/?q=server-daten.de#txt - Status should be green.

1 Like

Hold on. Saw a typo and fixed it. Testing it again.

Hi @JuergenAuer and many thanks for your reply. I've added a entry (and removed the port again) but it doesn't seem to work so I fear I'm still doing something wrong.

dns

1 Like

Your subname must be _acme-challenge.

The content must be the long string Certbot shows.

1 Like

These hooks come directly from the guide which is followed, which should enable automated DNS record adding and removing. The script uses curl for communication with desec.io. I recon the hook script couldn't cope with the added :81, so I would urge you to try using the hooks again but now with just the hostname.

No reason to do manual stuff if there's a perfectly fine hook to be used :slight_smile:

1 Like

@Osiris: I've tested it without the port first with the same result.

@JuergenAuer: Small problem: With all the testing I've ran into rate limits :confused: For the long string: Certbot only shows the string after aborting with the error message I've posted. So can I use this one again or is there another problem on my side?

1 Like

Even so, if I were you, I'd focus on the hook script. Let's Encrypt certificates are valid for just 90 days with the recommendation of renewing the cert after 60 days. And manually doing the DNS challenge every 2 months is quite cumbersome. So if you can get the hook script to function, that would be way better.

I would recommend the following steps:

  • use the staging environment for testing! That way you won't hit rate limits when you're just testing things out...
  • add the option --debug-challenges, which pauses certbot after it has added the challenge and before it signals the validation server to try to validate the challenge. During this pause, you can manually verify if the TXT record has been added or not. Sometimes it takes a few moments for the change to propogate.
  • debug the hook.sh script. It's a rather simple script, so shouldn't be too hard with some basic bash knowledge.

And no, once a challenge has failed, you can't reuse its token.

1 Like

Have to do more tries tomorrow (wife at home now :wink: ) But the problem persists that Certbot doesn't provide a token for the challenge to enter in the TXT entry. The first time I get to see this long string is when Certbot aborts with the error message I've posted above.

Is there perhaps something to prepare on my Raspi or the installed Apache?

A problem seems to be that this message:

"Setting challenge to WIUARhjNmkwo0XABnoid2oBiaeOi8W6TZqKNWEGWqE0 ...
Waiting 120s for changes be published."

Appears after Certbot has stopped. But it should appear before the 120s countdown starts. Any ideas what could be the problem?

That just depends on the options you've used to run certbot. If you use the hook script, the token gets passed on to the script.

That's the output of hook.sh after it has published the token. It could be that certbot doesn't pass through that output at the appropriate time, but only at the end. Can't reproduce that. My hook.sh that just outputs the token et c. outputs it during the run of certbot without any issue.

$ certbot_test certonly --manual --manual-auth-hook ./hook.sh --manual-cleanup-hook ./hook.sh --preferred-challenges dns -d example.org --debug-challenges
Saving debug log to .certbot_test_workspace/logs/letsencrypt.log
Plugins selected: Authenticator manual, Installer None
Requesting a certificate for example.org
Performing the following challenges:
dns-01 challenge for example.org
Running manual-auth-hook command: hook.sh
Output from manual-auth-hook command hook.sh:
Validation: HI3whFTHQW2OGQOPIIlgyAjg9SXspewsufbzjPmIr2Y\nToken: 

Waiting for verification...

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Challenges loaded. Press continue to submit to CA. Pass "-v" for more info about
challenges.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Press Enter to Continue

Nothing weird there.

It seems certbot will only output the contents of such a hook in total. So if a hook does a single echo, but afterwards waits 120 seconds, certbot won't "see" the echo content until those 120 seconds have passed.

Please use --debug-challenges to check if the token is actually available at the correct hostname when certbot is paused.

This is the output when using the --debug-challenges option.

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator manual, Installer None
Requesting a certificate for maddino.dedyn.io
Performing the following challenges:
dns-01 challenge for maddino.dedyn.io
Running manual-auth-hook command: ./hook.sh
Output from manual-auth-hook command hook.sh:
Setting challenge to xmuunTFDHMmLQUGlls2hXQhdaqm4xlY5zlBQCYCG8xI ...
Waiting 120s for changes be published.
Sa 10. Apr 21:53:15 CEST 2021
Token published. Returning to certbot.

Error output from manual-auth-hook command hook.sh:
curl: (22) The requested URL returned error: 401
curl: (22) The requested URL returned error: 401
curl: (22) The requested URL returned error: 401

Waiting for verification...


Challenges loaded. Press continue to submit to CA. Pass "-v" for more info about
challenges.


Press Enter to Continue
Challenge failed for domain maddino.dedyn.io
dns-01 challenge for maddino.dedyn.io
Cleaning up challenges
Running manual-cleanup-hook command: ./hook.sh
Output from manual-cleanup-hook command hook.sh:
Deleting challenge xmuunTFDHMmLQUGlls2hXQhdaqm4xlY5zlBQCYCG8xI ...
Token deleted. Returning to certbot.

Error output from manual-cleanup-hook command hook.sh:
curl: (22) The requested URL returned error: 401
curl: (22) The requested URL returned error: 401
curl: (22) The requested URL returned error: 401

Some challenges have failed.

IMPORTANT NOTES:

  • The following errors were reported by the server:

    Domain: maddino.dedyn.io
    Type: unauthorized
    Detail: Incorrect TXT record "" found at
    _acme-challenge.maddino.dedyn.io

    To fix these errors, please make sure that your domain name was
    entered correctly and the DNS A/AAAA record(s) for that domain
    contain(s) the right IP address.

HTTP 401 is an "Unauthorized" error. Are your desec.io credentials in .dedynauth correctly configured?

DEDYN_TOKEN=here comes the token
DEDYN_NAME=maddino.dedyn.io

Nothing more in it. And the credentials itself should be correct because they obviously work in my Router settings. It seems to be a really silly mistake I'm making...

I've just set up an account on Dedyn and it works without any issue here. I've used the hook.sh from the guide you've posted:

server letsencrypt # certbot certonly --staging --manual --manual-auth-hook ./hook.sh --manual-cleanup-hook ./hook.sh --preferred-challenges dns --debug-challenges -d osirisinferi.dedyn.io
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator manual, Installer None
Requesting a certificate for osirisinferi.dedyn.io
Performing the following challenges:
dns-01 challenge for osirisinferi.dedyn.io
Running manual-auth-hook command: ./hook.sh
Output from manual-auth-hook command hook.sh:
Setting challenge to ImIENO6aidYuqE8RyY5yuprdnbY0JpwcmRU5dm2F00M ...
Waiting 120s for changes be published.
Sat 10 Apr 22:24:54 CEST 2021
Token published. Returning to certbot.

Waiting for verification...

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Challenges loaded. Press continue to submit to CA. Pass "-v" for more info about
challenges.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Press Enter to Continue

Cleaning up challenges
Running manual-cleanup-hook command: ./hook.sh
Output from manual-cleanup-hook command hook.sh:
Deleting challenge ImIENO6aidYuqE8RyY5yuprdnbY0JpwcmRU5dm2F00M ...
Token deleted. Returning to certbot.


IMPORTANT NOTES:
 - Congratulations! Your certificate and chain have been saved at:
   /etc/letsencrypt/live/osirisinferi.dedyn.io/fullchain.pem
   Your key file has been saved at:
   /etc/letsencrypt/live/osirisinferi.dedyn.io/privkey.pem
   Your certificate will expire on 2021-07-09. To obtain a new or
   tweaked version of this certificate in the future, simply run
   certbot again. To non-interactively renew *all* of your
   certificates, run "certbot renew"
server letsencrypt # 

However, if I modify my token ever so slightly (changed a 4 into a 5):

server letsencrypt # certbot certonly --staging --manual --manual-auth-hook ./hook.sh --manual-cleanup-hook ./hook.sh --preferred-challenges dns --debug-challenges -d osirisinferi.dedyn.io --dry-run
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator manual, Installer None
Cert not due for renewal, but simulating renewal for dry run
Simulating renewal of an existing certificate for osirisinferi.dedyn.io
Performing the following challenges:
dns-01 challenge for osirisinferi.dedyn.io
Running manual-auth-hook command: ./hook.sh
Output from manual-auth-hook command hook.sh:
Setting challenge to fzaPyWXmxM2S6Ro8TYmEpU0pxeEgRDSupKmM85B-5jE ...
Waiting 120s for changes be published.
Sat 10 Apr 22:31:01 CEST 2021
Token published. Returning to certbot.

Error output from manual-auth-hook command hook.sh:
curl: (22) The requested URL returned error: 401 
curl: (22) The requested URL returned error: 401 
curl: (22) The requested URL returned error: 401 

Waiting for verification...

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Challenges loaded. Press continue to submit to CA. Pass "-v" for more info about
challenges.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Press Enter to Continue
Challenge failed for domain osirisinferi.dedyn.io
dns-01 challenge for osirisinferi.dedyn.io
Cleaning up challenges
Running manual-cleanup-hook command: ./hook.sh
Output from manual-cleanup-hook command hook.sh:
Deleting challenge fzaPyWXmxM2S6Ro8TYmEpU0pxeEgRDSupKmM85B-5jE ...
Token deleted. Returning to certbot.

Error output from manual-cleanup-hook command hook.sh:
curl: (22) The requested URL returned error: 401 
curl: (22) The requested URL returned error: 401 
curl: (22) The requested URL returned error: 401 

Some challenges have failed.

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: osirisinferi.dedyn.io
   Type:   dns
   Detail: DNS problem: NXDOMAIN looking up TXT for
   _acme-challenge.osirisinferi.dedyn.io - check that a DNS record
   exists for this domain
server letsencrypt # 

Now I got exactly the same errors you got, three times in a row, 401, due to a purposely invalidated token. And if I change the 5 back into a 4 (i.e., valid token), I can successfully get a cert again.

So please double, triple and even quadruple check the validity of your token. Also note that the "Token identifier" is not the same as the actual token.

1 Like

That's the stupid mistake I've mentioned. Thank you so much! I've mismatched the token identifier with the actual token. Still I'm confused why I can access the url because in my router I've added the identifier instead the actual token too.

Tank you!

2 Likes

Great! I was confused at first too, so easy mistake to make.

The tokens are only shown once. If one looses the token, one would need to generate a new one.

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.