Certificate renewal failed using manual DNS challenge

Help
1

I'm trying to renew nutthause.com certificate with certbot manual dns challenge/validation using acme-dns-auth.py but errors out with dns challenges failed for nutthause.com, incorrect txt records, and:
Running manual-auth-hook command: /etc/letsencrypt/acme-dns-auth.py
manual-auth-hook command "/etc/letsencrypt/acme-dns-auth.py" returned error code 1
Error output from manual-auth-hook command acme-dns-auth.py:
Traceback (most recent call last):
File "/etc/letsencrypt/acme-dns-auth.py", line 4, in
import requests
ImportError: No module named requests

I have renewed nutthause.com domain successfully for the past two years running with same dns challenge for these subdomains, " -d helios.nutthause.com -d media.nutthause.com -d media2.nutthause.com -d silo-omv.nutthause.com -d silo2-omv.nutthause.com -d www.nutthause.com -d nutthause.com", and with a wildcard dns challenge renewal " -d *.nutthause.com -d nutthause.com"- so I don't have to add a new subdomain for each new host. Yet I pretty much get the same errors listed below and above. I have CNAME/Alias records created for the domain and its subdomains, but no txt records. It keeps complaining about invalid txt records yet in the past I did not have txt records, do I need txt records? I created txt records in the domain tools at Zone Edit ( @ nutthause.com _acme-challenge.nutthause.com=8rw3dccMTTIbI_YJ8hgYgpw_eF_iSiQ8PdS24Or0Dfc , and .nutthause.com _acme-challenge.nutthause.com=8rw3dccMTTIbI_YJ8hgYgpw_eF_iSiQ8PdS24Or0Dfc ) ran the same command as below with the same results, and thus deleted the txt records.

I was running Mint 19.3 and did an inplace update to Mint 20 April 2023 and afterwords the errors occured. Did these have any bearing on the errors I'm getting now? Perhaps there is a problem with acme-dns-auth.py as there are errors regarding acme-dns-auth.py in the output below. I don't know the root cause or causes for the failed renewal, so I'm asking for help!

DNS quires:
nslookup nutthause.com dns0.zoneedit.com
Server: dns0.zoneedit.com
Address: 64.68.198.83#53

Name: nutthause.com
Address: 184.179.76.127

nslookup -q=TXT nutthause.com dns0.zoneedit.com
Server: dns0.zoneedit.com
Address: 64.68.198.83#53

nutthause.com text = "_acme-challenge.nutthause.com=8rw3dccMTTIbI_YJ8hgYgpw_eF_iSiQ8PdS24Or0Dfc"

My domain is:
nutthause.com

I ran this command:
sudo /usr/bin/certbot certonly --manual --manual-auth-hook /etc/letsencrypt/acme-dns-auth.py --preferred-challenges dns --debug-challenges -d *.nutthause.com -d nutthause.com

It produced this output:
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator manual, Installer None
Cert is due for renewal, auto-renewing...
Renewing an existing certificate
Performing the following challenges:
dns-01 challenge for nutthause.com
dns-01 challenge for nutthause.com

NOTE: The IP of this machine will be publicly logged as having requested this
certificate. If you're running certbot in manual mode on a machine that is not
your server, please ensure you're okay with that.

Are you OK with your IP being logged?

(Y)es/(N)o: Y
Running manual-auth-hook command: /etc/letsencrypt/acme-dns-auth.py
manual-auth-hook command "/etc/letsencrypt/acme-dns-auth.py" returned error code 1
Error output from manual-auth-hook command acme-dns-auth.py:
Traceback (most recent call last):
File "/etc/letsencrypt/acme-dns-auth.py", line 4, in
import requests
ImportError: No module named requests

Running manual-auth-hook command: /etc/letsencrypt/acme-dns-auth.py
manual-auth-hook command "/etc/letsencrypt/acme-dns-auth.py" returned error code 1
Error output from manual-auth-hook command acme-dns-auth.py:
Traceback (most recent call last):
File "/etc/letsencrypt/acme-dns-auth.py", line 4, in
import requests
ImportError: No module named requests

Waiting for verification...

Challenges loaded. Press continue to submit to CA. Pass "-v" for more info about
challenges.

Press Enter to Continue -v
Challenge failed for domain nutthause.com
Challenge failed for domain nutthause.com
dns-01 challenge for nutthause.com
dns-01 challenge for nutthause.com
Cleaning up challenges
Some challenges have failed.

IMPORTANT NOTES:

  • The following errors were reported by the server:

    Domain: nutthause.com
    Type: unauthorized
    Detail: Incorrect TXT record
    "8rw3dccMTTIbI_YJ8hgYgpw_eF_iSiQ8PdS24Or0Dfc" (and 1 more) found at
    _acme-challenge.nutthause.com

    Domain: nutthause.com
    Type: unauthorized
    Detail: Incorrect TXT record
    "8rw3dccMTTIbI_YJ8hgYgpw_eF_iSiQ8PdS24Or0Dfc" (and 1 more) found at
    _acme-challenge.nutthause.com

    To fix these errors, please make sure that your domain name was
    entered correctly and the DNS A/AAAA record(s) for that domain
    contain(s) the right IP address.

My web server is (include version):
$ nginx -v
nginx version: nginx/1.18.0 (Ubuntu)

The operating system my web server runs on is (include version):
Linux Mint 20 Ulyana

My hosting provider, if applicable, is:
ZoneEdit Domains

I can login to a root shell on my machine (yes or no, or I don't know):
Yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):
No, linux cmd

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):
certbot --version
certbot 0.40.0

Any help here to resolve this and allow me to renew the nutthause.com certificates would be appreciated

2

Hi @linuxnutt,

Did you just manually download the file acme-dns-auth.py into your /etc/letsencrypt directory?

If you look at

it says:

Requires Certbot >= 0.10, Python requests library.

Maybe you had requests installed (manually or from the OS) in your old Linux distribution but don't have it in your updated distribution?

It seems to me that you're doing things very manually in your setup, but may not have read the associated documentation.

https://eff-certbot.readthedocs.io/en/stable/using.html#getting-certificates-and-choosing-plugins

The purpose of using acme-dns-auth.py is to use the acme-dns service (perhaps the original one hosted by its inventor, or perhaps your own or someone else's hosted copy) to create the TXT records that Let's Encrypt challenges you to create to prove that you own the domain names in the certificate. This is an alternative to creating them manually in your DNS zone. These TXT records' required contents are randomly generated and are different every time you renew your certificate, so if you create them manually (instead of automatically with acme-dns) you would need to create different ones in the DNS zone every time.

4 Likes
3

Upgrading this definitely won't fix your problem, but this release is almost four years old. If you don't have a newer OS package for Certbot available, you might want to use a newer recommended installation method for Certbot.

3 Likes
4

I see that you were able to obtain a wildcard cert:
image
I also see that it isn't being renewed on a regular schedule.

Were you able to automate this last renewal?
[I'm ignoring the use of the word "manual" in your posts and topic]

2 Likes
5

rg305, I have been using the documentation provided here:
How To Acquire a Let's Encrypt Certificate Using DNS Validation with acme-dns-certbot on Ubuntu 18.04 | DigitalOcean. After posting the help topic, when I noticed from the Digital Ocean tutorial, running sudo certbot renew - "The renewal process can run start-to-finish without user interaction, and will remember all of the configuration options that you specified during the initial setup." So yes it appears certbot renew provided the necessary logic and automation.
Anyway, I'm still at odds why didn't the command, sudo /usr/bin/certbot certonly --manual --manual-auth-hook /etc/letsencrypt/acme-dns-auth.py --preferred-challenges dns --debug-challenges -d *.nutthause.com -d nutthause.com renew the certificate?, but certbot renew did? As you pointed out their has been past renewals albeit not on a regular schedule. I have always used the above command. These host are located on a local domain behind a firewall with no 80 or 443 access from the internet.
Here the --manual argument is used to disable all of the automated integration features of Certbot.

6

Where can I find this new recommended instillation method for certbot? Yes mint 20 LTS 5 years support sunset 4/30/2025 . I like running stable versions.

7

Because it is set to "--manual".
Try that same command without it.

2 Likes
8

I cann't run this command now I'll have to wait until the near expiration of wild card *.nutthause.com certificate as these have already been renewed 10-9-2023?
sudo /usr/bin/certbot certonly --manual-auth-hook /etc/letsencrypt/acme-dns-auth.py --preferred-challenges dns --debug-challenges -d *.nutthause.com -d nutthause.com

9

When using --manual-auth-hook, as required for the acme-dns-auth.py script, you need to use the --manual plugin. And with the hooks, the --manual plugin is automatable.

2 Likes
10

So looks like we are gong in circles here yes I agree --manual is needed as explained in read the docs,
User Guide — Certbot 2.6.0 documentation, and how I have used it above to renew the certificate. But do we ACTUALLY KNOW WHY IT IS FAILING? This is my question. Why does sudo certbot renew work?

11

Lets have a look at the renewal config file:
in folder: /etc/letsencrypt/renewal/

3 Likes
12 
Lets have a look at the renewal config file:
in folder: `/etc/letsencrypt/renewal/

There is no "config" file in renewal, but there are 4 "conf" files:
/etc/letsencrypt/renewal$ ll
total 24
drwxr-xr-x 2 root root 4096 Jul 11 00:44 .
drwxr-xr-x 9 root root 4096 Jul 11 00:44 ..
-rw-r--r-- 1 root root 640 Jul 11 00:44 helios.nutthause.com-0001.conf
-rw-r--r-- 1 root root 650 Jul 10 18:34 helios.nutthause.com.conf
-rw-r--r-- 1 root root 640 Jul 10 18:34 nutthause.com-0001.conf
-rw-r--r-- 1 root root 554 Jan 22 2021 nutthause.com.conf

Which one are we looking at and what are we looking for? Apparently in the last renewal yesterday two were renewed:
The following certs were successfully renewed:
/etc/letsencrypt/live/helios.nutthause.com/fullchain.pem (success)
/etc/letsencrypt/live/nutthause.com-0001/fullchain.pem (success)
The following certs could not be renewed:
/etc/letsencrypt/live/helios.nutthause.com-0001/fullchain.pem (failure)

Additionally, the following renewal configurations were invalid:
/etc/letsencrypt/renewal/nutthause.com.conf (parsefail)

1 renew failure(s), 1 parse failure(s)

13

What shows?:
certbot certificates

3 Likes
14

What shows?: certbot certificates`

Ok here are the 4 conf files. How does this help us with figuring out why 
the renewal attempts were failing?
Two that renewed: This one is the wild card certificate *.nutthause.com
***
1
***
cat nutthause.com-0001.conf
# renew_before_expiry = 30 days
version = 0.40.0
archive_dir = /etc/letsencrypt/archive/nutthause.com-0001
cert = /etc/letsencrypt/live/nutthause.com-0001/cert.pem
privkey = /etc/letsencrypt/live/nutthause.com-0001/privkey.pem
chain = /etc/letsencrypt/live/nutthause.com-0001/chain.pem
fullchain = /etc/letsencrypt/live/nutthause.com-0001/fullchain.pem

# Options used in the renewal process
[renewalparams]
manual_public_ip_logging_ok = True
server = https://acme-v02.api.letsencrypt.org/directory
pref_challs = dns-01,
authenticator = manual
manual_auth_hook = /etc/letsencrypt/acme-dns-auth.py
account =  # some long number, I changed this not knowing its 
significance to others! #
***
2.
***
# cat helios.nutthause.com
cat: helios.nutthause.com: No such file or directory
root@www:/etc/letsencrypt/renewal# cat helios.nutthause.com.conf 
# renew_before_expiry = 30 days
version = 0.40.0
archive_dir = /etc/letsencrypt/archive/helios.nutthause.com
cert = /etc/letsencrypt/live/helios.nutthause.com/cert.pem
privkey = /etc/letsencrypt/live/helios.nutthause.com/privkey.pem
chain = /etc/letsencrypt/live/helios.nutthause.com/chain.pem
fullchain = /etc/letsencrypt/live/helios.nutthause.com/fullchain.pem

# Options used in the renewal process
[renewalparams]
authenticator = manual
server = https://acme-v02.api.letsencrypt.org/directory
pref_challs = dns-01,
account = # some long number #
manual_auth_hook = /etc/letsencrypt/acme-dns-auth.py
manual_public_ip_logging_ok = True

The two that didn't renew:
***
3 
***
# cat helios.nutthause.com-0001.conf 
# renew_before_expiry = 30 days
version = 0.40.0
archive_dir = /etc/letsencrypt/archive/helios.nutthause.com-0001
cert = /etc/letsencrypt/live/helios.nutthause.com-0001/cert.pem
privkey = /etc/letsencrypt/live/helios.nutthause.com-0001/privkey.pem
chain = /etc/letsencrypt/live/helios.nutthause.com-0001/chain.pem
fullchain = /etc/letsencrypt/live/helios.nutthause.com-0001/fullchain.pem

# Options used in the renewal process
[renewalparams]
pref_challs = dns-01,
authenticator = manual
server = https://acme-v02.api.letsencrypt.org/directory
manual_auth_hook = /etc/letsencrypt/acme-dns-auth.py
account = # Some long number #
***
4   It looks like this one can be deleted as was used back when tls-sni-01 
was a thing. How do I remove it?
***
# cat nutthause.com.conf 
# renew_before_expiry = 30 days
version = 0.31.0
archive_dir = /etc/letsencrypt/archive/nutthause.com
cert = /etc/letsencrypt/live/nutthause.com/cert.pem
privkey = /etc/letsencrypt/live/nutthause.com/privkey.pem
chain = /etc/letsencrypt/live/nutthause.com/chain.pem
fullchain = /etc/letsencrypt/live/nutthause.com/fullchain.pem

# Options used in the renewal process
[renewalparams]
authenticator = standalone
account = # some long number #
standalone_supported_challenges = tls-sni-01
server = https://acme-v02.api.letsencrypt.org/directory

**********
Do I need: (I would like to clear out the clutter if possible in 
/etc/letsencrypt directory as long as it does effect the functioning 
of Letsencrypt. )
 I probably only need:
/etc/letsencrypt/live/nutthause.com-0001/fullchain.pem as this
one is the wildcard certificate allowing me to create new subdomains
 as I deploy the different host etc. 

But, is there a security concern for
not having specific sub1domains listed on the certificate as I have
done early on with helios.nutthause.com.conf listing the seven
specific subdomains. Is there a security risk for not specifically 
having the subdomains listed?
15

With this:

certbot delete --cert-name nutthause.com

Best to not manually modify / delete files in the /etc/letsencrypt folders. Use Certbot commands only. You can easily cause problems difficult to repair.

As for below, please be patient. It's a little bit of a mess a volunteer needs to parse through the details so don't accidentally make things worse.

3 Likes
16

Yes, I agree. Your web server (services) run on your private network so I can't check which certs are used. But, if your services are configured to use your wildcard cert folder you could delete the other three using the certbot delete --cert-name X I showed earlier.

I don't know of any. But, even if you had a concern about a wildcard DNS entry and cert why make one?

3 Likes
17

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.