TXT records not found when renewing wildcard cert

[faxi05]

Hi there --

first of all, happy new year, and TIA for looking at this.

I have encountered an issue trying to renew a wildcard cert. I created both TXT records as usual (procedure has remained the same for years) and made sure they were showing at https://toolbox.googleapps.com/apps/dig/#TXT/_acme-challenge.icetestng.com. before advancing within cerbot.

However, certbot insists the DNS entries were not available. I have been repeating the process a few times in the last 48 hours, but the result is always the same.

What am I doing wrong?

My domain is:
icetestng.com

I ran this command:

certbot certonly --manual --preferred-challenges dns --server https://acme-v02.api.letsencrypt.org/directory --domains *.icetestng.com --domains icetestng.com --debug-challenges

It produced this output:

Certbot failed to authenticate some domains (authenticator: manual). The Certificate Authority reported these problems:
  Domain: icetestng.com
  Type:   unauthorized
  Detail: No TXT record found at _acme-challenge.icetestng.com

  Domain: icetestng.com
  Type:   unauthorized
  Detail: No TXT record found at _acme-challenge.icetestng.com

Hint: The Certificate Authority failed to verify the manually created DNS TXT records. Ensure that you created these in the correct location, or try waiting longer for DNS propagation on the next attempt.

Some challenges have failed.

letsencrypt.log:

2024-01-01 11:35:10,439:DEBUG:certbot._internal.error_handler:Encountered exception:
Traceback (most recent call last):
  File "/snap/certbot/3566/lib/python3.8/site-packages/certbot/_internal/auth_handler.py", line 108, in handle_authorizations
    self._poll_authorizations(authzrs, max_retries, max_time_mins, best_effort)
  File "/snap/certbot/3566/lib/python3.8/site-packages/certbot/_internal/auth_handler.py", line 212, in _poll_authorizations
    raise errors.AuthorizationError('Some challenges have failed.')
certbot.errors.AuthorizationError: Some challenges have failed.

2024-01-01 11:35:10,439:DEBUG:certbot._internal.error_handler:Calling registered functions
2024-01-01 11:35:10,439:INFO:certbot._internal.auth_handler:Cleaning up challenges
2024-01-01 11:35:10,440:DEBUG:certbot._internal.log:Exiting abnormally:
Traceback (most recent call last):
  File "/snap/certbot/3566/bin/certbot", line 8, in <module>
    sys.exit(main())
  File "/snap/certbot/3566/lib/python3.8/site-packages/certbot/main.py", line 19, in main
    return internal_main.main(cli_args)
  File "/snap/certbot/3566/lib/python3.8/site-packages/certbot/_internal/main.py", line 1869, in main
    return config.func(config, plugins)
  File "/snap/certbot/3566/lib/python3.8/site-packages/certbot/_internal/main.py", line 1600, in certonly
    lineage = _get_and_save_cert(le_client, config, domains, certname, lineage)
  File "/snap/certbot/3566/lib/python3.8/site-packages/certbot/_internal/main.py", line 131, in _get_and_save_cert
    renewal.renew_cert(config, domains, le_client, lineage)
  File "/snap/certbot/3566/lib/python3.8/site-packages/certbot/_internal/renewal.py", line 399, in renew_cert
    new_cert, new_chain, new_key, _ = le_client.obtain_certificate(domains, new_key)
  File "/snap/certbot/3566/lib/python3.8/site-packages/certbot/_internal/client.py", line 428, in obtain_certificate
    orderr = self._get_order_and_authorizations(csr.data, self.config.allow_subset_of_names)
  File "/snap/certbot/3566/lib/python3.8/site-packages/certbot/_internal/client.py", line 496, in _get_order_and_authorizations
    authzr = self.auth_handler.handle_authorizations(orderr, self.config, best_effort)
  File "/snap/certbot/3566/lib/python3.8/site-packages/certbot/_internal/auth_handler.py", line 108, in handle_authorizations
    self._poll_authorizations(authzrs, max_retries, max_time_mins, best_effort)
  File "/snap/certbot/3566/lib/python3.8/site-packages/certbot/_internal/auth_handler.py", line 212, in _poll_authorizations
    raise errors.AuthorizationError('Some challenges have failed.')
certbot.errors.AuthorizationError: Some challenges have failed.
2024-01-01 11:35:10,440:ERROR:certbot._internal.log:Some challenges have failed.

My web server is (include version):

nginx

The operating system my web server runs on is (include version):

Ubuntu 20.04

I can login to a root shell on my machine (yes or no, or I don't know):

yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):

no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):

certbot 2.8.0

[bruncsak]

@faxi05 welcome to the community!

Happy new year for you too!

You have way too many TXT records for the domain _acme-challenge.icetestng.com (I counted 55). A challenge record has no use after the challenge is verified. If there are too many accumulated, the verification of the new challenge will fail. Please clean-up all the records. The best practice is to remove the challenge TXT record immediately after the verification. Did you consider the DNS automation with a DNS plug-in?

[faxi05]

That worked right away, thanks a lot!

I guess learning something new is a perfect way to kick off a new year.

I think I have always misunderstood this output from certbot:

(This must be set up in addition to the previous challenges; do not remove,
replace, or undo the previous challenge tasks yet. Note that you might be
asked to create multiple distinct TXT records with the same name. This is
permitted by DNS standards.)

That probably relates to the TXT records created during one renewal process, right? I always assumed records from previous runs would be stored and checked again.

Thanks again!

[bruncsak]

Yes, it is for the same renewal process. One challenge record is for the wildcard domain the other one is for the apex domain.

[Osiris]

@faxi05 Please consider using a DNS plugin. I see some user on the internet has developed a Certbot plugin for SchlundTech: GitHub - couchtyp/certbot-dns-schlundtech: SchlundTech XML Gateway DNS Authenticator plugin for EFF's Certbot. Note that this would be a third party plugin without support from Certbot specific, although you could always open a thread here.

Alternatively you could use acme-dns. See GitHub - joohoi/acme-dns: Limited DNS server with RESTful HTTP API to handle ACME DNS challenges easily and securely. for more info about that. This is especially a good choice if your DNS provider doesn't offer an API (but yours does) or does not have proper user rights management for their API (e.g., one token can change anything in the DNS zone, which is considered a security risk. Preferably the API token used can only add/remove TXT RRs for the _acme-challenge label.)