Urgent Failed to renew Certs Secondary Validation Failed

Help
1

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

We have multiple failure on *.tenncare.tn.gov. and *.tcam.tn.gov. both are hosted in route53 and we are using certbot to renew certs. this worked falwless durng our september procurement. we started seeing this issue from lastweek.

My domain is: sito-web.tcc.tenncare.tn.gov

I ran this command: certbot version: 4.2.0 '--config-dir', '/tmp/certbot/sito-web_tcc_tenncare_tn_gov', '--work-dir', '/tmp/certbot/sito-web_tcc_tenncare_tn_gov', '--logs-dir', '/tmp/certbot/sito-web_tcc_tenncare_tn_gov', '--rsa-key-size', '2048', '--key-type', 'rsa', 'certonly', '--non-interactive', '--debug-challenges', '--verbose', '--agree-tos', '--email', 'test@example.com', '--authenticator', 'dns-route53', '--preferred-challenges', 'dns-01', '--server', 'https://acme-v02.api.letsencrypt.org/directory', '--force-renewal', '--no-eff-email', '--domains', 'sito-web.tcc.tenncare.tn.gov']

It produced this output: HTTP 200
Server: nginx
Date: Tue, 11 Nov 2025 16:48:14 GMT
Content-Type: application/json
Content-Length: 813
Connection: keep-alive
Boulder-Requester: 2792642036
Cache-Control: public, max-age=0, no-cache
Link: https://acme-v02.api.letsencrypt.org/directory;rel="index"
Replay-Nonce: 9cWC3FLMw4nUrwDpWU_dxg6F68WLPipbk_jaZv5op144zeKt6XE
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

{
"identifier": {
"type": "dns",
"value": "sito-web.tcc.tenncare.tn.gov"
},
"status": "invalid",
"expires": "2025-11-18T16:47:18Z",
"challenges": [
{
"type": "dns-01",
"url": "https://acme-v02.api.letsencrypt.org/acme/chall/2792642036/611341452276/-YTwKQ",
"status": "invalid",
"validated": "2025-11-11T16:47:39Z",
"error": {
"type": "urn:ietf:params:acme:error:dns",
"detail": "During secondary validation: DNS problem: query timed out looking up TXT for _acme-challenge.sito-web.tcc.tenncare.tn.gov",
"status": 400
},
"token": "4O5CedaRpqtrEcLwa3bh7HrVDpImpGbXE62ZJ-G6jBg",
"validationRecord": [
{
"hostname": "sito-web.tcc.tenncare.tn.gov",
"addressUsed": ""
}
]
}
]
}

My web server is (include version):

The operating system my web server runs on is (include version):

My hosting provider, if applicable, is: route53

I can login to a root shell on my machine (yes or no, or I don't know):

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): certbot 4.2.0

2

Please don't use this option if it's not necessary. It will nog "magically" fix any error and usually only leads to hitting rate limits and wasting Let's Encrypts resources.

This suggests some kind of geoblocking on your DNS servers is taking place. Let's Encrypt validats from multiple locations all around the world.

2 Likes
3

Your DNS is badly broken. This isn't directly related to getting certificates, other than that the CA needs to get data from DNS to confirm that you control the name, but it impacts anyone trying to connect to anything under tn.gov at all.

It looks like there are servers not responding to requests, and the domain record on .gov saying it should be DNSSEC-signed but it actually isn't signed. And maybe there are some NS delegations that are inconsistent. I'm honestly not sure where you should start to untangle that mess.

You need a working domain name before you can start worrying about trying to get certificates for it.

4 Likes
4

we are able to procure the certificate back in september for this same domain. do you think something has changed after that?. tn.gov is managed onprem by a different team. we have subdomain tenncare.tn.gov hosted in aws route53. and sometimes it is working and sometimes its failing. not today. but last week. do you think it is because of geo blocking? or propagation delay?. when we manaully add the txt record and while checking that. we are getting the resonpse back. but with certbot its failing

5

Possibly; I don't know of an easy way for me on the outside to find out what your DNS infrastructure used to look like a couple months ago.

Well, Let's Encrypt does need to check from multiple places around the world, and the "during secondary validation" part of the error message implies that from some locations (the "primary" one) it could in fact validate. But I think sometimes it's just that their DNS resolver can work around misconfigured DNS zones and sometimes it can't.

The only "delay" that might be applicable is the amount of time that it takes for your DNS servers to all process an update and get back in sync. The CA checks your authoritative servers directly. Route 53's update API allows one to see when that process is complete, and I believe the certbot Route 53 plugin uses that to ensure that the changes have happened before asking the CA to validate, so I don't think that would be a problem in your case.

Are you checking the authoritative servers directly, as seen by outside networks, with a resolver that understands DNSSEC?

4 Likes
6

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.