LetsEncrypt renewal error - Error finalizing order :: While processing CAA, SERVFAIL looking up CAA

Help
1

I am attempting to renew a cert with 84 domains. When doing so, I get a version of this error every time:

Error finalizing order :: Rechecking CAA for “pvpnew.anet.live” and 10 more identifiers failed. Refer to sub-problems for more information. It errors for different domains every time.

The log subproblems look like this:

{
  “type”: “urn:ietf:params:acme:error:caa”,
  “detail”: “Error finalizing order :: While processing CAA for pvpnew.anet.live: DNS problem: SERVFAIL looking up CAA for pvpnew.anet.live - the domain’s nameservers may be malfunctioning”,
  “status”: 403,
  “identifier”: {
    “type”: “dns”,
    “value”: "pvpnew.anet.live”
  }
},

I’ve done this same renewal many times before. The domains don't have DNSSEC, and they don't have CAA records. The error always happens for between 3-10 domains, and has different domains

I am running this command sudo certbot certonly --webroot -d pvpnew.anet.live -d [83 other domains]

I am using certbot 2.8.0

Should I be doing something differently on the LetsEncrypt side? The DNS provider doesn't think anything is wrong on their side.

3 Likes
2

Hi @bennyt2, and welcome to the LE community forum :slight_smile:

The nameservers for the domain "anet.live" are:

anet.live nameserver = diva.ns.cloudflare.com
anet.live nameserver = duke.ns.cloudflare.com

The only thing I can think of ... is that there is a problem within CF itself.
[hard to diagnose, and even harder to prove]

4 Likes
3

Hi @rg305,

Thanks for your reply. My gut is that it's a CF problem too. Thanks for confirming I'm not doing something silly on the LE side.

The LE error is saying that it's processing the CAA, but there is no CAA, and never has been. Is there a simpler way to describe this error (e.g. this error really means that there is a DNS checking problem). Does LE always check for CAAs? This hasn't been a problem in the past. Thanks!

2 Likes
4

Yes, that error really means that there is a DNS checking problem. I took a quick look and couldn't see anything wacky, but I don't have time to really dig into it right now.

And LE (along with all other public CAs) needs to check for a CAA record. The DNS server doesn't need to have one, but it needs to successfully reply "no records" instead of an error.

I wonder, given you're talking about 84 domains, if it's just taking so long that the overall process is timing out. Let's Encrypt's servers have an overall timeout for the request, and if it hits that then it will report an error with the part it's currently working on, when really it's just that's as far as it got in the process.

It's also not impossible that there's some sort of rate limiting happening on the DNS provider's side, where it sees a bunch of requests coming in for all those domains all at once and throttles them in some way, though I wouldn't really expect that from Cloudflare.

And there have been changes in the last couple months with how Let's Encrypt checks DNS, particularly around large responses. But I wouldn't think that'd be what you're hitting, assuming those 84 names are actually each individually small responses.

As a workaround, you might want to look into separating out at least some of those 84 domains onto separate certificates. Sometimes having many smaller certificates is easier to manage than one big certificate, and this might be one of those times.

@jcjones: If you have a chance can you take a look at what's happening with this cert's DNS requests?

8 Likes
5

CAA record checking always happens. One thing you can try is adding a CAA record; if there’s some problem in the “no record” case, adding a record might work. I haven’t investigated this particular issue.

8 Likes
6

Looks strongly like rate limiting. Here are the CAA lookup times in seconds for the most recent one, sorted:

"_messagetimems","_messagetime","validationlatency"
"1706813373343","02/01/2024 18:49:33.343 +0000","10.804"
"1706813372630","02/01/2024 18:49:32.630 +0000","10.092"
"1706813371725","02/01/2024 18:49:31.725 +0000","9.181"
"1706813370584","02/01/2024 18:49:30.584 +0000","8.043"
"1706813370575","02/01/2024 18:49:30.575 +0000","8.043"
"1706813370417","02/01/2024 18:49:30.417 +0000","7.879"
"1706813370149","02/01/2024 18:49:30.149 +0000","7.609"
"1706813369832","02/01/2024 18:49:29.832 +0000","7.294"
"1706813369087","02/01/2024 18:49:29.087 +0000","6.534"
"1706813368880","02/01/2024 18:49:28.880 +0000","6.343"
"1706813367299","02/01/2024 18:49:27.299 +0000","4.7620000000000005"
"1706813365824","02/01/2024 18:49:25.824 +0000","3.286"
"1706813365820","02/01/2024 18:49:25.820 +0000","3.283"
"1706813365689","02/01/2024 18:49:25.689 +0000","3.148"
"1706813364683","02/01/2024 18:49:24.683 +0000","2.145"
"1706813364112","02/01/2024 18:49:24.112 +0000","1.581"
"1706813363916","02/01/2024 18:49:23.916 +0000","1.3780000000000001"
"1706813363816","02/01/2024 18:49:23.816 +0000","1.2770000000000001"
"1706813363629","02/01/2024 18:49:23.629 +0000","1.091"
"1706813363612","02/01/2024 18:49:23.612 +0000","1.074"
"1706813363595","02/01/2024 18:49:23.595 +0000","1.056"
"1706813363543","02/01/2024 18:49:23.543 +0000","1.005"
"1706813363374","02/01/2024 18:49:23.374 +0000","0.834"
"1706813363369","02/01/2024 18:49:23.369 +0000","0.832"
"1706813363363","02/01/2024 18:49:23.363 +0000","0.832"
"1706813363323","02/01/2024 18:49:23.323 +0000","0.774"
"1706813363311","02/01/2024 18:49:23.311 +0000","0.769"
"1706813363301","02/01/2024 18:49:23.301 +0000","0.762"
"1706813363299","02/01/2024 18:49:23.299 +0000","0.758"
"1706813363094","02/01/2024 18:49:23.094 +0000","0.562"
"1706813363075","02/01/2024 18:49:23.075 +0000","0.535"
"1706813363022","02/01/2024 18:49:23.022 +0000","0.483"
"1706813363009","02/01/2024 18:49:23.009 +0000","0.471"
"1706813362990","02/01/2024 18:49:22.990 +0000","0.451"
"1706813362924","02/01/2024 18:49:22.924 +0000","0.385"
"1706813362924","02/01/2024 18:49:22.924 +0000","0.386"
"1706813362894","02/01/2024 18:49:22.894 +0000","0.353"
"1706813362894","02/01/2024 18:49:22.894 +0000","0.356"
"1706813362893","02/01/2024 18:49:22.893 +0000","0.356"
"1706813362876","02/01/2024 18:49:22.876 +0000","0.337"
"1706813362867","02/01/2024 18:49:22.867 +0000","0.329"
"1706813362842","02/01/2024 18:49:22.842 +0000","0.306"
"1706813362825","02/01/2024 18:49:22.825 +0000","0.287"
"1706813362812","02/01/2024 18:49:22.812 +0000","0.274"
"1706813362794","02/01/2024 18:49:22.794 +0000","0.261"
"1706813362793","02/01/2024 18:49:22.793 +0000","0.256"
"1706813362787","02/01/2024 18:49:22.787 +0000","0.255"
"1706813362786","02/01/2024 18:49:22.786 +0000","0.243"
"1706813362784","02/01/2024 18:49:22.784 +0000","0.252"
"1706813362770","02/01/2024 18:49:22.770 +0000","0.232"
"1706813362770","02/01/2024 18:49:22.770 +0000","0.231"
"1706813362768","02/01/2024 18:49:22.768 +0000","0.23"
"1706813362763","02/01/2024 18:49:22.763 +0000","0.225"
"1706813362759","02/01/2024 18:49:22.759 +0000","0.218"
"1706813362759","02/01/2024 18:49:22.759 +0000","0.217"
"1706813362759","02/01/2024 18:49:22.759 +0000","0.227"
"1706813362756","02/01/2024 18:49:22.756 +0000","0.224"
"1706813362753","02/01/2024 18:49:22.753 +0000","0.204"
"1706813362752","02/01/2024 18:49:22.752 +0000","0.212"
"1706813362750","02/01/2024 18:49:22.750 +0000","0.208"
"1706813362745","02/01/2024 18:49:22.745 +0000","0.204"
"1706813362744","02/01/2024 18:49:22.744 +0000","0.195"
"1706813362729","02/01/2024 18:49:22.729 +0000","0.187"
"1706813362724","02/01/2024 18:49:22.724 +0000","0.183"
"1706813362720","02/01/2024 18:49:22.720 +0000","0.179"
"1706813362720","02/01/2024 18:49:22.720 +0000","0.182"
"1706813362720","02/01/2024 18:49:22.720 +0000","0.183"
"1706813362720","02/01/2024 18:49:22.720 +0000","0.182"
"1706813362711","02/01/2024 18:49:22.711 +0000","0.173"
"1706813362706","02/01/2024 18:49:22.706 +0000","0.167"
"1706813362705","02/01/2024 18:49:22.705 +0000","0.173"
"1706813362704","02/01/2024 18:49:22.704 +0000","0.163"
"1706813362703","02/01/2024 18:49:22.703 +0000","0.165"
"1706813362701","02/01/2024 18:49:22.701 +0000","0.163"
"1706813362699","02/01/2024 18:49:22.699 +0000","0.166"
"1706813362693","02/01/2024 18:49:22.693 +0000","0.161"
"1706813362692","02/01/2024 18:49:22.692 +0000","0.153"
"1706813362685","02/01/2024 18:49:22.685 +0000","0.147"
"1706813362679","02/01/2024 18:49:22.679 +0000","0.141"
"1706813362677","02/01/2024 18:49:22.677 +0000","0.135"
"1706813362674","02/01/2024 18:49:22.674 +0000","0.136"
"1706813362663","02/01/2024 18:49:22.663 +0000","0.114"
"1706813362628","02/01/2024 18:49:22.628 +0000","0.091"

The other queries timed out.

7 Likes
7

Thanks everyone for the excellent and prompt help! @petercooperjr I'll try your suggestion for smaller certs.

6 Likes
8

Smaller certs...
OR
Smaller validation groups.

Note: Once validated, LE can hold those up validations to 30 days.
So...
validate
validate
validate
...
validate
validate

Then get a cert with all those validated names on it :slight_smile:
[if you must have a single cert for that many names]

5 Likes
9

The CAA check is going to happen at issuance time though, which is the issue here. Splitting things up helps for validation but not CAA :confused:

7 Likes
10

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.