Detail: DNS problem: server failure at resolver looking up CAA for

greetings, i used to get my certificate by certbot-auto and everything was just fine until a month ago, when the renew turned into a problem. I tried to remove all certificates and start over, without success. can anybody give an advice or any help? Thank you.

My domain is: beth.stt.eesc.usp.br

I ran this command: ./certbot-auto -d beth.stt.eesc.usp.br

It produced this output:

./certbot-auto -d beth.stt.eesc.usp.br

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator apache, Installer apache
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for beth.stt.eesc.usp.br
Enabled Apache rewrite module
Waiting for verification…
Cleaning up challenges
Failed authorization procedure. beth.stt.eesc.usp.br (http-01): urn:acme:error:connection :: The server could not connect to the client to verify the domain :: DNS problem: server failure at resolver looking up CAA for usp.br

IMPORTANT NOTES:

  • The following errors were reported by the server:

    Domain: beth.stt.eesc.usp.br
    Type: connection
    Detail: DNS problem: server failure at resolver looking up CAA for
    usp.br

    To fix these errors, please make sure that your domain name was
    entered correctly and the DNS A/AAAA record(s) for that domain
    contain(s) the right IP address. Additionally, please check that
    your computer has a publicly routable IP address and that no
    firewalls are preventing the server from communicating with the
    client. If you’re using the webroot plugin, you should also verify
    that you are serving files from the webroot path you provided.

My web server is (include version): apache

The operating system my web server runs on is (include version): linux mint

I can login to a root shell on my machine (yes or no, or I don’t know): yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel): no

Hi @la0408,

Maybe it is because Let’s Encrypt can’t manage a large dns answer trying to query CAA for usp.br:

$ dig @8.8.8.8 caa usp.br

; <<>> DiG 9.10.3-P4-Ubuntu <<>> @8.8.8.8 caa usp.br
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 4608
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 69, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;usp.br.                                IN      CAA

;; ANSWER SECTION:
usp.br.                 3599    IN      CAA     0 issuewild "symantec.com"
usp.br.                 3599    IN      CAA     0 issuewild "telesec.de"
usp.br.                 3599    IN      CAA     0 issue "pki.dfn.de"
usp.br.                 3599    IN      CAA     0 issuewild "web.com"
usp.br.                 3599    IN      CAA     0 issue "telesec.de"
usp.br.                 3599    IN      CAA     0 issue "letsencrypt.org"
usp.br.                 3599    IN      CAA     0 issue "amazon.com"
usp.br.                 3599    IN      CAA     0 issue "netlock.hu"
usp.br.                 3599    IN      CAA     0 issue "pki.goog"
usp.br.                 3599    IN      CAA     0 issue "trustwave.com"
usp.br.                 3599    IN      CAA     0 issuewild "certsign.ro"
usp.br.                 3599    IN      CAA     0 iodef "mailto:stidomain@usp.br"
usp.br.                 3599    IN      CAA     0 issue "hongkongpost.gov.hk"
usp.br.                 3599    IN      CAA     0 issue "accv.es"
usp.br.                 3599    IN      CAA     0 issuewild "trustwave.com"
usp.br.                 3599    IN      CAA     0 issue "entrust.net"
usp.br.                 3599    IN      CAA     0 issuewild "globalsign.com"
usp.br.                 3599    IN      CAA     0 issue "www.pkioverheid.nl"
usp.br.                 3599    IN      CAA     0 issue "buypass.com"
usp.br.                 3599    IN      CAA     0 issue "kamusm.gov.tr"
usp.br.                 3599    IN      CAA     0 issue "sk.ee"
usp.br.                 3599    IN      CAA     0 issue "harica.gr"
usp.br.                 3599    IN      CAA     0 issue "symantec.com"
usp.br.                 3599    IN      CAA     0 issuewild "docusign.fr"
usp.br.                 3599    IN      CAA     0 issue "cfca.com.cn"
usp.br.                 3599    IN      CAA     0 issue "certsign.ro"
usp.br.                 3599    IN      CAA     0 issuewild "procert.net.ve"
usp.br.                 3599    IN      CAA     0 issue "identrust.com"
usp.br.                 3599    IN      CAA     0 issue "gca.nat.gov.tw"
usp.br.                 3599    IN      CAA     0 issue "comodoca.com"
usp.br.                 3599    IN      CAA     0 issuewild "quovadisglobal.com"
usp.br.                 3599    IN      CAA     0 issuewild "telia.com"
usp.br.                 3599    IN      CAA     0 issue "www.certinomis.com"
usp.br.                 3599    IN      CAA     0 issue "swisssign.com"
usp.br.                 3599    IN      CAA     0 issue "web.com"
usp.br.                 3599    IN      CAA     0 issuewild "wosign.com"
usp.br.                 3599    IN      CAA     0 issuewild "amazon.com"
usp.br.                 3599    IN      CAA     0 issue "disig.sk"
usp.br.                 3599    IN      CAA     0 issuewild "godaddy.com"
usp.br.                 3599    IN      CAA     0 issue "e-tugra.com"
usp.br.                 3599    IN      CAA     0 issue "telia.com"
usp.br.                 3599    IN      CAA     0 issue "wisekey.com"
usp.br.                 3599    IN      CAA     0 issuewild "startcomca.com"
usp.br.                 3599    IN      CAA     0 issue "camerfirma.com"
usp.br.                 3599    IN      CAA     0 issue "startcomca.com"
usp.br.                 3599    IN      CAA     0 issue "trustproviderbv.digitalcertvalidation.com"
usp.br.                 3599    IN      CAA     0 issue "d-trust.net"
usp.br.                 3599    IN      CAA     0 issue "wosign.com"
usp.br.                 3599    IN      CAA     0 issue "actalis.it"
usp.br.                 3599    IN      CAA     0 issue "globalsign.com"
usp.br.                 3599    IN      CAA     0 issuewild "comodoca.com"
usp.br.                 3599    IN      CAA     0 issuewild "letsencrypt.org"
usp.br.                 3599    IN      CAA     0 issue "docusign.fr"
usp.br.                 3599    IN      CAA     0 issue "secomtrust.net"
usp.br.                 3599    IN      CAA     0 issue "firmaprofesional.com"
usp.br.                 3599    IN      CAA     0 issue "godaddy.com"
usp.br.                 3599    IN      CAA     0 issue "edicomgroup.com"
usp.br.                 3599    IN      CAA     0 issue "certum.pl"
usp.br.                 3599    IN      CAA     0 issuewild "pki.goog"
usp.br.                 3599    IN      CAA     0 issue "procert.net.ve"
usp.br.                 3599    IN      CAA     0 issue "e-szigno.hu"
usp.br.                 3599    IN      CAA     0 issue "aoc.cat"
usp.br.                 3599    IN      CAA     0 issue "izenpe.com"
usp.br.                 3599    IN      CAA     0 issue "cht.com.tw"
usp.br.                 3599    IN      CAA     0 issue "quovadisglobal.com"
usp.br.                 3599    IN      CAA     0 issue "digicert.com"
usp.br.                 3599    IN      CAA     0 issue "fnmt.es"
usp.br.                 3599    IN      CAA     0 issuewild "digicert.com"
usp.br.                 3599    IN      CAA     0 issuewild "entrust.net"

;; Query time: 659 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Mon Feb 05 19:19:33 CET 2018
;; MSG SIZE  rcvd: 2269

You could try to add a CAA record for beth.stt.eesc.usp.br domain.

Anyway, @jsha or @cpu could give you more info regarding this issue.

Cheers,
sahsanu

2 Likes

Good catch, @sahsanu. Looks like that is the issue: https://unboundtest.com/m/CAA/Usp.br/LZDWZNZC.

We had a previous report of problems with big responses; I think this use case is worth fixing. I’ll file a Boulder issue. Thanks!

3 Likes

This set of CAA records also suggests what a huge and decentralized organization USP is!

3 Likes

We’re attempting to provision a cert for a subdomain of xyleme.com which appears to be suffering from the same issue: https://unboundtest.com/m/CAA/xyleme.com/UT37VSQ3

We’re thrilled to hear a fix is already being pursued and would love to follow along! Any chance you could post a link to the issue, @jsha?

@pd-aray

You can avoid the issue by making the domain’s CAA record set shorter, without losing anything.

It’s this right now:

0 iodef "mailto:........@xyleme.com"
0 issue "amazon.com"
0 issue "amazonaws.com"
0 issue "amazontrust.com"
0 issue "awstrust.com"
0 issue "letsencrypt.org"
0 issue "rapidssl.com"
0 issuewild "amazon.com"
0 issuewild "amazonaws.com"
0 issuewild "amazontrust.com"
0 issuewild "awstrust.com"
0 issuewild "letsencrypt.org"
0 issuewild "rapidssl.com"

Which results in a 626 byte DNS response, give or take.

But it’s not necessary to include more than one of Amazon’s domains. This would be equivalent, as long as Amazon doesn’t change up which domains they use in the future:

0 iodef "mailto:........@xyleme.com"
0 issue "amazon.com"
0 issue "letsencrypt.org"
0 issue "rapidssl.com"
0 issuewild "amazon.com"
0 issuewild "letsencrypt.org"
0 issuewild "rapidssl.com"

That would produce a response that’s ~420 bytes, so it wouldn’t trigger the Let’s Encrypt issue.

Moreover, when the issue and issuewild settings are identical, you can remove the issuewild ones, and it will work the same.

0 iodef "mailto:........@xyleme.com"
0 issue "amazon.com"
0 issue "letsencrypt.org"
0 issue "rapidssl.com"

That further would reduce it to ~314 bytes, though it’s not really necessary.

(I redacted your email address. It will probably get spam somehow anyway, but I don’t need to contribute to the problem.)

Edit: Incidentally, keeping the duplicate Amazon domains, but removing the issuewild settings, would reduce it to ~411 bytes.)

5 Likes

Thanks for the reminder. I actually forgot to file that issue the first time around; serves me right for responding that I “will” do something without having done it first. :slight_smile: Here it is now: https://github.com/letsencrypt/boulder/issues/3451.

3 Likes

Thanks for the tip. Reducing the number of CAA records seems to have solved the problem. I successfully renewed a cert some minutes ago.

@la0408 Could you try again, please?

1 Like

All good!
It worked like a charm. Certs renewed!
Thank you very much for all the help guys!
@gilmarmts @pd-aray @schoen @jsha @sahsanu@mnordhoff

Thanks for the excellent and actionable response, @mnordhoff!

I’m a little fuzzy on the details of the problem here, so I’d like to voice some assumptions based on the thread of conversation here and here as a sanity check. It sounds like:

  • boulder uses an internal instance of the unbound DNS server in production
  • boulder's DNS client is configured with a default message buffer capacity of 512 bytes
  • CAA record resolution for xyleme.com exceeds the DNS client’s message capacity (~626 bytes)
  • boulder reports a CAA record lookup server failure

To add a little context to our use-case, we’re an organization requesting certs on behalf of our customers. While we do have authority to serve SSL certs on specific subdomains for each customer, in most cases, we don’t have authority to modify their DNS records which are usually located on a parent domain.

Just looking for a little clarity so we can help educate our customers about potential DNS configuration improvements until the issue @jsha kindly linked is addressed. Thanks!

1 Like

You’re 99% correct, @pd-aray. However, to be really precise: the default limit of 512 bytes is actually a DNS feature rather than an internal buffer. In DNS, if you want to receive responses larger than 512 bytes from a server, you have to set a field in your request indicating that.

1 Like

The correction is much appreciated, thanks for the explanation, @jsha!

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.