Subdomain CNAME being ignored when validating CAA

[jvito]

We are getting the following error when trying to validate an order:
"During secondary validation: CAA record for onu.edu prevents issuance"

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

DNS entries:

northernnetwork.onu.edu

engagevanity.campuslabs.com. 300 IN CAA 0 issue "letsencrypt.org"
northernnetwork.onu.edu. 86400 IN CNAME engagevanity.campuslabs.com.

engagevanity.campuslabs.com

engagevanity.campuslabs.com. 300 IN CAA 0 issue "letsencrypt.org"

onu.edu

onu.edu. 3600 IN CAA 0 iodef "mailto:certificates@onu.edu"
onu.edu. 3600 IN CAA 0 issue "comodoca.com"
onu.edu. 3600 IN CAA 0 issue "sectigo.com"
onu.edu. 3600 IN CAA 0 issue "trust-provider.com"
onu.edu. 3600 IN CAA 0 issue "usertrust.com"
onu.edu. 3600 IN CAA 0 issuewild ";"

My domain is:

northernnetwork.onu.edu

I ran this command:

We are using Certes;
So the code calls

   // context.Location is a string value of the format "https://acme-v02.api.letsencrypt.org/acme/order/75419561/264362963117"
   var order = Acme.Order(new Uri(context.Location));
   var authz = (await order.Authorizations()).First();
   var challenge = await authz.Http()
   await challenge.Validate();`

It produced this output:

"error": {
    "type": "urn:ietf:params:acme:error:caa",
    "detail": "During secondary validation: CAA record for onu.edu prevents issuance",
    "status": 403
  },

Source: https://acme-v02.api.letsencrypt.org/acme/authz-v3/348724285397

Due to the documentation, I would expect the challenge.Validate() to inherit the CNAME's CAA record, however the error seems to be using the parent domain instead.

My web server is (include version):

Azure Functions is running the code being executed above.

The operating system my web server runs on is (include version):

Windows(IIS)

My hosting provider, if applicable, is:

Azure

I can login to a root shell on my machine (yes or no, or I don't know):

yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):

no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):

Certes 3.0.4

[JamesLE]

I'm a little stumped by this one, but as an FYI to help get troubleshooting started, engagevanity.campuslabs.com is itself a CNAME. There's a whole chain of CNAMEs here:

• northernnetwork.onu.edu to
• engagevanity.campuslabs.com to
• se-infra-reverseproxy.azurewebsites.net to
• waws-prod-bn1-013a.sip.azurewebsites.windows.net to
• waws-prod-bn1-013.eastus2.cloudapp.azure.com

For most DNS implementations, a CNAME precludes the existence of any other records, so the CAA record on engagevanity.campuslabs.com is ignored. I think our behavior here will be to follow the chain of CNAMEs to its end, waws-prod-bn1-013.eastus2.cloudapp.azure.com, and then look for CAA records up the tree. There are none, so the CAA check should succeed.

Either something has changed in your DNS very recently, or some part of my understanding is wrong, probably the latter. My understanding of the rest was wrong; see below!

[aarongable]

During CAA checking, a CNAME is only followed once, for the exact name it exists on. The CAA spec doesn't then ascend from there; it only ascends from the original name.

So if you're checking "foo.example.com", where "example.com" is a CNAME to "example.org", the three CAA records that are actually retrieved are "foo.example.com", "example.org", and "com".

In your particular case, the CAA record checks would be:

• waws-prod-bn1-013.eastus2.cloudapp.azure.com (due to the chain of CNAMES)
• onu.edu
• edu

James' point about CNAMEs precluding all other records is important here: from the perspective of the rest of the internet (and per the DNS spec) the CAA record that you've placed on northernnetwork.onu.edu doesn't exist, since no other records can exist alongside a CNAME.

[petercooperjr]

@aarongable

"foo.example.com", "example.org", and "com".

I think you meant example.com for the middle one.

But agreed to your point, any DNS implementation that's trying to have both a CNAME and a CAA record for the same name is pretty broken.

[aarongable]

@petercooperjr

I think you meant example.com for the middle one.

I did not -- note that the hypothetical setup has a CNAME from example.com to example.org, which causes the CAA checking algorithm to look for CAA records on example.org.

[petercooperjr]

Whoops, I was reading too quickly and misunderstood the scenario. You are, of course, correct.

[rg305]

@JamesLE

I'm a little stumped by this one

Me too.
This is the piece of info that tripped me up:

@jvito

During secondary validation

How/Why was that missed during primary validation?

Aren't all DNS settings not created equal?

[jvito]

@JamesLE I am also stumped.

@aarongable
Maybe my question was not clear.
There is only a CNAME record on northernnetwork.onu.edu NO CAA.

I believe it is getting the CAA from the target url of the CNAME, e.g. "engagevanity.campuslabs.com".

The root domain, "onu.edu", definitely has CAA records, but we just want to issue a certificate for the subdomain "northernnetwork.onu.edu".

I also don't understand what you mean by "During CAA checking, a CNAME is only followed once, for the exact name it exists on."

Wouldn't that mean, in my particular case, the CAA checks would be?

• engagevanity.campuslabs.com (due to the CNAME only being followed once??)
• onu.edu
• edu

OR

If the first check is only going to get the CAA from "waws-prod-bn1-013.eastus2.cloudapp.azure.com", Why do all the tools to view CAA records only look at the first CNAME, then the parent, e.g. the order seems to be

• northernnetwork.onu.edu
• engagevanity.campuslabs.com
• campuslabs.com (has CAA for letsencrypt.org)
• .com

e.g.

1. northernnetwork.onu.edu CAA Lookup FIRST CNAME to engagevanity.campuslabs.com
2. engagevanity.campuslabs.com CAA Lookup has a CNAME, NO CAA
3. campuslabs.com CAA Lookup has CAA for letsencrypt.org
4. .com

All,
I don't know what other information to supply. Or how to troubleshoot this further.

This also works for other domains setup in this manner. We used to following domain (northernnetwork.ratebeans.today) to troubleshoot the issue on our end, before posting this help request.

northernnetwork.ratebeans.today has a CNAME to engagevanity.campuslabs.com
ratebeans.today has a CAA record, that is NOT letsencrypt.org

Thanks for your help and explanations. Admittingly, I am at the limit of my understanding of DNS, but have been learning a lot from this conversation and reading over the past few days, so thank you for that.

[jvito]

@rg305
I would also like to know the answers to those validation questions.

SIDE NOTE
In our code we check for CAA issues before sending a cert order, specifically that we have zero CAA OR a letsencrypt.org entry. And that DNS lookup passes OK.

// host == "northernnetwork.onu.edu"
     while (caaRecords.Count == 0 && !string.IsNullOrWhiteSpace(host))
            {
                var records = (await _dnsClient
                    .QueryAsync(host, QueryType.CAA))
                    .Answers
                    .CaaRecords();
                caaRecords.AddRange(records);

                host = string.Join(".", host.Split(".").Skip(1));
            }
         return new CaaCheckStatus
            {
                Valid = caaRecords.Count == 0 || caaRecords.Any(r => r.Value == LETS_ENCRYPT),
                RestrictedTo = caaRecords.Select(x => x.Value).ToList()
            };

[petercooperjr]

@jvito

I also don't understand what you mean by "During CAA checking, a CNAME is only followed once, for the exact name it exists on."

Meaning that the chain of CNAMEs is followed at each level, but further checks of the parents of the targets of the CNAME aren't done. That is, given:

1. a.example.com CNAME to a.example.org
2. a.example.org CNAME to a.example.net

Then for a certificate request for a.example.com a CAA will get checked for a.example.net, but further checks on example.org and example.net aren't done if a.example.net doesn't have a CAA record. In that case, only example.com and com are checked, in sequence. I can see how saying "only once" was confusing, it didn't mean "only one layer of CNAME", just that further checks aren't done on parent domains of the targets of the CNAMEs.

@jvito

If the first check is only going to get the CAA from "waws-prod-bn1-013.eastus2.cloudapp.azure.com",

Yes, because that's what the chain of CNAMEs for northernnetwork.onu.edu ends up at.

@jvito

Why do all the tools to view CAA records only look at the first CNAME

Well, the DNS server for engagevanity.campuslabs.com is horribly broken trying to serve both a CNAME and a CAA record for that name, so I'm not surprised tooling is inconsistent at trying to make some sense of it.

@jvito

I don't know what other information to supply. Or how to troubleshoot this further.

The DNS server for engagevanity.campuslabs.com needs to be fixed so that it isn't trying to send both a CAA record and a CNAME record. The CAA record allowing Let's Encrypt to issue needs to be at the end of the chain of CNAMEs, or on a higher level of the original name being requested.

[rg305]

@petercooperjr

... or on a higher level of the original name being requested.

[that should also NOT contain both CNAME and CAA records]

[jvito]

Thank you. This is clear.

Checking our DNS setup of engagevanity.campuslabs.com, and will revert back with more questions and status.

EDIT:

Checked our DNS for "engagevanity.campuslabs.com"

• we did have a CNAME and an CAA
  • makes us wonder why Cloudflare let's the configuration happen!
• we changed our CNAME to an A record and we successfully ordered a cert! yay.

Thanks to all who helped me solve this. (I was very impressed at the level of attention this thread garnered.)

We are not really excited about the A record, because IP addresses in Azure can change, but we can solve that in other ways.