Unable to update challenge :: authorization must be pending

[petercooperjr]

I think the issue is only in staging; this caught the problem before it got deployed to production. (Which is, after all, the main point of the staging environment.)

[Nummer378]

@Osiris

Not sure what the current release cadence is, but it used to be once every week or maybe even once every two weeks.

It's typically around 1 week, but it doesn't appear to be a fully fixed schedule (for example, there has been a two week gap between this staging build and the previous one). Production is usually one build behind staging.

(I source my data from pulling the deployed build hourly: Let's Encrypts Boulder version history)

[aarongable]

We have merged the fix that Osiris linked above, and tagged a hotfix release which includes that fix. It should go to Staging soonish, and the current version which is exhibiting this broken behavior in Staging will not go to Prod.

In general, we release once weekly -- to Staging on tuesdays, and that same version to Prod on thursdays -- but we do not make any external commitments to that release cadence, and will regularly release more or less frequently than that depending on various circumstances.

[jcjones]

The hotfix to staging went out about an hour ago. Seems like the errors have died down in our logs.

[jsha]

Big thanks to @petercooperjr and @_az for the analysis. Y'all were exactly right, and it helped us zero in on the fix quickly.

[petercooperjr]

And thank you for all you do! (Especially since it looks like this is related to the extended DNS error messages which I think will be a big help to users eventually.) I'm guessing that even without us you'd have figured it out reasonably quickly anyway. Honestly kind of surprised that you wouldn't have had an automated test around an NXDOMAIN CAA record already, just because I know you usually cover all the bases.

And I would really appreciate it being clearer in the specs exactly what DNS responses are and aren't acceptable for CAA, since "errors" seem to not be but NXDOMAIN I guess sometimes is and sometimes isn't an "error". Seems like everything related to CAs involves piecing together bits of dozens of RFCs and other specifications; I don't envy your compliance department's job.