Clear Pending Authorizations

I can confirm for you that the rate limit adjustments for these are tied to the domain and should work the same across multiple accounts.

@cpu @pfg thanks for all the help and info you provided, things have been working great and we’ve put some things in place to allow us to switch to multiple accounts in the future.

Is it on the roadmap anywhere to be able to obtain the current rate limit counts for an account?

There’s an open issue for this, but to my knowledge it’s not on any public roadmap:

@cpu I’ve been watching our system over the last week and a half to see where authorizations could be piling up and have found something I was hoping you could help us with.

I grep’d all the authz URLs from our logs and ran a process to force a validation for any that were in a pending status. I’ve found several cases where the original authz and validation went through successfully but when I ran it through the manual process it was showing a pending status which doesn’t make sense to me.

Here’s the relevant logs and domains in question:

2016/11/16 01:58:02 [INFO][3297269676.composedb.com] acme: Obtaining bundled SAN certificate
2016/11/16 01:58:03 [INFO][3297269676.composedb.com] acme: received authz URL: https://acme-v01.api.letsencrypt.org/acme/authz/STHkEK_U99PyV2AM9Ooggl-EHnb3jJbirn-0ykbWUi0
2016/11/16 01:58:03 [INFO][3297269676.composedb.com] acme: Trying to solve DNS-01
2016/11/16 01:58:05 [INFO][3297269676.composedb.com] Checking DNS record propagation...
2016/11/16 01:58:56 [INFO][3297269676.composedb.com] sending validate for https://acme-v01.api.letsencrypt.org/acme/challenge/STHkEK_U99PyV2AM9Ooggl-EHnb3jJbirn-0ykbWUi0/345642834
2016/11/16 01:58:57 [INFO][3297269676.composedb.com] The server validated our request
2016/11/16 01:58:58 [INFO][3297269676.composedb.com] acme: Validations succeeded; requesting certificates
2016/11/16 01:58:58 [INFO][3297269676.composedb.com] Server responded with a certificate.

But when I ran it through our process to clean up pending authz it showed the following:

2016/11/17 13:25:00 [DEBUG][3297269676.composedb.com] https://acme-v01.api.letsencrypt.org/acme/authz/7LgTngwZrvIp0JNWC2qQYTxonGbyzMv59lk3Ag4TC-k: pending
2016/11/17 13:25:00 [DEBUG][3297269676.composedb.com] initiating validation for dns-01 on https://acme-v01.api.letsencrypt.org/acme/challenge/7LgTngwZrvIp0JNWC2qQYTxonGbyzMv59lk3Ag4TC-k/345642819
2016/11/17 13:25:00 [INFO][3297269676.composedb.com] sending validate for https://acme-v01.api.letsencrypt.org/acme/challenge/7LgTngwZrvIp0JNWC2qQYTxonGbyzMv59lk3Ag4TC-k/345642819
2016/11/17 13:25:01 [ERROR] validate error acme: Error 400 - urn:acme:error:connection - DNS problem: NXDOMAIN looking up TXT for _acme-challenge.3297269676.composedb.com Error Detail:
2016/11/17 13:25:01 [DEBUG] successful validation of https://acme-v01.api.letsencrypt.org/acme/authz/7LgTngwZrvIp0JNWC2qQYTxonGbyzMv59lk3Ag4TC-k

The authz 7LgTngwZrvIp0JNWC2qQYTxonGbyzMv59lk3Ag4TC-k does not seem to be related to the challenge STHkEK_U99PyV2AM9Ooggl-EHnb3jJbirn-0ykbWUi0/345642834. It belongs to the authz STHkEK_U99PyV2AM9Ooggl-EHnb3jJbirn-0ykbWUi0, which was created 54 seconds after the first authz and has status=valid. Looks like your client is somehow creating multiple authz for the same domain without actually trying to validate some of them.

@pfg thanks for pointing that out, it was a bad c/p from the log file. We do create to authz at a time in some cases where the FQDN is longer than 64 chars due to the current restriction on the Common Name. I’ve since updated the log snippets to reflect the correct authz URL.

I’m still confused. The first log snippet relates to authz STHkEK_U99PyV2AM9Ooggl-EHnb3jJbirn-0ykbWUi0. The second one seems to be for 7LgTngwZrvIp0JNWC2qQYTxonGbyzMv59lk3Ag4TC-k and shows your attempt to clear a pending auth by forcing validation (which is expected to fail) and which seems to have worked. Is the issue the “successful validation of …” wording? I assume that’s just an ambiguity in the client logging, meaning “validation request was successfully sent and received”, not “validation request was successful and returned a valid authz”.

@pfg sorry for the confusion on the last couple of posts but I’ve hopefully found the underlying issue. The library is not handling badNonce errors appropriately which is resulting in orphaned authorizations. Reading through the ACME spec, it makes reference to the case where a retrieved nonce may no longer be valid but without specifically stating the cases where that could be. While digging through our logs, we always get the badNonce error after a period of inactivity ~30 minutes so my assumption is there is a TTL associated with the nonces issued from boulder.

Any thoughts?

AIUI there’s no TTL as such, but the nonce service only holds a limited number of nonces, so they’d expire based on (overall) usage. They also get flushed whenever Let’s Encrypt restarts the WFE process, for example during code deployment. Clients should be able to recover from badNonce errors by obtaining a new nonce (the badNonce response contains a valid nonce :wink:) and resending the request.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.