BuyPass Staging - Failed Challenge stays "Processing"

Client dev
1

This threw me off during some integrated tests.

I failed a HTTP-01 Challenge against the BuyPass staging API.

The AcmeChallenge stayed on "processing". The AcmeAuthorization stayed on "processing". It appears that initial attempts to validate the challenge resulted in errors being created in the subproblems field.

I had to manually trigger the challenge, which led the authorization and order to transition to valid.

Does anyone know if this happens with other CAs, and if so which ones? I'm trying to figure out how to best handle this -- I was under the assumption this had to transition to invalid (as boulder and pebble do), but re-reading the RFC, there is no MUST and retries are mentioned. I have some code that defends against triggering a challenge that is processing, but that needs to be revisited now.

Also, for those that handle this - what is your general logic for catching it? My client was stuck in an endless loop, because I was polling for a status-change that never happened. I'm not sure what to analyze in the return payloads to determine that I've triggered this scenario.

4 Likes
2

Iirc zerossl did exponential backoff on failed challenges

2 Likes
3

I think in all cases it's best to have a max number of retries or max polling before you give up. Different CAs need different amounts of time and some are just much slower than others, but you can't trust any of them (!) to always transition an item to the next status.

5 Likes
4

AFAIK, this is expected behaviour for all CAs that support retrying challenges (server and/or client initiated). A challenge that is in invalid state can never be fulfilled, so if the server supports a retry, the challenge must stay in processing. This is also explicitly shown in section 7.1.6.

One possibility of handling this is to implement a polling timeout. If there is a problem document attached to a processing challenge, and the timeout has elapsed, you can treat the challenge like any other invalid challenge with a problem.

Regarding what other CAs support this: Posh-ACME has a feature comparison table: Assuming it is correct, 3 public/free CAs support challenge retrying (BuyPass, ZeroSSL, ssl.com)

6 Likes
5

I had a per-run limit, that was 30x with 3s timeouts.

I introduced , and was testing, a new routine that detects stuck orders and tries to complete them before doing automatic orders (renewals, queues, etc).

I’ll track state in the database to somewhat address this.

Yeah, rereading the RFC yesterday I saw all that. The annoying this is that there isn’t much spec or standardization about how the server indicates this is happening.

With BuyPass, I think I can track the number of subproblems to detect a change - which would indicate another failed attempt. I’m not sure about the others; but my initial thought is to associate a behavior with each CA and handle things appropriately.

I thought I was nearly done with the new version of my client; oh well!

This popped up while doing some pre-release testing. I have ‘http-01.a.example.com’ through z; and ‘dns-01.a’ through z. Each domain is set for that specific challenge, and I generate primary and backup certs off different CAs.

I’ve been trying to break the setup, to ensure I have the right reporting and tools to auto-recover or manually recover. I have been adding a lot of tools this week!

5 Likes
6

At the top of this thread you said the auth and challenge were both "processing". But, in your post at the CA Comparison thread (here) you have 'pending' for the Auth and 'processing' for the Challenge. Snip here:

Auth:
'identifier': {'type': 'dns', 'value': 'dev.aptise.com'},
 'status': 'pending',

Challenge:
'status': 'processing',
                 'token': 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX',
                 'type': 'http-01',
                 'url': 'https://api.test4.buypass.no/acme/authz/XXXXXXXXXXXXXXXXXXXX/1',

I posted a sequence in that CA Comparison thread showing that Buypass automatically revalidated the "pending" / "processing" (after 10 minutes) and went to "valid".

I posted there as an update for Ryan's table which said no CA other than ZeroSSL did that.

A discussion of all oddities of Buypass might get a bit long so thought I'd post here :slight_smile:

4 Likes
7

At the top of this thread you said the auth and challenge were both "processing".

Nice catch. Now I definitely need to store more data and figure out wtf is going on. It was definitely both "processing" before, and payload is a just a pprint of the json payload with a few ids redacted.

4 Likes