Orders stuck in `processing` state


#1

Hello,

We have several certificate orders that crashed around August 24th at 00:20 CET with error:

Error finalizing order :: Unable to meet CA SCT embedding requirements.

The orders are now stuck in processing mode. I investigated a few of those and I saw the authorizations appear to be valid, still the order is not moving forward.

I tried to submit them for finalization again, but it failed with error:

Order’s status (“processing”) is not acceptable for finalization

The affected orders are associated with the following names:

www.omarmohammad.nl
www.chickenq.co.uk
www.chickenq.com
www.redseafishbar.co.uk
www.sharonmemorialhall.net
www.wildcatgeographic.com
www.repeat.site
www.crohns.ai
www.cvng.io
www.svir.space

How can we proceed?


#2

Hi @weppos

looks that one of the Certificate Transparency logs had a timeout.

Isn’t it possible to create a new order?

Test it manual with one domain and the --stage - option.


#3

Indeed I can try, but I want to know if this is the path to follow. That would still leave the previous order in processing limbo.


#4

Every order has an expiration date. So I don’t think this is really a problem. And client developer may produce a lot of invalide or pending orders.


#5

Indeed. However invalid is a final state. pending is a hold state (it’s waiting client input). processing is an active state and it’s on the LE side. Hence I’m also wondering if there is any sort of self-healing mechanism on LE side.

In other words, more generally, I wonder what should I do whenever a cert stays in processing for a while. So far we treated processing as a short-living state, hence this issue caught us unprepared.


#6

Same thing you do when polling each challenge resource, probably. It pays to be defensive and enforce your own local deadline for all polling operations (UpdateChallenge and FinalizeOrder) according to your system’s requirements, rather than relying entirely on Boulder. Abandon the order if either deadline is exceeded.


#7

:wave: Hi @weppos!

There isn’t yet :disappointed: - we have an issue in the backlog to address that: https://github.com/letsencrypt/boulder/issues/3427

If you deactivate an associated authorization the order should become deactivated as well. That’s one suggestion I can think of but otherwise abandoning the order to wait for its expiry is acceptable. NewOrder requests won’t reuse an existing processing order.

Apologies, I know this is a subpar experience!


#8

No worries. As long as creating a new order is an acceptable practice, it works for us.
I also thought about invalidating an authorization. However, the new order would work better as in case all the authorizations have been validated (like the cases I am referring to) the issuance of the new order is almost immediate.


#9

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.