Boulder new-cert - retry after x seconds

Yep, the status code weirdness is a known bug (probably not documented as a divergence from the spec since it's not intentional):

I would recommend polling the /acme/authz/ resource rather than /acme/challenge. This avoids issues due to authz reuse, see the last paragraph of this post: