Can you please start using a header Caller-ID: or Bounce: (just bounce it back in the reply if it's in the request)
YOU use a nonce to keep track and make it a stateful conversation, and that's totally fine, but you don't honor anything in return (Set-Cookie: nor any other field), so (fe. with revokes) there is NO way of knowing to WHICH request you actually answered! (because Boulder-Requester only starts after you have a clue what it's about, understandably).
I have a client that will soon handle a few hundred a day, and if you want the world to keep the database clean (and ours), it would be a wise thing to at least help your customers!
Peter
PS. (It was pretty stupid from day 1 when HTTP was written in the nineties to NOT immediately include 1 field that is always bounced back, but that completely aside:-)
The semantics for matching requests with responses are well-defined in HTTP/1.1 and HTTP/2. With HTTP/1.1 it is just whatever was last on that TCP stream, or with pipelining, the server must respond in the same order the requests were sent off. With HTTP/2, you know from the stream ID.
I wonder, where are you encountering problems when talking to the ACME server?
Nonces are used to prevent replay attacks, not to associate requests with clients. The ACME spec intentionally doesn't define how nonces are scoped:
Other than the constraint above with regard to nonces issued in
"badNonce" responses, ACME does not constrain how servers scope
nonces. Clients MAY assume that nonces have broad scope, e.g., by
having a single pool of nonces used for all requests. However, when
retrying in response to a "badNonce" error, the client MUST use the
nonce provided in the error response. Servers should scope nonces
broadly enough that retries are not needed very often.
Somebody suggested to me that @PWNLUSA might be using one of those async HTTP libraries which don't expose the request context, during response processing. For example in this library, you don't directly have access to the request in the response callback. Pretty awkward ergonomics.
Even so, the solution would be to find a code structure where you can feed the request context in to the callback. Can't ask every website on the internet to carry a request ID around ...
I've used netty at $job, and you can clearly have state since you can simply add a response handler which carries the state you need. You don't need to do that if you don't want to, but then you can't complain if the API you talk to doesn't help you.
Just as a note, boulder does set a Boulder-Requester: 12345678 header on almost all API responses. The value of the header is the account ID of the requesting account, and could be used by an ACME client which is simultaneously managing multiple subscriber accounts. This header is set on revocation responses as well.
The revocation API has one quirk compared to other ACME APIs. A client can request revocation in either of two ways:
Providing the cert in question, the public key of the account which issued that cert, and signing the request with the account's private key (just like any other ACME API request); or
Providing the cert in question, the cert's public key, and signing the request with the cert's private key
In the former case, Boulder attaches the Boulder-Requester: header because it knows the account ID of the client requesting revocation. In the latter case, no ACME subscriber account has been involved in the revocation request: the request came directly from someone who controls the cert's private key. So in that one case, Boulder omits the Boulder-Requester: header, because there is no sane value to set it to.
Since it sounds like this software is a full ACME client (not just a mass-revoker for research purposes or something like that), and it sounds like revocation is the area in which you're running into the most trouble, I suggest you switch to signing your revocation requests with the ACME account key instead of the certificate key, and then you'll get Boulder-Requester: for free.
The new-acct call has the same problem. I'm now matching based on the mailto: you return, but the result is that we must use the CommonName and can't use 1 single email for it. Since our same software runs mail too, it's just a matter of rewriting the mail account, but handy is spelled different:-)
And that was EXACTLY why I request the mechanism to make it easier...
Right, part of software that does HTTP(s)/SMTP(s)/POP/IMAP/DNS/ACME/TCP/UDP/SSL/TIME/ and some others all in one.
Thank you! Solves 30% of the problem! Thank you for a serious and intelligent response!!
Similarly, just a small amount of digging shows that the NewAccount endpoint does set the Boulder-Requester: header when actually creating new accounts, but that the header is omitted when returning a pre-existing account. This appears to be a simple oversight, and one which I'm already fixing:
Stepping back a bit: I'm glad that this thread has been productive, and has resulted in one boulder-side fix and one your-client-side fix to resolve your difficulties. But this thread started off as "here's a solution to a problem, please implement it" without fully describing what the problem was (a situation sometimes known as The XY Problem). It took a bunch of back and forth to realize what the specific problems actually are, and to realize that there are other approaches which solve the problems in a better way.
Text communication is hard, and software engineering is hard. So in the future, it might be even more productive to start these conversations with a full description of the problem at hand (much like we request in the #help section!), so that the conversation can proceed from there.
For http/1 etc I'd recommend a new connection per request just as a browser would do. Trying to solve a problem that browsers couldn't is probably not a great use of your time and is more of an academic discussion than a fault in ACME or Let's Encrypt.
I'd also recommend using an established http/2 library rather than implementing your own, but I'm guessing you enjoy doing that
[Edit: I'm suggesting not use the boulder Id because other ACME CAs may not have that feature, so you lose compatibility]
I'm closing this topic for now. Thanks, everyone, for your contributions in answering the question here and trying to improve ACME client integrations.
