I was thinking about something @webprofusion said about running pebble in docker on Windows and wondering if it would be helpful if pebble ran natively on windows (maybe it does! I haven’t tried).
That got me wondering if there was anything else that would help client developers. Of course I can’t promise anything, but I would be interested to hear if anyone had anything they’d like.
Hi, I haven't tried it recently but it likely runs natively OK. docker is extremely well supported on windows (in conjunction with docker desktop) so really there's probably no need to do extra work just for windows. While it may not be familiar to all windows developers it's an easy way to have a predictable setup.
As an aside, it's possibly not super well known in the linux community but Windows has incredible support for linux based development. I can literally run a linux (even GUI!) version of an app just by picking WSL from a dropdown list and hitting run, with full debugging.
For client development, a full run through of composing the main API messages/conversations would help. So for instance if it provided sample account keys and the expected output of hashing and signing steps, example resulting json payloads before and after encoding etc. Just enough to register a new account, create an order, fetch authorizations, submit them and finalize the order. That in itself is probably quite a lot of work.
While the ACME spec does cover this in its own way, it's not really a substitute for developer documentation and stumbling blocks like proper encoding and signing (including json field order importance etc) perhaps cause some people to give up before they should. It's easy to assume/expect the person consuming an RFC is some kind of all round expert but the reality is likely to be different.
One thing that I’ve noted is that the ACME rfc doesn’t fully describe all the behaviour of boulder's implementation, such as how auth and order reuse works. Plus there’s features which aren’t supported. In that respect it would definitely make sense to have a less formal description of Let’s Encrypt, not just ACME.
A Warm Welcome to ASN.1 and DER - Let's Encrypt Is probably a good example of a less formal introduction that is structured towards teaching and not just specification.
Guide to best practices for ACME clients also comes to mind as another good acme client development resource, but the domain name has expired so they need to be accessed via docs/acme-ops.md at master · https-dev/docs · GitHub now
Yes that is good, it just doesn't tackle the practical parts of implementation - obviously once you have a working implementation then best practises will be useful!
Pebble runs fine natively on Windows last I checked assuming you have a working Go environment. It looks like the repo now has precompiled binaries which should remove the Go dependency if I understand correctly. Though the repo readme still assumes you're compiling the binary yourself. The biggest hurdles I had when setting it up originally revolved around just not being terribly familiar with Go rather than anything Pebble specific.
Isn't that what the Boulder divergences doc is for? Granted, it doesn't mention the auth/order re-use but it could.
Love this idea and would have loved it more if it existed when I was developing my client. Having known inputs and expected outputs makes it way easier to validate your code is behaving properly.
Kill ASN.1/DER/PEM with fire (and replace them with a simple, modern text-based format).
Goodness knows 99% of the complexity of implementing a Let’s Encrypt client (Site.js / lib / Auto Encrypt · GitLab) was jumping through hoops (and including megabytes of libraries) to make some parser somewhere happy when dealing with that unnecessarily complex trio of formats from the dark bowels of telecommunication history.
Everything else was fine. So I don’t really think there’s much you can improve without addressing this particular elephant in the room.
*exhales* 
That's not something Let's Encrypt is likely to be able to change. The entire world uses ASN.1/DER/PEM. Existing protocols demand it. You can't just change such a thing.
But you could abstract it away. Keep things plain text (JSON?) in the API and generate ASN.1/DER/PEM from that. You’d be saving folks making clients a huge amount of hassle that way.
I mean, take a look at this: Invalid signature on CSR when finalizing - #24 by stewe – you could make it so no one has to go through that again
(Says the guy currently trying to figure out why his staging tests are returning urn:ietf:params:acme:error:badCSR errors when nothing’s changed in the code.)
Maybe it depends on the programming language used, but I've never heard of any issue regarding ASN.1/DER/PEM before and there are a lot of ACME clients out there.
And from the thread you're linking to: "I use a self made class to build ASN.1 objects (…)" Well, don't write your own class to build ASN.1, let other dedicated people do that for you 
This can be pretty tricky! If you post a new thread with a test CSR, we may be able to help!
Getting ASN.1/DER right can be pretty tricky, and there have definitely been many bugs through the ages, but the CSR format is well-understood now. A "simple text-based format" may have even more problems -- Just go look at how many issues JWTs have had. I don't think anyone loves the CSR, but it's unlikely that changing ACME itself so fundamentally is something that's likely to happen soon.
Hey Matthew, I was not accurate enough in my phrasing. I didn’t mean that Let’s Encrypt should create a new format for serialising the certification information in. What I meant to say was that the API could accept just the certificate data in a plain JSON object and then render the CSR, etc., on Let’s Encrypt’s end. Thereby saving developers a world of complexity in their clients.
e.g., Instead of creating a CSR, why can’t I just send you an object like this:
// And maybe even with better key names, like keyType instead of kty, etc.
{
domains: [ 'dev.ar.al' ],
key {
e: 'AQAB',
kid: '-rW1mW8TI8FvgTHaMt-ZL4XVqk-jdPYKUreZKjpuM50',
kty: 'RSA',
n: 'xjZKRkLmNUVDB9bWRSCihOET-ZvwS_nCaPkg35SzfPd6wZHXyKsSItyT8NxfAXasVmF1KPIPb4FBXBvI9aYKjVN4blAX9O4WViHyD6HnBAvTE6E-xCDkqTQflfUm0SWYKPu0jYXTaN5-XrVBBzpAza2JEfcZIa8wsWZfeUqBR2HOV-9YK3H0RU3mBtgHugUX6PzLKCmQECtpKfceliV4ROt1WFunpOPVE8cML3eKhJ0Sk_TRAJMb5AWxG4ID2t_Oj0Nr6SyuovbJNmoapDcsl5nSaO0_y7Z05vS2oWJhq9UXc_9s0x7mMOh6RvDeJJl4WzojVINtXP1ifiHxYP24DQ'
}
}
(That’s the request whose CSR is failing with 400 urn:ietf:params:acme:error:badCSR Error finalizing order :: signature algorithm not supported, by the way.)
This would literally shave megabytes off of some code bases as it would mean you could remove the heavyweight frameworks that do this.
Perhaps not a big issue for Big Tech but for Small Tech and Small Web, that would make a huge difference.
(Also: thanks for the offer. I’m going to hit my head against this a bit more today and if I’m still stumped, I might just take you up on that. Appreciate it.)
If that's your issue, maybe you should change programming languages.
For example, Certbots acme library, written in Python, uses PyOpenSSL for the interface between Python and OpenSSL, which is just 55.8 kB in size (pyOpenSSL-22.0.0-py2.py3-none-any.whl).
I wouldn't know why a CSR generating library would be megabytes in size, except due to choices of the programming language and/or library itself. Of course OpenSSL has a certain size of its own, but using OpenSSL instead of doing everything itself is something a library in another language than Python could do too of course.
If that's your issue, maybe you should change programming languages.
I was responding to a thread that asked how Let’s Encrypt could make developer’s lives easier. I shared my own experience and my own recommendation. Changing programming languages and using a 1MB external tool instead of a 1MB library doesn’t solve the issue I’m attempting to address.
I believe I’ve said all I can on the matter so I’ll leave it here.
Your JSON version of the CSR does not include proof that you are in possession of the private key for that public key. One of the main points of the CSR is to include that proof (by signing the CSR with the private key for the public key it includes).
Of couse you could extend that JSON, but in the end you will end up with a complex thing that does a subset of what the CSR format does, but there is no existing library support for creating these things. Which in the end is forcing everyone to invest a lot more work. (Which isn't an issue for Big Tech, but for Small Tech and Small Web. Which is exactly what you wanted to avoid.)
Regarding the signature algorithm not supported: maybe you are using SHA-1, which has been deprecated for a looooooooooooooong time and finally got disabled recently? Rejecting SHA-1 CSRs and validation using TLS 1.0 / 1.1 URLs
Thanks, it’s a 2048-bit SHA256 key so I don’t think that’s the problem (and I appreciate your help but I don’t want to hijack this thread with debugging my issue. I’ll open a separate one if I can’t figure it out today.)
Nope, despite what my own comment says, the signature is apparently sha1. So, yeah, assuming that’s the issue. Thanks again.
PS. Pebble version 2.3.1 (which, as far as I can see is the latest release – Releases · letsencrypt/pebble · GitHub), is not catching this. Opened an issue here: Pebble should fail CSR requests signed using SHA-1 · Issue #384 · letsencrypt/pebble · GitHub
> openssl req -in test.csr -text -noout
Certificate Request:
Data:
Version: 1 (0x0)
Subject:
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (2048 bit)
Modulus: …
Exponent: 65537 (0x10001)
Attributes:
Requested Extensions:
X509v3 Subject Alternative Name:
DNS:dev.ar.al
TLS Feature:
status_request
Signature Algorithm: sha1WithRSAEncryption
Signature Value: …
Yeah, I suspect that one of the biggest things Let's Encrypt could do to make client dev easier is to ensure that there are clear test cases for the ACME spec (or at least for their implementation of it), including making changes in Pebble before even putting them into Boulder staging.
I'm not sure if this would make client dev easier, but it would definitely make it better. So +1 on this.
I think it would be great if there should be a list/repo/etc of official test scenarios that a client should implement in their test suite, and pass or otherwise gracefully handle.
The issues I encountered when developing a client:
The issues in the ecosystem I noticed:
Yes, in the early days of the ACME protocol I seem to remember that we did want to potentially get rid of ASN.1—I think I suggested that we could invent our own totally new way of cryptographically binding subject keys to requests!—but someone¹ pointed out that many people would have CSRs generated externally, or the ability to generate CSRs externally (e.g. from an embedded device, or from a hosting provider, or from a CDN). As a result, ACME ended up using CSRs inside the protocol and explicitly not requiring any other proof of possession of the subject key. (ACME does provide a cryptographic binding between the ACME request and the CSR, though.)
Having CSRs in the protocol is annoying and possibly unfortunate, but it dramatically increased compatibility of ACME CA services compared to the alternative.
External CSRs aren't the best from an automation point of view, so that's a tension (like, if the subject key is being generated outside of the ACME client itself, you have to find a way to cause the CSR to be generated at the appropriate time, and a way to copy it into the machine running the ACME client!), but they do have potentially cool security properties. For example, you could generate a keypair in an HSM or something and then use a separate machine to obtain a certificate referring to that keypair, without ever exporting it anywhere, and without requiring the ACME client to be trusted with any access to the private key material. Reducing trust in the ACME client is probably a desirable thing in some settings!
¹ Edit: I think the Mozilla contributors had a whole exercise of gathering requirements, or at least preferences, from various web hosts somehow, which I don't think I was privy to. They definitely ended up having a lot to say about what different certificate subscriber end-users would want out of ACME!