I noticed that, from a relatively small sample, it seems that ACME clients don't tend to verify the certificates they receive from the CA. The opportunity for abuse is low, but a malicious CA could still take advantage of the trust established in the earlier stages of the process.
wouldn't it make clients impossible to test with staging/pebble/private CA like smallstab/ as those will use untrusted cert for obvious reason? I think they may want to check the parse of certificate so make sure webserver won't crash on reload, but is should not verify against trusted anchor.
I also had a short look into hardening ACME clients but from a slightly different angle. I wanted to see whether there were any RCE or unauthorized file read/write opportunities by feeding acme.sh and Certbot malicious responses from the ACME server. I didn't find anything, but didn't look very hard either.
I think the modifications you've suggested make sense, though OCSP could be a bit iffy as involving an extra service necessarily makes the overall issuance process less reliable.
It also relies on client authors to correctly implement the validation routines, which honestly, is a big ask. You only need to look at the number of involved standards and the UX of openssl to realize that it's quite easy to make wrong assumptions about how things work.
Having a good reference implementation would go a long way to mitigate that.
The PR I put in for acme.sh overcomes this by letting you specify a root cert when performing verifications, which for testing/staging could be anything. I tested on LE staging, after downloading staging's root.
In production, this isn't necessary and the OS or environment's trust store/CA list can be used instead.
I agree with you on those points, thanks for sharing.
I think the checking cannot be "all or nothing", as it will depend on the use case. Some checks may be soft, while others might be hard. Defining how to do that across the board is... challenging.
The changes I proposed to acme.sh struggle with the limitations of interacting with openssl via command line. If I were to do even more extensive checking, it probably becomes quite fragile. I think clients that use more reliable APIs, such as those written in python and golang can do a more thorough job.
Ah I see. Then it seems one would need to fetch the root in preparation for the test. I'm not sure if there's a consistent approach available for this?
We were talking about this question during the early design of Let's Encrypt, in terms of to what extent the client trusts the CA, and for what purposes, and whether there is any risk from a CA-in-the-middle attack. But I'm not sure that anyone followed up on the overall threat modeling question about this trust relationship, especially after the ACME protocol itself went through a redesign.
It would be good to fuzz ASN.1 implementations some more.
+1 on everything you wrote. I noticed all these things years ago, though my concerns were not in terms of security, but about ensuring the client and server act as expected and we get the right certificate. IMHO, the amount of automated testing in Certbot and Letsencrypt is not anywhere close to what I would like it to be and there aren't any tests that cover certain concepts.
Also worth noting to your concerns: I think a large problem with the Ecosystem is the ACME Specification covers a "Protocol" which is implemented by Servers and Clients, who have increasingly diverged in ownership. Client Developers have often experienced issues stemming from how a Server Developer interprets the spec; because they have opposing roles in the spec, a strict interpretation from one side is often a lax interpretation from the other (and vice versa).
Hopefully in the next few days I'll have had time to propose a couple of additional paragraphs, at least. But if anyone else has any inputs or recommendations, do share and I'll take them into account.
To me, one of the (most) harmful things a malicious CA could do is replace the public key in the issued certificate, which wouldn't mesh with the private key. This could cause serious disruption (instant DOS). The harm can be mitigated of course by ensuring that the public key in the new certificate and the public key in the corresponding CSR (or private key corresponding to both) actually match.
Honestly though, IMO a malicious/hijacked CA could potentially cause about the same level of harm as a buggy/broken CA. I would think that the process of meshing certificate to private key (and verifying SANs and signature) would be mostly sufficient to mitigate most "bad issuance" threats. I suppose the CA could crash or hang the client (or cause a badly-written client to spit out excessive traffic).
Can you trust a malicious CA not to exploit/publish its root certificate(s)?
