In light of the recent (disappointing) announcement regarding ending OCSP support, what will happen to Must-Staple support? What should clients do to account for this change? The announcement makes no mention of Must-Staple.
Thinking of things like Certbot's --must-staple flag that people have hard-coded into their cron jobs etc.
Well, Let's Encrypt can't really honor must staple requests in the CSR once that change has happened. Stapling would become impossible and browsers would refuse the certificate without a stapled OCSP status.. I'm suuuure they also thought of that and would disable the must staple flag as an allowed TLS Feature once OCSPs get disabled.
Personally I think the whole "privacy" argument is flawed and is not a choice the CA should make for their users nor site operators. OCSP with purely stapled OCSP statusses and/or including must-staple is perfectly fine from a privacy perspective. Also, Firefox is the only browser, as far as I know, which still does OCSP requests, but this can be easily disabled.
Lowering the load/et c. on the CA infrastructure is another argument I cannot disagree with or not: that's purely up to the CA of course 
Yeah, so they can't just silently ignore this right? (Imagine expecting a cert with the strict property of Must-Staple yet getting a cert without Must-Staple.) It would have to break a lot of systems that use it?
Agreed. (Though, to clarify, that's not the discussion I meant to start, since I imagine this is an unpopular view.)
On another (related) note, this to me sounds like revocation is dying. Revocation has been broken for years, and while OCSP + must stapling is okay-ish in theory, in practice it was never well-supported (i.e. servers lazy loading OCSP, no proper retries etc.).
Passive revocation really seems like the way to go, so I'm looking forward to Let's Encrypts new profiles that will eventually allow for very short-lived certificates that do not support any revocation at all.
I'm not sure. I don't think web clients like browsers will expect (or not) a must staple TLS Feature. If ACME clients will break depends on how Let's Encrypt will disable the must staple feature: drop it silently or return an error?
Yeah, CRLs are definitely just a method to be in line with the BR without too much trouble compared to OCSP. Nobody (or at least no browser) is going to download the huge CRLs LE will be generating...
The browsers have recently stepped up their game with CRLs via a vendor mechanism like Firefox's CRLite, where they compress the CRLs and then push that to browsers via backchannels. While this works, it requires external infrastructure. Big browsers may have that, but for me TLS + PKI is more than just Chrome + Firefox - I'm thinking devices like IoT, server-to-server communication and friends that do not benefit of stuff like CRLite. For those you're correct - they rarely look at CRLs.
A multi-part answer:
This is a very early announcement of intent. We don't have all the details, such as how to deal with Must-Staple, worked out yet. There will be follow-up announcements as timelines and implementation details become firmer.
One possibility would be a transitional period in which we continue to operate OCSP services, but only provide responses for serials that were issued with the Must-Staple extension. This would allow us to greatly reduce the resources spent on OCSP infrastructure, but would not allow us to remove that infrastructure entirely, so it is not ideal from an operational complexity perspective.
Once we've turned off OCSP entirely, there are at least two paths forward for handling CSRs which request the Must-Staple extension: reject the finalize request, or accept the request and simply issue the certificate without that extension (much as we ignore requests for other extensions). Each of these has pros and cons, which we'll be weighing. It's also possible that our approach here will vary by profile. Again, we're happy to take input on what folks think the right path is, and we'll announce firmer details later.
Finally, the privacy risk of OCSP is very real. Without getting into details, CAs in general do get government subpoenas for data of various kinds. And perhaps even more importantly, querying OCSP is a cleartext request which can be passively monitored by entities other than the CA.
We look forward to a world in which revocation is unnecessary because all certificates have lifetimes shorter than an OCSP response has today. But even after we offer 6-day certs, it's going to take a long time to move the entire ecosystem in that direction, and we have to continue to function with both our costs and the whole internet's privacy in mind in the mean time.
Thanks for the reply Aaron. I'll be interested to follow the developments. I'm sure removing support for Must-Staple will be inconvenient for the operators who use it as it will certainly cause their automated systems to fail issuing certs.
What kinds of data are OCSP servers storing? 
Plaintext is not really an issue as the content is signed, so when privacy-preserving servers that actually implement stapling act as the clients, OCSP is a private, secure way to deliver certificate status, as it doesn't reveal anything surprising to observers. It's not like relying parties (browsers, etc) revealing which sites are being visited or anything like that.
Anyway, the privacy problem rests with clients' behavior regarding OCSP, not OCSP itself.
I do too, honestly. Let's do away with GB-sized CRLs too and just make cert lifetimes 7 days or less.
I wish that were true, but far from the case: client vendors are taking matters into their own hands and pushing their own CRLs because they want to control which certificates should be trusted. And revocation is their mechanism for doing that.
Any ideas what percent would require Must-Staple?
Because overall I'd guess loss of OCSP would have limited effect on such user-agents. They probably don't check OCSP anyway for the same reasons browsers quit (soft vs hard fail mostly).
Saying "plaintext isn't an issue when ocsp-stapling servers are making the requests", while factually true, does nothing to mitigate the fact that plaintext is a massive issue when end-users are making the requests, which is the vast majority of the time. Right now, today, browsers are revealing what sites are visited, and that's an issue that needs to be solved sooner rather than later.
There's nothing stopping MacOS and Windows from adopting the same CRLite-style infrastructure that Firefox/Safari/Chrome are -- they're software which implements a "please validate this cert for me" API with plenty of precedent for phoning home for updates. As ever, the story for Linux is more complex and fragmented 
So... ask them to stop?
It's mainly Firefox these days, right? Surely this is something Mozilla could discuss. Why cut off the privacy benefits of OCSP stapling from the many thousands(++) of servers that are legitimately stapling certs, and benefiting millions of end-user clients? OCSP stapling is a key benefit for servers to implement.
Borrowing Andrew Ayer's remarks from HN:
After I created OCSP Watch[1], I regularly detected CAs returning an OCSP response of unknown or unauthorized for certificates found in CT logs, indicating that they had basically forgotten that they had issued a certificate. I find that rather troubling. Indeed, OCSP Watch is currently reporting several forgotten NETLOCK certificates. (The certificates from other CAs are recently issued and will probably have OCSP responses provisioned in the near future.)
CRLs can't be used to detect this, because they only list revoked certificates rather than providing a definitive status for every issued certificate.
Ending OCSP leaves critical gaps in PKI reliability.
Even if that's not LE, this decision sets a precedent for other CAs going forward since LE is widely considered a leader in the CA space.
I would love to see any high-level aggregate stats you (or other CAs) could provide for what user agents are requesting OCSP.
We very purposefully try our hardest to never have that information.
*smacks forehead*
Of course! I feel silly for asking the question. Thank you for trying so hard!
From our privacy policy:
If your browser makes an OCSP request, our servers will automatically record your IP address, browser, and operating system in temporary server log files. We do not use data from OCSP requests to build profiles or identify individuals. Temporary server logs are used for operational purposes only and are normally deleted in less than seven days.
We don't normally have these logs turned on but we can temporarily enable them to gather operational data, which we may consider doing to answer questions like this. These are logged separately from the requests themselves to avoid pairing potentially identifying information like IP and user agent with the OCSP request.
Not really. I trust that the recent moves towards p256 as a default will free up enough capacity to move from 90 days to 30. And CRLs should work fine if we assume some sharding (even though I'm afraid they'll still be quite big to avoid privacy concerns, but "big" means megabytes here).
I just don't want to think about what the change management for moving to 30 days looks like.
anyway this means LE must include crlDistributionPoints as BR 7.1.2.11.2: and I kinda sure there are many thing will download CRLs without thinking about it: (like about everything that isn't browsers, and not sure if they cache crl at all)
You are assuming that the PKI is strained by key type, when really it's strained by CT logs.
And as Andrew said in the quoted text above, CRLs are not a comprehensive replacement for OCSP.
I was thinking of storage requirements, actually. It looks like they want to optimize for keysize, and it kinda makes sense.
Storage space isn't the constrant; rather, CPU and signing through the HSMs are much more expensive.