That is good.
this sounds confusing:
So, who does the TLS decryption?
It sounds like the cert is not being updated on the backend system - and that system might only be doing HTTP.
If so, then the WAF is the only one using the cert.
[please correct me where I'm wrong AND/OR confirm my assumptions]
Apparently the WAF decrypts the connection to inspect content (and obviously need to be), then forward to backend again as HTTPS. That's what I am told how to configure my backend. (I am told to open HTTPS port on 443, which surprises me). I am not sure if the WAF validates the backend HTTPS certificate.
And let me clarify about port 80, typed too fast before. Apparently the port 80 is intercepted and will generate HTTPS redirect by the WAF itself. Port 80 never passed to backend.
BTW google's proposed lifetime (max 7 year from cert notbefore) and MicroSoft's current requirement (minimum 8 years/ max 25 year of cert life left when requested) makes no cert can be in both trust store
LE's load would be relatively identical, because it doesn't really matter whether it's the client or the server that requests OCSP. Heavily client-requested OCSP responses can be easily alleviated via caching.
It's actually a mixture nowadays. They have a redis caching layer that is capable of live-signing, if needed. Though if I recall correctly, they do still sign ahead-of-time to avoid load spikes, where lot's of "fresh" OCSP requests are made simultenously, which could overload the OCSP responder.
OCSP is not only a performance problem in high issuance CAs like Let's Encrypt, it's also a compliance nightmare: Around 32% of all Let's Encrypt incidents on Bugzilla involve OCSP failures.
Revocation never really worked, so it's probably time to let it go for good. There has been a recent movement with the backend-driven CRLs, but for the ecosystem at large - beyond just the major browsers - no revocation and shorter lifetimes is probably the way to go.
However, I think the proposal is overly optimistic: The current proposal has a huge potential for lasting problems that will have a far greater reach than just the browsers.
It starts out with all of the small problems, like ACME clients being hardcoded to 90 days (sigh...), but this all fairly easily fixable in comparison. Problems start appearing once we continue reading the proposal further, because it wants to do much more than just reducing leaf certs to 90 days:
a maximum “term limit” for root CAs whose certificates are included in the Chrome Root Store. Currently, our proposed term duration is seven (7) years, measured from the initial date of certificate inclusion. The term for CA certificates already included in the Chrome Root Store would begin when the policy introducing the requirement took effect. CA owners would be encouraged to apply with a replacement CA certificate after five (5) years of inclusion, which must contain a subject public key that the Chrome Root Store has not previously distributed. For compatibility reasons, CAs transitioning out of the Chrome Root Store due to the term limit may issue a certificate to the replacement CA.
I like this idea, but I can't see how this can work on a global scale. Chrome wants this for their own Root Program, and for them it's easy: They already have their own platform verifier*, so for them it's easy to phase-out roots. Anyone who runs a 5-7-year old Chrome version has made questionable decisions already.
However, I think it's narrow-sighted for a root program to just consider their own browser: The CAs included have a userbase that reaches beyond just that single browser. For example, much of the Linux world relies on Mozilla's trust store via lists pulled from NSS. Mozilla has discouraged such behaviour**, but nevertheless this is the current state of the world. There are a bazillon tools out there that are not-browser based but still communicate using TLS: Scripts like curl, server-to-server applications, IoT devices, API clients, mobile and desktop applications and much, much more.
The great majority of these clients rely on a platform-provided trust store today, and those are insanely slow to update. ISRG Root X1 has only been included in Android 7.1.1, which was released over a year after ISRG Root X1 applied for inclusion. The major fragmentation and slow update rate of Android devices is the reason why we even have the "long chain" - a workaround for a problem that wouldn't exist if we were capable of updating platform trust stores.
It gets even worse when we look at ISRG Root X2: Google appears to have totally neglected their Android trust store and failed to ship any trust store updates in Android 13 at all. It is currently aimed for Android 14, close to 3 years after the inclusion request has been made. ISRG Root X2 is going to be 3 years old before even 10% of Android consumers have ISRG Root X2 in their trust store. This makes Google's proposed 2 year transition period absurd, since it doesn't even work with their own operating system.
This problem plagues much more systems than just Android though: Apple also only ships trust store updates via major OS upgrades, so if you don't have the latest and greatest iOS/macOS, you're out of luck: No TLS for you. Oracle hasn't moved either, ISRG Root X2 is not in Java's trust stores. Pretty much all embedded devices all tell the exact same story.
If Chrome requires roots to be limited to 7 years, either we will leave everything non-browser dead in the water or CAs will have to create cross-signed infrastructures: Long lived roots for non-Chrome, short-lived roots for Chrome. This however adds much more complexity and is actually just a workaround for the underlying problem.
The reason why root programs want shorter lived roots is cryptographic agility. The future - especially regarding quantum computing - is uncertain and switches may have to be made fast. This is currently fundamentally impossible with the slow-moving PKI ecosystem, hence efforts are being made to fix that.
Though I think if Google wants to be serious about this, don't just make new rules and require everyone to fix their trust stores: They own an operating system, so they should show us how it's done and implement proper trust store updates first - independent of major OS upgrades. This should be a background task...
*(https://community.letsencrypt.org/t/chrome-root-program-and-verifier-have-launched/183799) [if you can't read the topic: Chrome v105+ no longer relies on a platform-provided trust store]
**Actually they have just discouraged using the trust store for non-browser/non-SMIME/mixed purposes. Still, various tools like OpenSSL don't really have a concept of purpose-specific roots.
My conclusion to this proposal is: It's a good idea overall, and some of the points can already be brought forward today. This includes the 90-day proposal: To me, it feels like the easiest step out of the tree major things proposed and it should be doable with only moderate friction. It does involve end-users though, so sufficient time for transition periods should be given: Make it clear that this must be a priority, but allow companies to adjust their flows.
At the same time, we should invest in revisting how we do trust stores today. It is obvious that the current state won't work in the future - actually, it already doesn't work today, as we all saw when DST Root CA X3 expired - but blindly forcing shorter lifetimes and killing everyone's TLS isn't going to do any good either. We should actively discourage static trust stores that cannot be easily updated, propose good alternatives, implement them and then go for short lived roots.
well, they finally started to update root store by play store at the least
It seems like that's inevitable at this point that we're going to have shorter lived roots. We will introduce new roots (say, a new S1 short-lived root) with shorter 7-year lifetimes, but will continue to sign them with X1. So a Let's Encrypt chain will look like X1 -> S1-by-X1 -> E10 -> Leaf. Some performance/bandwidth-sensitive operators may choose to use S1-by-self -> E10 -> Leaf
It's not just Chrome here either; Mozilla's already moving in that direction of shorter lived roots.
wonder pqc algorithms will change to calculation, as it make longer chain much more painful
One could issue two certificates with the same Common Name and the same key pair, but with different expiry dates, right? That way at least the same intermediate can be used.
Interesting way to phrase "Google leadership doesn't value Internet security" -- the money is there, it's just not allocated well.
(Not picking on you, just highlighting that Google absolutely has/had money, but the latest layoffs show what they really value. How many higher-paid executives got laid off, by comparison...)
I think there's a second bottleneck: personnel and incident response times. With today's 90-day certs and recommended 60-day renewal period, if catastrophe strikes, a CA would have up to 30 days before the internet starts to fall apart. That's a long time to get people to data centers, generate new key material, even get a cross-sign. With 9-day certs and 6-day renewal periods, they'd have 72 hours to do all that. It's certainly possible, but would require a different scale of disaster preparedness.
Yeah same sentiment, phrased differently. Tech industry is weird at the moment with cash rich companies laying people off everywhere. Nonsensical to me!
Agree and the practicality of operating a CA is something all of these things need to take more into account otherwise we'll need to end up paying for individual certs again.
I've participated in disaster recovery planning and exercises on a (computing) scale equal to or greater than LE and it involves literally the whole team practicing your actual DR at least once a year to make sure all the plans are in working and identify any potential blockers, it also involves getting alternative team members [or random but qualified consultants] to perform the recovery (i.e. not the people who normally do it) to prove the plan is detailed enough and works in an actual disaster.
A proposal exists, and it will likely be entering a CA/B Forum discussion period in the next few weeks.
To clarify - the change is focused on constraining the time period which a self-signed root certificate is distributed by the Chrome Root Store (this promotes increased agility). There is no impact on validity (i.e., the maximum 25 year validity period permitted by the Baseline Requirements is still “fine”).
The discussion period for the proposal Ryan mentioned has begun. It is SC-063, and it makes OCSP optional for all certs and CRLs mandatory for non-short-lived-certs.
https://lists.cabforum.org/pipermail/servercert-wg/2023-April/003685.html
SC-063 v4 now entered voting period, and while there about two day left it looks passing as majorities supporting it
https://lists.cabforum.org/pipermail/servercert-wg/2023-July/
Wow; it's looking like 7-day certificates may actually become a thing that gets offered. Very curious how Certificate Transparency is going to deal with that.
There does look to be at least one No: [Servercert-wg] Discussion Period Begins - Ballot SC-063 V4: "Make OCSP Optional, Require CRLs, and Incentivize Automation" but I presume voting is majority rules.