And it appeared that the OCSP was updated at 13:00 UTC, but then there was an update at 19:00 UTC. I was expecting it update today (8/1) at 13:00 UTC(09:00 EDT), but has not happened.
Further, when I check one (good) cert, I get:
This Update: Jul 31 13:00:00 2018 GMT
But on another (revoked) cert I get:
This Update: Jul 31 19:00:00 2018 GMT
Revocation Time: Jul 31 18:59:56 2018 GMT
Is there a fixed schedule for updates to the OCSP server? If it is not fixed, is there particular criteria that is used?
Is there a document / FAQ / weblog post / etc which describes OCSP updates?
I believe the OCSP server updates almost immediately when a certificate is revoked. New Good repsonses are generated approximately daily, but the nextUpdate for them is always 7 days away, so these responses can still be used for purposes like OCSP Stapling for that 7 days even if the server generates a new response in the meantime.
You probably saw an update happen earlier than you expected because there was a massive outage the night before last. The operations team may have forced the regeneration of all OCSP responses as part of getting things working, or they may have shifted traffic to a different datacenter that produces different responses.
The reason there is not a fixed schedule is precisely because emergencies like these would render it invalid, and because a revocation causes an immediate update anyway. I'm not really sure how a fixed OCSP update schedule would be useful, anyway. Care to explain?
I was thinking that a bunch of revocations are pooled up, and then the front-end servers would be updated as a batch job every "x" hours or every "y" updates.
This way you can have a known-good database that answers queries from the public, and the data workflow would verify changes so that the front-end would get a snapshot of revocations.
This would allow a distributed pool of front-ends to be in sync, otherwise you could get inconsistent responses from different servers due to (async) replication delays.
If an emergency occurs, the same process can be run manually instead of being kicked off by the (proverbial) cron job.
For what it’s worth, the OCSP system uses a CDN. I have no idea how it’s configured, but it’s possible cached responses could present a misleading picture of how the schedule works.
It does exactly this, but in the revocation case they run that batch job every second so it feels instantaneous.
These kinds of details are actually documented in code. The server software that powers Let's Encrypt is open source and you can even run your own CA with it: