Ocsp server responses for new certificates

Issuance Tech
2

This is interesting - thanks for reporting it!

  1. No, we've had no known problems with OCSP distribution recently.

  2. Under all normal conditions, there will be no delay at all. A conservative wait time would be two seconds. But the ideal ACME client / OCSP stapling stack should reach out and double-check that a newly obtained certificate has a valid OCSP response available before actively using it. We try to guarantee this from our side, but this approach would be most fault-tolerant.

  3. I agree; we already try hard to ensure immediate availability, and failures are rare. Here's what our current architecture looks like:

  • Our front-end CDN caches our origin's responses for 12 hours. It doesn't have its own database: all uncached requests are forwarded to our origin.
  • When we revoke a certificate, we reach out and immediately purge its old (valid) OCSP response from the CDN's cache.
  • Our origin is load balanced (usually equally) across multiple data centers.
  • Our origin OCSP service reads directly from the same (ACID compliant, MySQL flavor) database that the rest of our CA stack uses.
  • An OCSP response is generated and stored in the database before an issued certificate is returned to the client.
  • Our database uses a simple primary (read/write) / replica (read-only) model.

If we have a long delay replicating from our primary DB to one of our replicas (and one of our CDN's POPs happens to hit the affected replica), that's when a client could have received a certificate without a valid OCSP response being available yet.

This is pretty rare: our replication lag is usually far lower (a few milliseconds) than the time it takes the client to retrieve the certificate and get its first OCSP request back through to one of our database servers. We monitor this metric closely and alert our SRE team when it's elevated. I just checked this metric at your issuance time, and it appears to have been too low for a failure to happen this way. So, I'm stumped.

If you observe this again, it would be really useful to know the timestamp again; the certificate's serial number; and which source and destination IP addresses you used for the OCSP request. We keep minimal OCSP logs for privacy reasons, but this will give us the best chance of tracking down a little more information about how your request was served.

4 Likes