We’re encountering an API outage stemming from a database problem and are working through it as we speak per https://letsencrypt.status.io/ . We’ve been through worse and will get through this too. Remember, it’s just another day as an SRE.
Would it be worth changing the appearance of the banner? Black on red isn’t the greatest contrast, so making it larger or changing it to just a red border instead of solid red (or both) might help people see it.
Ya, I cant imagine managing such an infrastructure – thanks for all your service and hard work – you guys are saving lives out here – for real!!!
We’re in the process of restoring API services right now. Before we re-enable certificate issuance we’re going to process the orphaned certificates queue.
I now have “Error creating new order :: too many certificates already issued for exact set of domains”. Like seriously, my backend has been down for hours and all my API services can’t be reached by my clients and I can’t ask for a certificate because I’ve tried to solve the problem.
I have deleted every certbot configuration files because I thought the problem was from my side. How can I retrieve a certificate?
Certificates that were issued during the outage and later incorporated into the database are currently counting against rate limits. We realize that means some people will incorrectly see rate limited status even though they never got certificates during the outage. We’re discussing ways to fix, and will update when we have a conclusion.