2018.07.30 Domain Resolution Interruption

On July 30, 2018, our systems administration team attempted to set “registry lock” (server{Update,Delete,Transfer}Prohibited) statutes on our primary domain, “letsencrypt.org”. This erroneously resulted in the domain being set to “clientHold” status, preventing DNS resolution and disrupting service for our subscribers and relying parties for a short period of time. We believe this was the result of administrative error or miscommunication at, or between, our reseller, Namecheap, and our registrar, Enom.

Our team is reviewing options for preventing this from happening again, as well as options for resolving issues like this faster should they occur again. We are considering the following actions:

  • Improve plan for escalating issues with Namecheap and Enom to improve resolution time
  • Simplify our domain administration situation by moving to direct registration with Namecheap, which should reduce the likelihood of administrative error
  • Possibly move to a different registrar with more specialized support for protecting high value domains
  • Improve backup domain options for incidents that do not resolve quickly enough
  • Possibly move our OCSP service to a separate domain name, to reduce the frequency and scope of changes (and their potential disruption)