Revoking certain certificates on March 4

Thanks for the notice. We’re deploying an upgraded version momentarily. Edit: https://checkhost.unboundtest.com/

8 Likes

A post was split to a new topic: You may need to use a different Authenticator Plugin

https://letsencrypt.org/caaproblem now has a link to a more efficient scanner and also points at the beefier https://checkhost.unboundtest.com server.

6 Likes

A post was split to a new topic: Error running certbot renew --force-renewal

I never received notice via email for this problem. I validated we had 45 affected certificates in the dump via account id. We have received prior emails affecting this account in the past, you may have a serious problem on your hands there.

A post was split to a new topic: Error processing CAA for domain

Your script does not retrieve the correct serialnumber when the server uses SNI. You need to indicate the servername:

openssl s_client -connect example.com:443 -showcerts -servername example.com </dev/null 2>/dev/null | openssl x509 -text -noout | grep -A 1 Serial\ Number | tr -d :

3 Likes

Thanks @kjo, that’s a good point.

Good point @kjo. I believe recent versions of openssl actually do use the connect string to set the SNI field, but I agree that older versions do not, and we should have the more robust command.

2 Likes

A post was split to a new topic: Hit rate limit renewing certificates

A post was merged into an existing topic: Hit rate limit renewing certificates

When I first heard about this problem, at about 1600 UTC, I had not received an email. But i thought to check, so I ran a script and discovered one current certificate was affected.

Several hours later, when I was about to double check my script after reading kjo’s comment, I noticed that I had received an email at about 2000 UTC.

I suppose the emails are still being sent out, but there are rumors going around that if you don’t get an email, you aren’t affected. Please someone from Let’s Encrypt staff, if you can edit the original article, make it clear that non-receipt of an email is not equivalent to being unaffected. Even if you’ve verified that Let’s Encrypt has your contact details and the address doesn’t get sent to spam, the email might just still be on its way.

6 Likes

That’s a great point. I’ll get this change worked up now. FWIW, the notification mailer has completed it’s run.

5 Likes

I’m not really sure if https://checkhost.unboundtest.com/ is working correctly, for one of my affected domains it’s showing The certificate currently available on [name] is OK ... but the file downloaded from https://letsencrypt.org/caaproblem/ is still showing missing CAA checking results for [name] at [datetime] and the certificate was last renewed almost a month ago (until now).

Edit: looks like the certificate was actually renewed a day after or so so it’s all good I guess

2 Likes

@vedranl Would you mind letting me know what the domains/certs in question are so I can check?

Looks like I can’t send private messages, any way to send this in private?

Maybe you renewed the certificate, so only the old one, not used anymore, is affected and will be revoked? (for example if you had the wrong configuration for certbot: renew-by-default which renew the certificate even when it’s not yet needed)

@Phil_LE Is there any plan to push back the start time of the revocations since some people (like me) literally just received the email notice in the last 2 hours which is just over 4 hours before midnight UTC?

1 Like

@fleish

I truly understand your sentiment, but I apologize that we must keep a strict deadline here or risk further baseline requirement violations. We definitely have an improvement to make in our notification emailer throughput. Unfortunately that doesn’t help at this very moment.

1 Like

@Phil_LE, the improvement to make is that in future, you should not waste away the five days of notification period on you deciding to make a tool to check which certificates were affected; you should have immediately informed the community and let them renew all of their certificates to resolve the issue.

Is there a place where we can see how your internal governance planned to handle an event such as this one? I think that the community is quite obviously telling you all that your process in this circumstance was wildly misguided and I want to know how you all plan to improve upon this in the future.

To be clear, the fact you had a bug is not the problem; the fact that you completely wasted the notification period is the problem.

5 Likes