Questions about Renewing before TLS-ALPN-01 Revocations

[Amazon Lightsail on Bitnami/Apache - Auto-Renew Script + Cronjob]
Ok, I remembered creating a script to renew my certificates (bitnami/apache server). I created a script here so edited it and changed this script (which is run daily by a cron job to check if the certs need renewing yet):

sudo nano /opt/bitnami/letsencrypt/scripts/renew-certificate.sh

I changed the number of days to renew in this script to 1. [Edit: 61 should do it... longer than 60 (previous expiry days) was needed]

I'm 50% confident that my certificates should now auto-renew before 28th Jan. Not sure if there's an even quicker way to force auto-renew.

This is what the line of code looked like for my auto-renew if you have similar:

sudo /opt/bitnami/letsencrypt/lego --tls --email="My Email is Here.com" --domains="firstDomainName.com" --domains="secondDomainName" --path="/opt/bitnami/letsencrypt" renew --days 61

I also changed my cronjob that runs at a certain day and time...[Edit: Not needed! Just manually run the script above lol! I'm so rubbish at this...]

I'm also struggling to reply to the right people/posts - keeps replying to Jillian regardless, what am I doing? :crazy_face:

4 Likes

A post was merged into an existing topic: Early renewal for bncert (bitnami)

I'm pretty new to all this, but I'm guessing when they revoke the cert, you won't have a cert until it's renewed again.

I'm using Lightsail and Bitnami on Apache too... do you have an autorenew script?

When the certificate is revoked, your server will continue to serve the certificate it has but it won’t be trusted by all clients. Not all clients and browsers check OCSP (whether your site has a valid or revoked certificate). This will result in errors or warnings for some visitors to your site until you renew and replace the revoked certificate.

4 Likes

If you don't know with which challenge type your certificate has been issued, most likely your hosting provider has done this for you. If that is the case, then it's also your hosting provider which will have to renew your certificate in time before the revocation of the old certificate takes place.

You cannot see which challenge was used to issue a certificate from looking at the actual website I'm afraid.

3 Likes

I'm afraid the "5 days rule" that's mandated by the CA/Browser Forum Baseline Requirements. So with issues like this, it will always be on a short notice.

Don't forget to change it back once the certs have been succesfully renewed :wink:

4 Likes

We manage hundreds of domains across SaaS solutions and standard websites. SSL certificates are one point in a long chain of infrastructure which we have found automated solutions such as Caddy, or use various other providers to make things work.

For most of us, SSL generation, authentication and application are a black box of "magic".

The complete lack of information on how to identify which domains are affected (in bulk) and how to proceed to resolve the issue for the most common scenarios, is quite frankly, terrible.

Replies saying "it's easy" completely miss the point - landing a plane is easy if you've been trained in that skill. If not, it's going to be like the ending of Die Hard.

Please provide more information to help the potentially millions of domains this could be affecting.

4 Likes

What are the steps for users with an older version of Caddy, such as 0.11.5?

:grimacing: That's many years old. Definitely upgrade to Caddy 2:

2 Likes

Hey Scott, you mention Caddy -- are you frustrated about how Caddy "just works" in this case or are you just using that as an example, but you don't actually use it? How can I help?

AFAIK Let's Encrypt sent emails about all affected domains to all clients who specified an email address (which is always recommended), and clearly defined which certificates are affected as well.

2 Likes

Unfortunately, we cannot provide that information in the e-mail because some accounts have too many domains or serials to list in an e-mail and there are too many ACME clients in use with different ways of configuring and managing this information.

If you only use or primarily use the TLS-ALPN-01 challenge you should renew all of your certificates for the account id provided in the e-mail. If there are a few domains using a different challenge or you can’t figure out which accounts manage which domains, it won’t be problematic to renew them early too.

3 Likes

Thanks, that's helpful. :blush:

I could upgrade, but not in time. According to this notification, this needs to happen in the next 2 days. Does this mean it's game over for me? Or is there any way to comply, and then upgrade at a later date?

My point is that I run a web agency, and have a few SaaS tools. I got an e-mail telling me about this problem, with no more information other than an ID (an ID I've never seen before).

I can check the Caddy version and I'm sure that part will be fine.

However, on the web agency side we manage a mixture of client sites - some on their own hosting (at a wide range of providers) and some on ours.

I have no idea which domains are affected - as I said, we have hundreds on our books.

Delete the certificate from Caddy's storage (disk, I don't even remember where v1 stores its certs anymore, ha) and reload it. But then upgrade :wink:

4 Likes

This is my point - it's not a problem for people who are living and breathing SSL certificates, but for people who don't this is a complete mystery as to what the next steps are.

Please think about who uses Let's Encrypt - it's everybody from seasoned experts through to novices. You need to cater for all levels.

I appreciate some clients will have too many domains to list, but why not provide a way for us to at least see a list somewhere, or truncate the list in the e-mail so it gives us a little bit of a head start.

2 Likes

I understand you are stressed. I think you can rest easy as long as you deploy your clients' sites with an ACME client configured to use an email address you monitor.

To be fair, I think Let's Encrypt does quite a good job for a lean non-profit organization, and frankly, they don't have to cater to all levels (though of course, they do try, and they do a good job). If you do have these kinds of requirements, there are commercial CAs who are happy and more able to cater to you.

(As an aside, if you want the Caddy project to better support your business, please consider sponsoring! Different tiers offer different modes and urgency of private support / help for your company.)

PS. This kind of thing has only happened a couple of times... ever, in all of history... at this scale. (And actually this event is pretty small compared to the last mass revocation.) We're all just doing our best. So anyway, you can understand too that CAs are required to remediate within 5 days or fail compliance audits. They want to help you better, but are resource constrained.

8 Likes

Hm, I'm unsure I did it right.

mv ~/.caddy/acme ~/.caddy/_acme  # Keeping a backup just in case
systemctl restart caddy

The journalctl log then says.

Jan 26 08:49:09 ragdoll systemd[1]: Stopping Caddy HTTP/2 web server...
Jan 26 08:49:10 ragdoll systemd[1]: Stopped Caddy HTTP/2 web server.
Jan 26 08:49:10 ragdoll systemd[1]: Started Caddy HTTP/2 web server.
Jan 26 08:49:10 ragdoll caddy[27195]: Activating privacy features... 2022/01/26 08:49:10 [INFO][FileStorage:/etc/ssl/caddy] Started certificate maintenance routine
Jan 26 08:49:11 ragdoll caddy[27195]: done.
Jan 26 08:49:11 ragdoll caddy[27195]: Serving HTTPS on port 443

But (1) the acme folder is not recreated and (2) the SSL checker still says my certificate is from December 2021. Is there another mechanism with which to reload the certificate or should I expect it to recreate this folder once the certificate is renewed?

Don't get me wrong, I appreciate Let's Encrypt a lot - it's been a fantastic resource for millions (if not billions) of people.

However, not being able to find a list of domains affected is a huge issue. It's a simple ask to stop me from mobilising my entire team today to go through and manually check every single one.

2 Likes

When I say “problematic” I am primarily speaking to the utilization of Let’s Encrypt resources and possible rate limiting. We generally recommend renewal attempts begin 30 days until expiration and rarely recommend a force renewal to solve a problem. In this case, we want subscribers to focus on renewing their certificates with less concern about “wasting” resources. But please be mindful of sending requests too quickly and try to spread the renewals over a reasonable timeframe.

I’m sorry we were not able to provide a better head start regarding the specifics of affected domains. It is a balance between timelines of notification and implementation of how to supply more information. According to the Baseline Requirements, the clock for when we MUST revoke the certificates by begins when we receive notice of mis-issuance according. Before the clock expires, we have to confirm the report, fix the bug, deploy the fix, notify subscribers, and revoke certificates. Sometimes, that first part can take a while and in order to provide maximal time for subscribers to replace their certificates we choose to provide the information we have readily available. Our incident report will be posted later this week and contain more information about the timeline leading up to the e-mail you received.

As noted above, you likely fit the case to simply renew all certificates that you have. Once that is complete and depending on your client and current renewal configurations, you should make sure the renewal after this “force renewal” has some randomization to it and all the certs are not renewed at the same time in exactly 60 days.

7 Likes

Forgive my ignorance but 2 questions if you're able to help;

  1. How can I tell if this affects my particular certificate? I only have 1 to worry about.

and

  1. How do I go about remediating the issue? I am running Apache2 on Ubuntu 20.04.

Thanks.

2 Likes