Certbot's Cert Revocation needs an update

[stwm]

Here's what happened to me (more information here).

I have some 50+ domains, all with certs that have sub-domains, and I made a mistake when doing off-site backups and sent some certs, including the private keys, in cleartext - oops!

As the first time I had to revoke certificates, which I understand we are obligated to do under the terms of service for receiving the certs in the first place, I followed the revoke procedure as directed here.

However, when I went to issue new certificates, I've created a catch 22 problem which appears to be caused by the fact that I listened to the advice certbot gave and told it to delete all the existing cert files. -ugh!-

This isn't good advice as deleting the certs prohibits Apache from running (starting) and therefore certbot can't / won't authenticate, and the old files are now gone so you can't authenticate from the old private keys. Further, it's not fun because creating all new certificates is a huge headache because certbot always trashes the apache configuration file. SURE, it works OK for the most basic configuration but do something out of the ordinary and it breaks and has to be fixed by hand.

To avoid having the web sites all down for a long time, one is forced to RESTORE THE BACKUP, thereby using the revoked keys. Then, the user has to reconstitute all the various certs, which ALSO may be non-trivial when there are lots of domains, lots of subdomains which aren't common in any way with other domains, and when there may not be easy records to go by to "do it by hand" using certbot certonly -d ,_{. [...]}

SURELY it would be better if, when asked to do a revocation, certbot could ask if the environment is now secure and if it should just create a new certs, just like the old ones, instead of, or in addition to, asking about deleting now revoked (and other) certs.

As a programmer myself, this seems like a modest request, but I really don't know if it is or not. SURELY it would help people who are already under the heavy stress of having compromised security not make additional mistakes like I did.

[Phil]

@bmw Tagging you in on this. Maybe this should just be a github request?

[griffin]

@stwm

However, when I went to issue new certificates, I've created a catch 22 problem which appears to be caused by the fact that I listened to the advice certbot gave and told it to delete all the existing cert files. -ugh!-

This isn't good advice as deleting the certs prohibits Apache from running (starting) and therefore certbot can't / won't authenticate

There's actually nothing wrong with deleting the revoked certificates and their private keys from permanent storage. As long as you don't restart Apache afterwards, the deleted certificates and their private keys will remain in RAM. Since the Let's Encrypt authentication servers ignore any invalid (expired and/or revoked) certificates during the authentication process, there should be no trouble immediately acquiring new certificates to replace the deleted, revoked certificates. Usually, certbot would restart the webserver automatically after acquiring the new certificates, making the entire process fairly seamless.

One additional thought:

I'm not sure if deleting the revoked certificates is actually necessary to replace them. I would hope that certbot would treat revoked certificates as "expired' for purposes of determining if they need to be renewed.

[griffin]

@stwm

certbot always trashes the apache configuration file. SURE, it works OK for the most basic configuration but do something out of the ordinary and it breaks and has to be fixed by hand.

I hear you here. I often recommend using certonly --apache or certonly --webroot -w path combined with --deploy-hook "apachectl -k graceful" to avoid modification of the apache configuration files when complications may arise. I've suggested splitting the acquisition and installation processes to @bmw on more than a few occasions.

[jvanasco]

I can see how this is frustrating. Consider if the instructions said something else though... imagine what would happen if someone procured new Certificates and then went into the LetsEncrypt archive to delete the old Certificates -- they would likely quickly get confused trying to figure out which are good and which are bad. There is then a decent risk the user will delete the new good certificates, and deploy with the revoked ones.

I don't think there is going to be a good/happy solution to situations like this. Despite this user's personal experience, I do think the current approach is one of the most useful at minimizing additional damage for the widest number of users and scenarios.

That being said, I do think it would make more sense if it certbot moved revoked certificates/keys out of /archive and placed them in a /revoked namespace. The symlinks in /live could be updated to reflect this; maybe place symlinks in /archive too, as linking to a /revoked area would give insight about the file status even if the target is removed.

[bmw]

I'm sorry for the trouble and frustration @stwm. I agree that what you describe is a problem, we should modify Certbot to help avoid it, and I appreciate you and others here taking the time to write up your thoughts and suggestions.

If you're interested, we'll track making a change on our end to help with this at Warn user for broken webserver configurations when deleting a certificate (after revoking) · Issue #8755 · certbot/certbot · GitHub.