Migration error, losing data

Help
1

My domain is: multiple and not necessary relevant (on test server).

I ran this command:
apt-get purge certbot

It produced this output:
Full /etc/letsencrypt directory deletion

My web server is (include version): Apache/2.4.38 (Debian)

The operating system my web server runs on is (include version): Debian GNU/Linux 10 (buster)

My hosting provider, if applicable, is: Test Linux box at home.

I can login to a root shell on my machine (yes or no, or I don't know): yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): 0.31.0

Hello,

While trying to migrate from Debian apt-get certbot to snap certbot, and willing to better clean packages, I ran "apt-get purge certbot" after "apt-get remove certbot" in order to better clean the package, but it appears this was a bad idea : this command deleted all the /etc/letsencrypt directory, then I have lost all my configurations.

Fortunately it was on a test server, but nevertheless, it was very hard to recover : this broke the Apache server making impossible to do a clean reinstall, and I had to deal with an approximative backup in order to be able to solve this while still having lost my original certificates, which I need despite it is a test server.

Now it seems to work, but my previous certificates are probably still out there in the cyberspace, and likely impossible to revoke.

How can I deal with this situation ?

1 Like
2

Hi @Gingko

please explain: Did you publish the private key?

If yes, that's wrong.

If no, the private key doesn't exist (because you have deleted it), so a certificate revocation isn't required. Without the private key nobody can use the certificate.

So your problem doesn't exist.

2 Likes
3

Certificates: yes, they are public.
Private keys? I hope not!

2 Likes
4

Thank you for you answers.

I've never published anything, neither private nor public.
I was just worried about orphan data related to these certificates staying outside.

1 Like
5

… and by the way, is it normal that "apt-get purge certbot" deletes the whole /etc/letsencrypt/ directory without any confirmation nor any warning ?

1 Like
6

The thing about the confirmation or warning you should ask the developers of apt, but yes, the apt-get purge behaviour is to remove any residual files too:

purge

purge is identical to remove except that packages are removed and purged (any configuration files are deleted too).

(From apt-get(8) — apt — Debian stretch — Debian Manpages)

2 Likes
7

Here's a fun tutorial I created:

1 Like
8

“configuration files” does not necessary means "installed certificates", this is very unclear (and if yes, it would be stated in large, bold, red and flashy characters in all related documentation and help files).

It is somewhat like if "apt-get purge apache2" was deleting all web sites data and configuration files added on the corresponding server. (I just tried this, it deletes its own added configuration files but not virtual server files added by user).

9

In this case configuration files are also under /etc/letsencrypt/ so... next time if you are unsure what the purge will do you can check postrm and prerm scripts.

/var/lib/dpkg/info/package_name.postrm
/var/lib/dpkg/info/package_name.prerm

In certbot case you could see this in postrm script /var/lib/dpkg/info/certbot.postrm

[...]
case "$1" in
    purge)
        rm -rf /etc/letsencrypt /var/log/letsencrypt
    ;;
[...]

I suppose this kind of things happen to say us that we should backup, backup and backup :stuck_out_tongue:

3 Likes
10

Other man-pages say things like:

Removing a package removes all packaged data, but leaves usually small (modified) user configuration files behind, in case the remove was an accident. (…) On the other hand you can get rid of these leftovers by calling purge even on already removed packages. Note that this does not affect any data or configuration stored in your home directory.

(From Ubuntu Manpage: apt - command-line interface)

Perhaps the package can "protect" certain things from purging, I don't know.

In any case, apt-get purge behaviour is not something the certbot developers can do anything about and any complaint about that should be taken up with the apt developers (about the warning) or the certbot packages from Debian, if a package can influence the purge behaviour (about protecting certain paths from purging, if that's even possible).

Are those scripts automatically generated by apt or dpkg? Because I can't find it in the list of files:

https://packages.debian.org/buster/all/certbot/filelist

https://packages.debian.org/buster/all/python3-certbot/filelist

Ah, I found it.. The postrm script is added by the Debian maintainers to the package. It's not in the "data" section of the package, but in the "control" section of the .deb file. That part isn't maintained by the certbot team, but by the Debian package maintainers.

I'd advise @Gingko to use the --simulate option of apt before running possible damaging things like apt-get purge.

3 Likes
11

If I call apt-get purge apache2 repeatedly, even 10 times, it does not remove added configuration files.
I just get a message saying that the apache2 package is not installed, thus it cannot be deleted.

Debian packages are handled by scripts created by developers using dpkg instructions, it is not up to dpkg developers to avoid deleting not empty directories containing files not directly installed by the package.

12

Let's see if one of the maintainers of the certbot package in Debian can take a look at it.

@hlieberman

Resume:
apt-get purge certbot command removes the entire /etc/letsencrypt/ dir, maybe it should remove all but /etc/letsencrypt/{archive,live}/ dirs to avoid the problem suffered by @Gingko in this post.

Appreciate your comments.

Cheers,
sahsanu

3 Likes
13

I'm pretty sure those scripts are created by the Debian package maintainers? The certbot repository does not have a filename "postrm" nor does any file contain the text "postrm" or "purge".

Also, if I understand the purpose of the purge command correctly, the removal of /etc/letsencrypt is probably very much intented. Why would it be added to the postrm script otherwise?

3 Likes
14

Yes, you are correct, those control files (postrm, prerm, etc.) are created by package maintainers not certbot developers.

3 Likes
15

In that case (still doing comparison with the apache2 package), why the "apt-get purge apache2" command does not delete the /etc/apache2 directory (at least if it contains user added files) while this directory has been very created during the package installation?
This is handled by the very same scripts created by the Debian package maintainers, AFAIK.

16

Different maintainers, very different script.

2 Likes
17

Hello!

Thanks for tagging me into the thread. The purge behavior is something that has been discussed a couple of times before, with people weighing in on both sides to argue for (and against) keeping live certificates on purge.

The compromise I came up with is that newer versions of the certbot package will warn the user on purge if there are unexpired certificates still in /etc/letsencrypt/live/*. It will then prompt the user to make a decision.

Unfortunately, it seems that you're on an older version of the certbot package that hadn't had the update to the purge script yet.

In general, running apt purge is something you should only do when you want to remove all traces of a package on the system; apt remove will safely leave configuration files on the system. This is mentioned in the manpages for apt as discussed previously. I'm sorry that you got bit by this corner case; it's something we want to avoid, certainly.

That being said, this shouldn't have broken apache in any significant way once you deleted the certbot created configuration file. If apache was disrupted to a larger extent, please let me know -- that would be a serious bug.

3 Likes
18

@hlieberman If Apache is configured to use a file from /etc/letsencrypt and that directory is removed, then Apache will be broken afterwards. That's a common issue, also when users are using certbot delete and would certainly be the case when running apt-get purge as both commands won't automatically remove any reference to the deleted files from the Apache configuration file(s).

2 Likes
19

Using SSL using LetsEncrypt implies using configuration items like these ones in configuration files :

SSLCertificateFile /etc/letsencrypt/live/[config]/cert.pem
SSLCertificateKeyFile /etc/letsencrypt/live/[config]/privkey.pem
SSLCertificateChainFile /etc/letsencrypt/live/[config]/chain.pem

If the /etc/letsencrypt directory is deleted, these file are no longer found and Apache cannot start.

If you have dozens of virtual files on your web server, you may have that config found as many times as there are virtual files, it may be very cumbersome to repair.

20

That's expected; if you're deleting certbot and its keys, I'd also expect you to be removing the configuration files that certbot created.

That being said, if you're saying the issue occurs even when running certbot delete and the apache installer-plugin, that sounds like a bug to me.

1 Like