There's a lot going on in this thread, but the root cause that started problems here is I think /etc/letsencrypt was manually modified to contain multiple accounts for the same server. While Certbot does contain code to try and help you if you find yourself in this situation, I don't think Certbot has ever been willing to create accounts like this for you (with a couple exceptions that do not apply here). In general, Certbot is quite sensitive to unexpected changes to /etc/letsencrypt and I would advise people avoid manually changing things here when they can.
To respond to the specific thing you flagged in your post though:
I'm assuming you're talking about
The problem there is cd241a1b5a9a82e9b3e9b97a841440e3 is a staging account according to:
certbot unregister is operating on the default ACMEv2 production server and cannot find the account there. /root/certbot/certbot-auto unregister --staging --account cd241a1b5a9a82e9b3e9b97a841440e3 should work.
I think the reason the error message says ACMEv1 is the ACMEv2 directory is/was a symlink to the ACMEv1 directory on this system so that's the real path being checked on disk.
Thanks for the response Brad. You're certainly on target about the multiple accounts. @donottaptheglass ported the directory structure from a different machine, but the v1 directories didn't make it over.
It makes sense that v2 was symlinked to v1 based on that output from unregister. I'm assuming that the staging account path with no version is some extreme legacy based on @_az's initial comments.
Funny you should mention that... that type of crossover seems to have been a trend. I will definitely be advising people not to use .cli files with account information...
To close, is there any kind of "repair my legacy disaster" logic built into certbot that can be invoked? I didn't see any symlinks based on the ls -la outputs throughout this topic, which is a bit confusing given the outputs I was seeing from certbot.
The original/ACMEv1 staging server had no version number in the URL.
I personally haven't seen this problem (m)any other times, but that could make sense. In general, if the user is ever manually setting the --account flag or config option, something strange is probably going on in my opinion since Certbot doesn't let you set up multiple accounts so the account value should be irrelevant.
Just to make sure we're all on the same page, I don't think this is just a legacy disaster, but a /etc/letsencrypt config that Certbot never would have created on its own.
There is no built in way to recover from configurations like this unfortunately. The easiest option is to blow away /etc/letsencrypt and start over.
If that's not feasible, I would do the following, but it will force renewal of all your certificates which may not be an option if you would hit rate limits:
Backup /etc/letsencrypt with a command like sudo cp -Ppr /etc/letsencrypt/ /etc/letsencrypt.bak.
Delete all of your accounts with a command like sudo rm -rf /etc/letsencrypt/accounts/.
Run sudo certbot register to create a new account.
Remove all account information that Certbot stores in your configuration files with a command like sudo sed -i '/^account =/d' /etc/letsencrypt/renewal/*. You should delete account values from any of your own files/scripts here as well.
Run sudo certbot renew --force-renewal.
After that, you should be good to go!
I think the ls commands were always run at the wrong level to see the symlink. The symlink that is created is from /etc/letsencrypt/accounts/acme-v02.api.letsencrypt.org/directory to /etc/letsencrypt/accounts/acme-v01.api.letsencrypt.org/directory.
It seems like .cli files should just NOT use any account information.
Especially when intended to be used by production AND staging environments.
[there is no single account value that can be used by both systems]
Agreed. Honestly I'm not sure what value the .cli provides at all under the current architecture, if any. I don't know how it overrides/meshes with the individual configuration files. It could serve as a default fallback for certain unspecified settings in the individual configuration files, but that seems dubious.
After deleting all accounts, registering a new account, and removing the account line from all of the ini files, I am able to dry run and issue certs normally.
I meant to write this sooner, but we have been so busy buttoning up small things here and there - server migrations are no joke!
I have learned a lot, thank you so much! Big thanks to you as well @griffin!
