Renewal problems -- a mix of old and new!

[erlend]

Hello all!

To get to the point where I can even write these lines, I've gotten a fair few nudges that I might as well include the actual domain name(s) concerned, and maybe that really is true, but I'm fairly certain my issue is generic, and so, for now, I'll resist the full disclosure. I apologise in advance if I'm just being stupid and wasting all of our time in so doing.

Once upon a time, I registered example.com, and then, a bit later, example.org and example.net for good measure. But I wanted to use https, and I wasn't aware of Server Name Indication, so when I first set up Let's Encrypt, in 2018, I only did it for the .com domain.

I then some time later realised that SNI existed and that setting it up wasn't quite beyond an idiot like myself, so I tried that, and, it worked! But ... only somewhat.

My current situation is: I've got a .com domain that autorenews its certificate, and I've been able to pull down certificates for the .net and the .org in the past, but I can't get the autorenewal to work for them. I also notice I've got two different entries under /etc/letsencrypt/accounts:

drwx------ 3 root root 4096 Feb 29 2020 acme-staging-v02.api.letsencrypt.org
drwx------ 3 root root 4096 Dec 3 2018 acme-v02.api.letsencrypt.org

There's a config file under /etc/letsencrypt for my example.com, and empty ones for the .net and the .org one. In trying to handcraft a config file to replace the empty ones, and then running: certbot -q renew, I've gotten various failures; the closest I've gotten to feeling like I was maybe doing something right was this error message: You've asked to renew/replace a seemingly valid certificate with a test certificate (domains: example.org). We will not do that unless you use the --break-my-certs flag!. Skipping.

My version of certbot is 0.40.0.

I am fairly sure I lifted over part of an old (but functional, and still to this day operational) letsencrypt configuration from an earlier installation, because in the (divergent) subdirectories of /etc/letsencrypt/account, I see different account strings, and the same (different numbers) for the /acme/acct/ in the URL listed in the different regr.json files.

If you have read this far, you already have my appreciation. What I'm hoping to do is get the .net and the .org domains to also auto-renew, without ruining anything about my wonderfully auto-renewing .com domain. What would be a good place to start?

Kind regards,
E

[rg305]

@erlend

In trying to handcraft a config file to replace the empty one

^never a good idea.

@erlend

I am fairly sure I lifted over part of an old (but functional, and still to this day operational) letsencrypt configuration from an earlier installation, because in the (divergent) subdirectories of /etc/letsencrypt/account, I see different account strings, and the same (different numbers) for the /acme/acct/ in the URL listed in the different regr.json files

^more bad advice or self-correction attempt.

@erlend

What would be a good place to start?

Let's begin with: what do you have?
What shows:
certbot certificates

[erlend]

Hello, and thanks for responding!

Running that command shows me what I'd expect:

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Renewal configuration file /etc/letsencrypt/renewal/example.net.conf produced an unexpected error: renewal config file {} is missing a required file reference. Skipping.
Renewal configuration file /etc/letsencrypt/renewal/example.org.conf produced an unexpected error: renewal config file {} is missing a required file reference. Skipping.

---

Found the following certs:
Certificate Name: example.com
Domains: example.com
Expiry Date: 2021-06-22 12:50:45+00:00 (VALID: 49 days)
Certificate Path: /etc/letsencrypt/live/example.com/fullchain.pem
Private Key Path: /etc/letsencrypt/live/example.com/privkey.pem

The following renewal configurations were invalid:
/etc/letsencrypt/renewal/example.net.conf
/etc/letsencrypt/renewal/example.org.conf

---

... since, somehow, the conf-files for the non-working domains are 0-byte files. (I didn't put them there, to my recollection, and I don't remember populating the .com one, the one that's actually working, with correct parameters!)

[Rip]

Hi @erlend
You'll have to have at least a bare bones config file for each domain even they all point to the same resource folder.
(@rg305 ~ I'm not sure if you can "Alias" example b and example c in example a config cuz I've never tried it before)...

[rg305]

Try removing the broken configs:
certbot delete --cert-name example.net
certbot delete --cert-name example.org
then retry:
certbot certificates

[erlend]

rootprompt# certbot delete --cert-name example.net
Saving debug log to /var/log/letsencrypt/letsencrypt.log
An unexpected error occurred:
TypeError: remove: path should be string, bytes or os.PathLike, not NoneType
Please see the logfiles in /var/log/letsencrypt for more details.

... but despite that ominous-looking output, that did remove the (invalid, 0-byte) config files!
EDIT: Formatting

[rg305]

@rg305

then retry:
certbot certificates

[erlend]

Only the expected one:

---

Found the following certs:
Certificate Name: example.com
Domains: example.com
Expiry Date: 2021-06-22 12:50:45+00:00 (VALID: 49 days)
Certificate Path: /etc/letsencrypt/live/example.com/fullchain.pem
Private Key Path: /etc/letsencrypt/live/example.com/privkey.pem

---

Is there magic available in some equally elegant command to simply set the faulty ones back up?

[rg305]

@erlend

Is there magic available in some equally elegant command to simply set the faulty ones back up?

Yes, but the magic may require some RTFM time or at least following good advice/guide(s) - LOL
Please don't try to help certbot in any way other than passing it parameters/values in the command line.
[certbot 0.40.0 should be capable of handling all the necessary things behind the scenes for you]

To that end, try repeating your steps in obtaining the dot com domain with the dot net and dot org ones.
If that fails, then please reply with the error message(s) and the full command provided.

[griffin]

Hi

Honestly, we're going to need the output of the following command (sooner if not later):

sudo ls -lRa /etc/letsencrypt

Please put three backticks above and below the output, like this:

```
output
```

[erlend]

I suspect you're asking me to use that, rather than what I've been trying to use blockquote for. I'm vaguely familiar with such syntax from the use of Slack! Testing ...
Some command output here
Argh. Yeah, that would have looked much better. I'll use that from now on!

I've already been nudged in the right direction by @rg305 I believe; it's only now dawning on me that I probably ran certbot as a different user when I renewed my "bonus" domains last. (I was probably trying out this newfangled concept of not doing absolutely everything as the root user )

As irresistible as it is to keep troubleshooting (I've honestly never before used a forum that felt so much like a live chat before!) my bedtime is upon me; I had hoped to launch this message in a bottle and check back in the morning, but then your swift and helpful responses kept me pinned I'll have another crack at it when I get up, but I'm already further along! (searching for fullchain.pem outside the /etc file structure was an eye-opener. You know, 90 days is a perfectly valid interval for a certificate's validity, but when you're as old as I am, that's ample time in which to forget all of your own good intentions!)

[griffin]

The moral of the story here:

Certbot is a complex, particular, fickle, unrelenting, mysterious mistress. She knows what she does and does not like. Attempting to jury rig her file cabinet can result in loss of limb, hair, and sanity.

[erlend]

Well, I fixed it! Thanks for your help!

What got me unstuck was realising I could use certbot with more options than simply "renew" and then watch that fail. And what had me being so cagey about causing too much upheaval was that my most important domain was auto-renewing, and I didn't want to jeopardize that in an attempt to fix the others.

It turned out I'd gotten my most recent certs by running certbot as a different-than-root user and in standalone mode, then (presumably) copied those into the /etc/letsencrypt file structure where I probably believed they needed to be. And my manual tinkering was aiming at getting those auto-renewals in line with what I observed to be working for my "main" (.com) domain, when I didn't realise that the framework for it just wasn't there.

The manner in which I've fixed it would probably horrify you (there were a fair few for-loops to properly symlink things in /etc/letsencrypt/live to things in /etc/letsencrypt/archive) but I guess all I have is a hammer and, with your help, I was able to see the "right" nails in my mess

Thanks for the assistance! And, lesson learned ... if I ever (God forbid!) set up any new domain certificates using certbot, I'll be sure not to go off the beaten path and leave a loaded gun pointing at my feet! All that's required is for me to remember all of what just happened, and I can do that. I think. Probably.