The following renewal configurations were invalid

BPL · June 6, 2020, 4:35pm

Ok, no worries… I’m quite familiar with python so it won’t be very difficult to understand what’s certbot-auto script does… although bash scripting isn’t my forte though hehe. In any case, I’ll try to fully understand how certbot-auto works first so I won’t mess up when trying to uninstall it… In any case, I hope the official package will work ok with all the files generated by certbot-auto… it should, right?

Btw, I’ve noticed this last time i’ve run certbot-auto certificates it autoupgraded automatically, ie:

[root@remote1 bpl.expert]# certbot-auto certificates
Upgrading certbot-auto 1.4.0 to 1.5.0...
Replacing certbot-auto...
Creating virtual environment...
Installing Python packages...
Installation succeeded.
Saving debug log to /var/log/letsencrypt/letsencrypt.log

I mean, probably this certbot-auto is more than good enough… but i still feel more comfortable installing from packages… so my provisioning scripts are more “robust” in case to add/deploy to new servers.

Osiris · June 6, 2020, 4:42pm

The EPEL repo EL-8 has version 1.4.0 of certbot. So if you do switch to the EPEL version of certbot in stead of certbot-auto, you could get a warning message saying the certificate renewal file was generated with a newer version of certbot. I believe however this only matters if you’d update a certificate or would get a new certificate with this new version 1.5.0. Also, I’m not expecting any breaking changes in the renewal configuration files between 1.4.0 and 1.5.0 so the warning can be ignored.

BPL · June 6, 2020, 4:51pm

Oh, ok, that’s really relevant info definitely… btw, from what i’ve been able to decypher by reading certbot-auto is when you run it, it tries to autoupgrade automatically (unless specified otherwise) and a python virtualenv will be created at /opt/eff.org by default. Such virtualenv will contain all certbot python scripts as well as its python dependencies. Now I’m just trying to confirm whether other data files have been installed. You see? This is why also installing from packages is really handy… uninstalling, querying package files would be trivial that way… , totally my bad… i screwed up here by trusting random unofficial tutorials from the internet

Ok, in any case, if i wanted to invalidate all the existing certificates before uninstalling certbot-auto and installing from packages so i wouldn’t get any warning (even if that doesn’t matter)… would there be any easy way to do it?

BPL · June 6, 2020, 5:07pm

So far i’ve found these directories:

/etc/letsencrypt
/var/lib/letsencrypt
/var/log/letsencrypt
/opt/eff.org

Not sure if I’m still missing anything…

Osiris · June 6, 2020, 5:10pm

There’s no reason to do that.

Only removing the virtualenv is enough. Removing/invalidating certificates could lead to running into rate limits.

BPL · June 6, 2020, 5:16pm

Mmm, why do you think i shouldn’t uninstall /var/lib/letsencrypt and /var/log/letsencryp as well?

Btw… I’ve also discovered that by activating the virtualenv certbot binary will be available in PATH…

Ok, I won’t consider removing the installed certificates then.

Osiris · June 6, 2020, 5:19pm

Because it contains contents generated by the certbot python program, not the certbot-auto wrapper script.

Nothing strange there. However, it would be the certbot Python script installed by the certbot-auto wrapper script, not the same certbot Python script installed by your CentOS package manager. Having both installed could and/or would lead to strange behaviour.

BPL · June 6, 2020, 5:26pm

Ok… btw there is also this official reference at https://certbot.eff.org/docs/install.html#certbot-auto which says:

To remove certbot-auto, just delete it and the files it places under /opt/eff.org, along with any cronjob or systemd timer you may have created.

Summing up, it seems that removing the certbot-auto as well as the virtualenv and then installing the centos package should be good enough, right? In any case, I’ll be trying first on the staging box to be 100% sure i won’t screw up the production server

Osiris · June 6, 2020, 5:30pm

Ah, good to know, thanks!

Yes, I agree.

BPL · June 6, 2020, 5:59pm

Ok, here’s the new status report

I’ve removed certbot-auto + /opt/eff.org virtualenv and installed certbot from the official centos package and now I’ve got certbot 1.4 up and running and it’s displaying correctly the info from the installed certificates in both the production/staging servers.

So, given the current situation let’s come back to the initial question… considering certbot certificates output:

[root@remote1 ~]# certbot certificates
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Found the following certs:
  Certificate Name: bpl.expert
    Serial Number: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx                   
    Domains: domain1 domain2 domain3 domain4 domain5 domain6 domain7
    Expiry Date: 2020-08-07 16:59:58+00:00 (VALID: 61 days)
    Certificate Path: /etc/letsencrypt/live/bpl.expert/fullchain.pem
    Private Key Path: /etc/letsencrypt/live/bpl.expert/privkey.pem
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

How would i add new certificates not only for the existing production domain names domain1 domain2 domain3 domain4 domain5 domain6 domain7 set but also for the staging domain names local1.domain1 local1.domain2 local1.domain3 local1.domain4 local1.domain5 local1.domain6 local1.domain7.

I mean, the whole idea is being able to use ssl while testing my webapps in the staging box… and right now I’ve got local1.domain1 ... local1.domain7 in /etc/hosts pointing out to the staging server ip

Does it make sense?

Osiris · June 6, 2020, 6:02pm

Is the staging servers IP a public IP address? Or private? If private, are those staging domain hostnames resolvable from the “outside” web?

BPL · June 6, 2020, 6:17pm

The staging servers are centos vagrant boxes spinned out from my windows development machine, they’re named local1, local2, local3, ... so on my development box my hosts file would look like:

# My websites (staging1)
192.168.0.101 local1.domain1
192.168.0.101 local1.domain2
192.168.0.101 local1.domain3
192.168.0.101 local1.domain4
192.168.0.101 local1.domain5
192.168.0.101 local1.domain6
192.168.0.101 local1.domain7

# My websites (staging2)
192.168.0.102 local2.domain1
192.168.0.102 local2.domain2
192.168.0.102 local2.domain3
192.168.0.102 local2.domain4
192.168.0.102 local2.domain5
192.168.0.102 local2.domain6
192.168.0.102 local2.domain7

# My websites (staging3)
192.168.0.103 local3.domain1
192.168.0.103 local3.domain2
192.168.0.103 local3.domain3
192.168.0.103 local3.domain4
192.168.0.103 local3.domain5
192.168.0.103 local3.domain6
192.168.0.103 local3.domain7

# My websites (staging4)
192.168.0.104 local4.domain1
192.168.0.104 local4.domain2
192.168.0.104 local4.domain3
192.168.0.104 local4.domain4
192.168.0.104 local4.domain5
192.168.0.104 local4.domain6
192.168.0.104 local4.domain7

So answering your question, yeah, the ips would be private ones and they couldn’t be reached from the outside. I’m giving you all this info so you’ll be able to advice properly how to proceed, I mean, right now my nginx files look something like:

server {
    server_name domain1 local1.domain1;
    root /var/www/domain1;

    # common
    include common/security.conf;
    include common/general.conf;

    # flask app
    location / {
        proxy_pass http://unix:/run/gunicorn/domain1.socket;
    }
}

and I’d have one of these nginx files for each one of my domains (domain1, … , domain7)

Osiris · June 6, 2020, 6:27pm

And the same goes for the hostnames, as they only exist in your hosts file, right?

In that case you can’t use a Let’s Encrypt certificate for those specific hostnames, as the Let’s Encrypt validation server needs to be able to validate the hostname, either by connecting to its IP address or by resolving a special TXT resource records on the DNS zone.

You could however use wild card certificates. I.e., get a certificate for *.domain1, *.domain2 et cetera. This would however require the use of the dns-01 challenge, which could be harder to set up compared to the http-01 challenge. Another reason to switch to the CentOS package manager to install certbot: it’s relatively hard to install one of the DNS plugins while using certbot-auto.

BPL · June 6, 2020, 6:33pm

Yeah, exactly, both vagrant boxes ips and hostnames would be private to my development box

Not sure I haven’t understood much out this explanation though but let me see whether I’ve understood a bit though… are you suggesting i should delete somehow the existing approved certificates for domain1, ..., domain7 and somehow create new certificates for *.domain1, ..., *.domain7? I didn’t know it was possible to get certificates for wildcarded domains?

Another option (just random thought without any theorical basis) maybe… would it be possible somehow to use the approved certificates for the official domains in the staging boxes?

Anyway, just to let you know at this point I’m a bit confused so if you could please suggest me what’d be the easiest way to proceed here, I’d really appreciate it . I think at this point I’ve made clear my main goal is being able to use ssl in the staging boxes before deploying to production, so… This shouldn’t be a very rare use-case, right?

Thanks in advance!

Osiris · June 6, 2020, 6:38pm

I would recommend expanding the existing certificates, not deleting them and making a new one. Although in the end it comes down to the same. Although if you delete the certs, try making the wildcards and for some reason don’t succeed, you’ve got zero certificates Always make a backup though.

For quite some time now.

Sure, I don’t see why not. It would mean using the “regular” hostnames though, not local.domain variants.

Not a very rare use case no. Sometimes people use their own CA for local developing, as it’s perfectly possible to include your own root on your own systems. However, using Let’s Encrypt certificates is possible too.

BPL · June 6, 2020, 6:58pm

Ok, so, in terms of certbot commands, how should i proceed to expand the existing ones?

That’s actually a wise advice… speaking of which… I think when I’d copied from production to staging I’d only copied the /etc/letsencrypt folder and I forgot to copy also the /var/lib/letsencrypt one, which seems to also contain *.pem files. Although I guess many of the errors when doing curl weren’t being produced by that fact though (speaking of which, certbox nginx metadata would contain subdomain support although i didn’t understand much of the syntax).

Really interesting, thanks, I’ll read through.

Mmm, just to be sure i’ve understood… you mean, before deploying i’d need to edit the hosts files so domain1, ... , domain7 would point out to the staging boxes, right? Mmm, if that’s the case, it seems like a quite unoptimal way of developing

Oh yeah, that’s the whole goal of this thread… I’d like to learn how to use the Let’s encrypt certificates on staging rather than some manual openssl generated ones…

In any case, please let me know how should I proceed from here so I can expand the existing ones to support wildcards.

Thanks!

Osiris · June 6, 2020, 7:27pm

Hm, my certbot hasn’t put pem files in /var/lib/letsencrypt, only some backups from the apache plugin. But backing up that directory doesn’t hurt.

For fully automated certificate generation, you’d need to use one of the DNS plugins. Which plugin you’ll need, depends on which DNS server is used for your domains. The plugin needs to add and remove TXT records to the DNS zone files of the domains for the validation required for a wildcard certificate. You can click on the DNS plugin from the list of plugins to go to the documentation of that plugin.

For example, if CloudFlares DNS services is used:

certbot --cert-name bpl.expert --dns-cloudflare --dns-cloudflare-credentials ~/.secrets/certbot/cloudflare.ini -d domain1 -d *.domain1 -d domain2 -d *.domain2 (also include for domain3 to 7 in the same manner obviously)

I myself run my own BIND and use the dns-rfc2136 plugin.

BPL · June 6, 2020, 8:31pm

Oh, cool… so certbot is a plugin based tool… in my case I think I’ll need to use digitalocean’s plugin. Btw, if I check the certbot plugins:

[root@remote1 ~]# certbot plugins
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
* nginx
Description: Nginx Web Server plugin
Interfaces: IAuthenticator, IInstaller, IPlugin
Entry point: nginx = certbot_nginx._internal.configurator:NginxConfigurator

* standalone
Description: Spin up a temporary webserver
Interfaces: IAuthenticator, IPlugin
Entry point: standalone = certbot._internal.plugins.standalone:Authenticator

* webroot
Description: Place files in webroot directory
Interfaces: IAuthenticator, IPlugin
Entry point: webroot = certbot._internal.plugins.webroot:Authenticator
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
[root@remote1 ~]#

It seems the certbot centos rpm package doesn’t include any additional plugin while the certbot github repo has by default many of them. I guess to install the https://pypi.org/project/certbot-dns-digitalocean/ I’d need to enable the virtualenv first and pip install it, right? Or is there a more “proper” way to install plugins?

Ok, cool, I’ve already generated a Digitalocean token so it seems it’ll be a matter of tweaking the command you’ve posted somehow after I install the digitalocean plugin… btw, let’s say i run the command you’ve posted but i don’t have the nginx files in the server, in that case certbot won’t be able to edit them but… would it still be possible to do that job with certbot or I’d run out of luck and I’d need to edit all the nginx config files myself manually?

BIND? What does that mean in this context?

Osiris · June 6, 2020, 9:03pm

Use your CentOS package manager. I previously linked to a list of packages earlier: https://pkgs.org/search/?q=python3-certbot-dns

Thats the DNS software I use, just as an example.

system · July 6, 2020, 8:59pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.