Certbot-auto renewal problem (again)

Help
1

My domains are:
svn.boberglund.com
video.boberglund.com

I ran this command:
sudo certbot-auto renew --dry-run

It produced this output:
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/svn.boberglund.com-0001.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cert is due for renewal, auto-renewing...
Plugins selected: Authenticator webroot, Installer None
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for svn.boberglund.com
Cleaning up challenges
Attempting to renew cert (svn.boberglund.com-0001) from /etc/letsencrypt/renewal/svn.boberglund.com-0001.conf produced an unexpected error: Missing command line flag or config entry for this setting:
Input the webroot for svn.boberglund.com:. Skipping.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/svn.boberglund.com.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cert is due for renewal, auto-renewing...
Plugins selected: Authenticator webroot, Installer None
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for svn.boberglund.com
http-01 challenge for video.boberglund.com
Cleaning up challenges
Attempting to renew cert (svn.boberglund.com) from /etc/letsencrypt/renewal/svn.boberglund.com.conf produced an unexpected error: Missing command line flag or config entry for this setting:
Select the webroot for svn.boberglund.com:
Choices: ['Enter a new webroot', '/var/www/html']

(You can set this with the --webroot-path flag). Skipping.
All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/svn.boberglund.com-0001/fullchain.pem (failure)
  /etc/letsencrypt/live/svn.boberglund.com/fullchain.pem (failure)

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
** DRY RUN: simulating 'certbot renew' close to cert expiry
**          (The test certificates below have not been saved.)

All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/svn.boberglund.com-0001/fullchain.pem (failure)
  /etc/letsencrypt/live/svn.boberglund.com/fullchain.pem (failure)
** DRY RUN: simulating 'certbot renew' close to cert expiry
**          (The test certificates above have not been saved.)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
2 renew failure(s), 0 parse failure(s)

My web server is (include version):
Apache/2.4.41 (Ubuntu)

The operating system my web server runs on is (include version):
Ubuntu 18.04.3 LTS

My hosting provider, if applicable, is:
Self-hosted

I can login to a root shell on my machine (yes or no, or I don’t know):
yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel):
no, just command line tools via PuTTY

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you’re using Certbot):
certbot --version
certbot 0.31.0

sudo certbot-auto --version
certbot 1.0.0 <== This is very strange, I thought it would be 0.37.0 or similar

I need to renew my certs again and like first time it does not work…
I have 3 different domains served by the same Apache server on my single incoming line with a dynamic IP address. So both of the listed domains above use the same IP address but are separate virtual servers and they are only open for https communications.
Additionally I have opened a 3rd virtual host with the domain name home.boberglund.com in order to have a location to be used for the certbot renewal since it seems to need access to the webroot for verification purposes.
Note that all the three subdomains are set to exactly the same external IP address.

The two virtual hosts I use the cert for are protected from external access by requiring a user/password login using basic authentication. So even when using https, the site requires login.

The first time I had created the certs and was checking the update function I had to ask here for help and my problem then was solved by advising me to use certbot-auto rather than certbot and to edit the conf file as shown in this thread.
Back in November I managed to renew these certs so they last until come January.
But now I am again at a loss on how to do it. I have not edited my files since then but I have updated Ubuntu several times so it is fully current. Could that have broken certbot-auto and/or certbot so it won’t work anymore?

Here is my renewal configuration file:
/etc/letsencrypt/renewal/svn.boberglund.com.conf

# renew_before_expiry = 30 days
version = 0.31.0
archive_dir = /etc/letsencrypt/archive/svn.boberglund.com
cert = /etc/letsencrypt/live/svn.boberglund.com/cert.pem
privkey = /etc/letsencrypt/live/svn.boberglund.com/privkey.pem
chain = /etc/letsencrypt/live/svn.boberglund.com/chain.pem
fullchain = /etc/letsencrypt/live/svn.boberglund.com/fullchain.pem

# Options used in the renewal process
[renewalparams]
account = b5058df1c187177209688fe263dcd9e9
pref_challs = http-01,
authenticator = webroot
server = https://acme-v02.api.letsencrypt.org/directory
[[webroot_map]]
video.boberglund.com = /var/www/html

Notice that the ONLY site reachable without login is home.boberglund.com, the two sites I need the cert for are both requiring user login.

2

http-01 challange need to have port 80 open to the internet. do you control DNS server of borberglund.com?

3

I have port 80 open on home.boberglund.com.
It is served from /var/www/html/
This URL will get you the Apache default page:
http://home.boberglund.com/apache_index.html
DNS for video.boberglund.com, svn.boberglund.com and home.boberglund.com all point to the same IP addres, currently 176.10.164.48

So what is needed is for certbot or certbot-auto to use /var/www/html/ as the directory to put test files into and access it via the IP address rather than the domain name:
http://176.10.164.48/apache_index.html
is proof of concept…

Or else use home.boberglund.com when validating the site address…

How can that be accomplished?

And why does certbot-auto --version report the version as 1.0.0 when it should have been something like 0.31.0 or similar???:

sudo certbot-auto --version
certbot 1.0.0

4

That's honestly not quite possible.
Can you show the output of the configuration file again?

/etc/letsencrypt/renewal/svn.boberglund.com-0001.conf

Because the software is currently on version 1.0.0?

Thank you

1 Like
5

See below.

Well, as far as I can remember I installed certbot-auto using wget to download a version that did not have a bug in parameter saving, so this software should not be affected by apt upgrades, right?

EDIT:
I checked them now and certbot is in /usr/bin/certbot whereas certbot-auto is in /usr/local/bin/

$ ls -l /usr/local/bin/certbot-auto
-rwxr-xr-x 1 root bosse 71650 Dec  4 00:27 /usr/local/bin/certbot-auto
$ ls -l /usr/bin/certbot
-rwxr-xr-x 1 root root 385 Mar 12  2019 /usr/bin/certbot

So in fact certbot-auto seems to have been upgraded whereas certbot is the same as before dated in March.

Here is the content of the asked for conf file, but why is it suffixed with "-0001"?:

cat /etc/letsencrypt/renewal/svn.boberglund.com-0001.conf
# renew_before_expiry = 30 days
version = 0.31.0
archive_dir = /etc/letsencrypt/archive/svn.boberglund.com-0001
cert = /etc/letsencrypt/live/svn.boberglund.com-0001/cert.pem
privkey = /etc/letsencrypt/live/svn.boberglund.com-0001/privkey.pem
chain = /etc/letsencrypt/live/svn.boberglund.com-0001/chain.pem
fullchain = /etc/letsencrypt/live/svn.boberglund.com-0001/fullchain.pem

# Options used in the renewal process
[renewalparams]
account = b5058df1c187177209688fe263dcd9e9
pref_challs = http-01,
authenticator = webroot
server = https://acme-v02.api.letsencrypt.org/directory
6

And: Is there a limitation to how sites must operate in order to use these certs?
For example, can they not be password protected at all?
(Using Apache basic auth)

7

QUESTION:
Can I create a new certificate to be used for all of my three sites where the main site is home.boberglund.com?
The home site does not need to be password protected so it would then be available on port 80.
What site is used for validation of a certificate containing multiple sites?
Right now I have a combined cert for svn and video, how can I add home and make it the primary one?
(By primary I mean the site where the renewal will happen concerning test files and such)

EDIT:
Concerning the conf file(s)
I have two conf files in the directory you pointed me to, these are named as follows:

$ ls -l /etc/letsencrypt/renewal/
total 8
-rw-r--r-- 1 root root 579 Oct 15 06:58 svn.boberglund.com-0001.conf
-rw-r--r-- 1 root root 607 Oct 15 07:02 svn.boberglund.com.conf

The content differs only in the use of suffix -0001 on files mentioned inside.
But there is also one content difference, this is only present in the suffix-free conf file:

[[webroot_map]]
video.boberglund.com = /var/www/html
8

Fixed a solution now!

So I have created a completely new cert using certbot-auto as follows:

sudo certbot-auto --duplicate certonly -d home.boberglund.com,video.boberglund.com,svn.boberglund.com

The home.boberglund.com site is just a test site and is the only one on this server that is open for port 80.

During execution of the command I had to select the following:
Authentication
3: Place files in webroot directory (webroot)

Webroot for home.boberglund.com: /var/www/html
Webroot for video.boberglund.com: /var/www/html
Webroot for svn.boberglund.com: /var/www/html

(So same webroot for all 3 sites)

Then it completed successfully and I got a message like this:

  Congratulations! Your certificate and chain have been saved at:
   /etc/letsencrypt/live/home.boberglund.com/fullchain.pem
   Your key file has been saved at:
   /etc/letsencrypt/live/home.boberglund.com/privkey.pem
   Your cert will expire on 2020-03-26. To obtain a new or tweaked
   version of this certificate in the future, simply run certbot-auto
   again. To non-interactively renew *all* of your certificates, run
   "certbot-auto renew"

Next I used certbot-auto delete to remove the soon to expire cert for svn.boberglund.com.

Then I edited the sites-available files for svn and video and changed the certificate entries there to point to the new certificate files shown above.
Reloaded Apache and checked that I could reach the corresponding websites and that Firefox now shows them to expire on March 26 rather than next week.

Finally I tested the renew command shown in the exit message from certbot-auto above:

sudo certbot-auto renew --dry-run
....
Congratulations, all renewals succeeded. The following certs have been renewed:
  /etc/letsencrypt/live/home.boberglund.com/fullchain.pem (success)
.....

Finally done! And now I could put the command for renewal into the sudo crontab, I guess?
Like so to run every night at 3:15:

sudo crontab -e
15 3 * * * certbot-auto renew

EDIT:
Forgot to show the renew conf that was saved by the new cert creation:

# renew_before_expiry = 30 days
version = 1.0.0
archive_dir = /etc/letsencrypt/archive/home.boberglund.com
cert = /etc/letsencrypt/live/home.boberglund.com/cert.pem
privkey = /etc/letsencrypt/live/home.boberglund.com/privkey.pem
chain = /etc/letsencrypt/live/home.boberglund.com/chain.pem
fullchain = /etc/letsencrypt/live/home.boberglund.com/fullchain.pem

# Options used in the renewal process
[renewalparams]
authenticator = webroot
account = b5058df1c187177209688fe263dcd9e9
server = https://acme-v02.api.letsencrypt.org/directory
[[webroot_map]]
video.boberglund.com = /var/www/html
svn.boberglund.com = /var/www/html
home.boberglund.com = /var/www/html

Notice that there are the three webroots automatically inserted as they were not earlier.

2 Likes
9

AUTOMATIC RENEWAL SYSTEM?
So now that the certbot-auto manual renewal seems to work I was looking at the way to automate the process.
I have discovered that there is a process running from /etc/cron.d/certbot, which uses an old certbot. The script looks like this:

SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

0 */12 * * * root test -x /usr/bin/certbot -a \! -d /run/systemd/system && perl -e 'sleep int(rand(43200))' && certbot -q renew

I think it was put there by an early certbot installation attempt made in August.
Clearly it is not really working well since I find this in the logfile:

2019-12-28 05:24:12,196:DEBUG:certbot.main:certbot version: 0.31.0
2019-12-28 05:24:12,198:DEBUG:certbot.main:Arguments: ['-q']
2019-12-28 05:24:12,199:DEBUG:certbot.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#apache,PluginEntryPoint#manual,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
2019-12-28 05:24:12,214:DEBUG:certbot.log:Root logging level set at 30
2019-12-28 05:24:12,216:INFO:certbot.log:Saving debug log to /var/log/letsencrypt/letsencrypt.log
2019-12-28 05:24:12,218:INFO:certbot.storage:Attempting to parse the version 1.0.0 renewal configuration file found at /etc/letsencrypt/renewal/home.boberglund.com.conf with version 0.31.0 of Certbot. This might not work.

So it uses an older version of certbot rather than the certbot-auto I have tested with.
And it warns against a version difference between itself and the renewal script.

How should I change this such that the /etc/cron.d/certbot script uses certbot-auto instead?
Can I simply just replace certbot with certbot-auto in the command line?
And do I then have to “activate” the new version in some way?

1 Like
10

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.