Certbot 1.29 missing a required file reference

Help
1

My domain is: grembeirn.duckdns.org

I ran this command: certbot renew --dns-duckdns-credentials /etc/letsencrypt/renewal/grembeirn.duckdns.org.conf

It produced this output: Renewal configuration file /etc/letsencrypt/renewal/grembeirn.duckdns.org.conf is broken.
The error was: renewal config file {......} is missing a required file reference

My web server is (include version): nginx

The operating system my web server runs on is (include version): Raspbian kernel v5.10.103

My hosting provider, if applicable, is:

I can login to a root shell on my machine (yes or no, or I don't know): yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): v1.29

Running weekly pip to keep my certbot up to date:

/opt/certbot/bin/python3 -m pip install --upgrade pip
/opt/certbot/bin/pip install --upgrade certbot
pip3 install certbot_dns_duckdns -U

The config file /etc/letsencrypt/renewal/grembeirn.duckdns.org.conf was no longer there. And the directory /etc/letsencrypt/renewal/ was gone.
Have recreated the directory and put the conf file back from a backup (content untouched).

The folder /etc/letsencrypt/live/grembeirn.duckdns.org was also gone. Recreated the folder and also copied working keys privkey.pem and fullchain.pem into the folder.

But I no longer seem to have a cert.pem and chain.pem file in the folder.

Also recreated the archive folder.

Content of the config file

# renew_before_expiry = 30 days
version = 1.29.0
archive_dir = /etc/letsencrypt/archive/grembeirn.duckdns.org
cert = /etc/letsencrypt/live/grembeirn.duckdns.org/cert.pem
privkey = /etc/letsencrypt/live/grembeirn.duckdns.org/privkey.pem
chain = /etc/letsencrypt/live/grembeirn.duckdns.org/chain.pem
fullchain = /etc/letsencrypt/live/grembeirn.duckdns.org/fullchain.pem

# Options used in the renewal process
[renewalparams]
account = acountid
pref_challs = dns-01,
authenticator = dns-duckdns
dns_duckdns_propagation_seconds = 60
dns_duckdns_token = tokenid
server = https://acme-v02.api.letsencrypt.org/directory
dns_duckdns_credentials = /etc/letsencrypt/renewal/grembeirn.duckdns.org.conf
key_type = rsa
2

The files in /live/ are (relative) symbolic links to the corresponding directory in the /archive/ directory. E.g., /live/example.com/privkey.pem is a symbolic link to ../../archive/example.com/privkey.pem. Certbot expects those symbolic links to be actual symbolic links.

cert.pem is simply the first certificate from fullchain.pem and chain.pem are the certificates after the first one.

I guess it didn't. Certbot is currently on version 2.9.0, also on PyPi. 1.29.0 is almost 2 years old.

I'm guessing that's not actually the command you've been using, as it doesn't make much sense to provide the certbot-dns-duckdns a Certbot renewal configuration file as the plugin credentials :wink: Also, the option is saved into the renewal configuration file, so you don't have to mention it again with the renew subcommand. Unless you need to update it, as you need to do currently :slight_smile:

1 Like
3

Believe me, it is an exact copy paste of the command which is in my script which runs every week:

sudo /opt/certbot/bin/python3 -m pip install --upgrade pip
sudo /opt/certbot/bin/pip install --upgrade certbot
sudo pip3 install certbot_dns_duckdns -U
sudo certbot renew --dns-duckdns-credentials /etc/letsencrypt/renewal/grembeirn.duckdns.org.conf

Output I would get is along the lines of:

 sudo /opt/certbot/bin/pip install --upgrade certbot
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
...
pip3 install certbot_dns_duckdns -U
Requirement already satisfied: pip in /opt/certbot/lib/python3.7/site-packages (24.0)
...
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Requirement already satisfied: certbot in /opt/certbot/lib/python3.7/site-packages (2.7.4)
...
pip3 install certbot_dns_duckdns -U
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Requirement already up-to-date: certbot_dns_duckdns in /usr/local/lib/python3.7/dist-packages (1.3)
...
certbot renew --dns-duckdns-credentials /etc/letsencrypt/renewal/grembeirn.duckdns.org.conf
Renewal configuration file /etc/letsencrypt/renewal/grembeirn.duckdns.org.conf is broken.

And I was getting regular updates. But I guess there has been a mix up of some sort. I once tried getting Certbot and DuckDNS plugin running with snap but that never worked. Using pip I got it working. I recently purged all remains of snap. I guess that's where things went wrong.

Anyway, water under the bridge. How do I get this thing rolling again?

When going from scratch following the exact instructions from [Certbot Instructions | Certbot]) I end up with version 1.29 again.

4

Can you show the contents of that file. Because as @osiris already noted that is probably not the file with your duckdns security settings.

Okay to redact account numbers and of course any passwords you might have put in there. Normally Certbot makes that file but in your case something odd is happening.

4 Likes
5

Yikes!

I would change this line:

to just this:
sudo certbot renew

4 Likes
6

@rg305

Executing just

sudo certbot renew

Gives the same result:

Processing /etc/letsencrypt/renewal/grembeirn.duckdns.org.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Renewal configuration file /etc/letsencrypt/renewal/grembeirn.duckdns.org.conf is broken.
The error was: renewal config file  {'version': '1.29.0 archive_dir = /etc/letsencrypt/archive/grembeirn.duckdns.org cert = /etc/letsencrypt/live/grembeirn.duckdns.org/cert.pem privkey = /etc/letsencrypt/live/grembeirn.duckdns.org/privkey.pem chain =', '/etc/letsencrypt/live/grembeirn.duckdns.org/chain.pem fullchain': '/etc/letsencrypt/live/grembeirn.duckdns.org/fullchain.pem', '[renewalparams] account': ['accountid pref_challs = dns-01', 'authenticator = dns-duckdns dns_duckdns_propagation_seconds = 60 dns_duckdns_token = tokenid server ='], 'https://acme-v02.api.letsencrypt.org/directory dns_duckdns_credentials': '/etc/letsencrypt/renewal/grembeirn.duckdns.org.conf key_type = rsa'} is missing a required file reference
Skipping.

I've redacted the accountid and tokenid in the output.

@MikeMcQ, the content of the config file is as follows:

# renew_before_expiry = 30 days
version = 1.29.0
archive_dir = /etc/letsencrypt/archive/grembeirn.duckdns.org
cert = /etc/letsencrypt/live/grembeirn.duckdns.org/cert.pem
privkey = /etc/letsencrypt/live/grembeirn.duckdns.org/privkey.pem
chain = /etc/letsencrypt/live/grembeirn.duckdns.org/chain.pem
fullchain = /etc/letsencrypt/live/grembeirn.duckdns.org/fullchain.pem

# Options used in the renewal process
[renewalparams]
account = acountid
pref_challs = dns-01,
authenticator = dns-duckdns
dns_duckdns_propagation_seconds = 60
dns_duckdns_token = tokenid
server = https://acme-v02.api.letsencrypt.org/directory
dns_duckdns_credentials = /etc/letsencrypt/renewal/grembeirn.duckdns.org.conf
key_type = rsa
7

That's a very interesting circle going on there: your renewal configuration file is refering to itself for the credentials, but it's lacking those credentials.

3 Likes
8

Let's ignore the duckdns security settings because the token value might work in the certbot renewal conf file.

But, your Certbot folders are still faulty. It looks like you tried to manually recover after the needed files went missing. But, did not create all the required symlinks are supporting folders.

In your case I think the easiest way forward is to start over. And, use the proper command for renew which is just certbot renew.

So, delete all the folders under /etc/letsencrypt and re-run the initial command (below).

Do this with --dry-run first and let us know if this works before doing anything else

sudo certbot certonly --dry-run --preferred-challenges dns --authenticator dns-duckdns --dns-duckdns-token <your-duckdns-token> --dns-duckdns-propagation-seconds 60 -d grembeirn.duckdns.org

Edit: Add -d (domain) to command above. Probably would have prompted you anyway

4 Likes
9

You are installing from PIP, which means your Certbot version will be limited by the underlying Python version in your environment.

That being said, 1.29 requires python 3.7 - which is the minimum python into the 2.7.x series.

Your pip might be trying to install from a local cache only. Try upgrading pip and certbot consecutively:

pip install --upgrade pip
pip install --upgrade certbot

That may get you a newer pip and wipe out whatever configuration setting is using the old certbot.

3 Likes
10

@jvanasco I'm already doing the pip upgrade and certbot upgrade on a weekly basis.

When executing the suggested commands again, I get following output:

 sudo /opt/certbot/bin/pip install --upgrade certbot
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Requirement already satisfied: certbot in /opt/certbot/lib/python3.7/site-packages (2.7.4)

Followed by a bunch of similar "Requirement already satisfied" statements.
And the second command returns:

 sudo /opt/certbot/bin/pip install --upgrade pip
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Requirement already satisfied: pip in /opt/certbot/lib/python3.7/site-packages (24.0)

BTW, I've noticed the following:

sudo /opt/certbot/bin/certbot --version
certbot 2.7.4
sudo certbot --version
certbot 1.29.0

So for some reason I have two versions floating around. How to get rid of the old version?

BTW, when doing the renew command using explicitly the most recent version, the error about the config file is the same. So the behaviour of both versions is identical.

1 Like
11

You need to identify how you installed that other Certbot. Try which certbot to see the different paths to the two certbots. Then find out how the other one was installed (probably using apt although I can't find a Debian version with 1.29.0).

1 Like
12

@Osiris To be clear, these commands have been working successfully for the past 2 years. The certs got updated every 60d with the help of my script doing these checks (amongst other updates like apt-get etc.)

What can I say :person_shrugging:

13

type this:

which certbot

That will show you what the certbot command is invoking and where.

I would NOT remove the old Certbot without knowing who/how/why it was installed.

Instead, I would update the shell configuration file to prefer the new certbot. In a bash profile, it would be something like:

PATH="/opt/certbot/bin/certbot:${PATH}"
export PATH

That would be towards the end of the file, as it needs to appear after anything that might put the unwanted certbot in the front.

This won't be active in the current terminal window without a reread/reconfigure, but should apply to new windows. Typing which certbot should show you the preferred path.

As the profile only works for a specific user, this needs to be done for each user that invokes certbot.

Edit: You'll also need to inspect your FS to see if these are both saving into /etc/letsencrypt or if the /opt installation is saving files into it's own location like /opt/certbot/etc/letsencrypt

3 Likes
14

@MikeMcQ
Running the certonly option, I got this:

sudo /opt/certbot/bin/certbot certonly --dry-run --preferred-challenges dns --authenticator dns-duckdns --dns-duckdns-token token
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Python 3.7 support will be dropped in the next planned release of Certbot - please upgrade your Python version.
Account registered.
Please enter the domain name(s) you would like on your certificate (comma and/or
space separated) (Enter 'c' to cancel): grembeirn.duckdns.org
Simulating a certificate request for grembeirn.duckdns.org
Waiting 30 seconds for DNS changes to propagate
The dry run was successful.

I haven't done anything else next.

15

Excellent. Before we proceed you should sort out which Certbot version and install type you will use. You have been given good advice about that and I don't have more to add about that.

Once you get that sorted run this and show results of this

sudo certbot --version
3 Likes
16

@jvanasco

updated the .profile at the end to state

# fix an issue with duplicate certbot versions
if [ -d "/opt/certbot/bin" ] ; then
    PATH="/opt/certbot/bin:$PATH"
fi

After logout/login I can see the path /opt/certbot/bin is part of my $PATH variable

I now get

which certbot
/opt/certbot/bin/certbot
certbot --version
certbot 2.7.4
sudo which certbot
/usr/local/bin/certbot
sudo certbot --version
certbot 1.29

So it seems sudo goes to the other old version. The certbots I located on my system are (using locate):

/opt/certbot/bin/certbot (version 2.7.4)
/usr/bin/certbot (version 2.7.4)
/usr/local/bin/certbot (version 1.29)

Updated the sudo $PATH variable with visudo.

Now I'm getting

sudo which certbot
/opt/certbot/bin/certbot
sudo certbot --version
certbot 2.7.4

I guess I'm got to go for the next steps.

1 Like
17

Looks good.

Just do a quick check in /etc/letsencrypt and under /opt to ensure everything is being saved into /etc/letsencrypt. I don't think Certbot can be installed with a different default (under /opt) and you must invoke it with alternate paths, but I am not sure.

3 Likes
18

Yes, looks good. Try same command as above but without --dry-run then

sudo certbot certonly --preferred-challenges dns --authenticator dns-duckdns --dns-duckdns-token <your-duckdns-token> --dns-duckdns-propagation-seconds 60 -d grembeirn.duckdns.org
4 Likes
19

@MikeMcQ
Have done:

 sudo certbot certonly --preferred-challenges dns --authenticator dns-duckdns --dns-duckdns-token tokenid

Which in the end resulted in:

Successfully received certificate.
Certificate is saved at: /etc/letsencrypt/live/grembeirn.duckdns.org-0001/fullchain.pem
Key is saved at:         /etc/letsencrypt/live/grembeirn.duckdns.org-0001/privkey.pem
This certificate expires on 2024-06-11.
2 Likes
20

I don't know why it would have created a -0001 version if you had deleted the subfolders. Or even at all since that was the same domain name in the prior cert.

What does this say now

sudo certbot certificates
4 Likes