Certbot broken, need help to fix it

morten · July 11, 2019, 12:22pm

My certbot installation is broken, and I am unable to fix it.

root@stage:~# certbot
An unexpected error occurred:
pkg_resources.VersionConflict: (certbot 0.31.0 (/usr/lib/python3/dist-packages), Requirement.parse('certbot>=0.34.0'))
Please see the logfile '/tmp/tmpbyuigqhc' for more details.

The file '/tmp/tmpbyuigqhc' contains this:

2019-07-11 10:55:42,014:DEBUG:certbot.log:Exiting abnormally:
Traceback (most recent call last):
File "/usr/bin/certbot", line 11, in
load_entry_point('certbot==0.31.0', 'console_scripts', 'certbot')()
File "/usr/lib/python3/dist-packages/certbot/main.py", line 1335, in main
plugins = plugins_disco.PluginsRegistry.find_all()
File "/usr/lib/python3/dist-packages/certbot/plugins/disco.py", line 206, in find_all
plugin_ep = PluginEntryPoint(entry_point)
File "/usr/lib/python3/dist-packages/certbot/plugins/disco.py", line 52, in init
self.plugin_cls = entry_point.load()
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 2290, in load
self.require(*args, **kwargs)
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 2307, in require
items = working_set.resolve(reqs, env, installer)
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 854, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.VersionConflict: (certbot 0.31.0 (/usr/lib/python3/dist-packages), Requirement.parse('certbot>=0.34.0'))
2019-07-11 10:55:42,014:ERROR:certbot.log:An unexpected error occurred:

Probably my certbot installation got broken as part of fiddling with installing different certbot versions (perhaps even distributions) and certbot-auto.

Reinstalling certbot does not solve the issue.

Maybe certbot is wrongfully configured somehow, e.g. to use an invalid python installation?

Phil · July 11, 2019, 2:09pm

Hi @morten,

Can you work through removing all of the versions of certbot you can find via your systems package manager, certbot-auto, pip, etc?

morten · July 11, 2019, 3:21pm

Hi @Phil

I already removed the pip and certbot-auto installations, so there is only one certbot executable now:

root@stage:~# which certbot
/usr/bin/certbot

I installed this certbot as follows:

apt-get install python-certbot-nginx
apt-get install python3-certbot-dns-cloudflare

I have installed certbot on several other servers using these commands and it works fine on all of them (because the certbot is not broken there).

The package manager reports a lot. Not sure if below is helpful.

root@stage:~# dpkg ii certbot ii python-certbot-nginx ii python3-acme ii python3-asn1crypto ii python3-certbot ii python3-certbot-dns-cloudflare ii python3-certbot-nginx ii python3-certifi ii python3-cffi-backend ii python3-chardet ii python3-configargparse ii python3-configobj ii python3-cryptography ii python3-future ii python3-idna ii python3-josepy ii python3-ndg-httpsclient ii python3-openssl ii python3-parsedatetime ii python3-pkg-resources ii python3-pyasn1 ii python3-requests ii python3-requests-toolbelt ii python3-rfc3339 ii python3-setuptools ii python3-six ii python3-urllib3 ii python3-zope.component ii python3-zope.hookable ii python3-zope.interface -l | grep -e certbot
0.31.0-1+ubuntu16.04.1+certbot+1 all automatically configure HTTPS using Let's Encrypt
0.31.0-1+ubuntu16.04.1+certbot+1 all transitional dummy package
0.31.0-1+ubuntu16.04.1+certbot+1 all ACME protocol library for Python 3
0.22.0-2+ubuntu16.04.1+certbot+1 all Fast ASN.1 parser and serializer (Python 3)
0.31.0-1+ubuntu16.04.1+certbot+1 all main library for certbot
0.23.0-1ubuntu0ppa1~ubuntu16.04.1 all Cloudflare DNS plugin for Certbot
0.31.0-1+ubuntu16.04.1+certbot+1 all Nginx plugin for Certbot
2017.4.17-2+ubuntu16.04.1+certbot+1 all root certificates for validating SSL certs and verifying TLS hosts (python3)
1.10.0-0.1+ubuntu16.04.1+certbot+1 amd64 Foreign Function Interface for Python 3 calling C code - runtime
3.0.4-1+ubuntu16.04.1+certbot+2 all universal character encoding detector for Python3
0.11.0-1+certbot~xenial+1 all replacement for argparse with config files and environment variables (Python 3)
5.0.6-2+ubuntu16.04.1+certbot+1 all simple but powerful config file reader and writer for Python 3
1.9-1+ubuntu16.04.1+certbot+2 amd64 Python library exposing cryptographic recipes and primitives (Python 3)
0.15.2-4+ubuntu16.04.1+certbot+3 all Clean single-source support for Python 3 and 2 - Python 3.x
2.5-1+ubuntu16.04.1+certbot+1 all Python IDNA2008 (RFC 5891) handling (Python 3)
1.1.0-2+ubuntu16.04.1+certbot+1 all JOSE implementation for Python 3.x
0.4.2-1+certbot~xenial+1 all enhanced HTTPS support for httplib and urllib2 using PyOpenSSL for Python3
17.3.0-1~0+ubuntu16.04.1+certbot+1 all Python 3 wrapper around the OpenSSL library
2.4-3+ubuntu16.04.1+certbot+3 all Python 3 module to parse human-readable date/time expressions
33.1.1-1+certbot~xenial+1 all Package Discovery and Resource Access using pkg_resources
0.1.9-2+certbot~xenial+1 all ASN.1 library for Python (Python 3 module)
2.18.1-1+ubuntu16.04.1+certbot+1 all elegant and simple HTTP library for Python3, built for human beings
0.8.0-1+ubuntu16.04.1+certbot+1 all Utility belt for advanced users of python3-requests
1.0-4+certbot~xenial+1 all parser and generator of RFC 3339-compliant timestamps (Python 3)
33.1.1-1+certbot~xenial+1 all Python3 Distutils Enhancements
1.11.0-1+ubuntu16.04.1+certbot+1 all Python 2 and 3 compatibility library (Python 3 interface)
1.21.1-1+ubuntu16.04.1+certbot+1 all HTTP library with thread-safe connection pooling for Python3
4.3.0-1+ubuntu16.04.1+certbot+3 all Zope Component Architecture
4.0.4-4+ubuntu16.04.1+certbot+1 amd64 Hookable object support
4.3.2-1+ubuntu16.04.1+certbot+1 amd64 Interfaces for Python3

I think this error message hints at the solution:

pkg_resources.VersionConflict: (certbot 0.31.0 (/usr/lib/python3/dist-packages), Requirement.parse('certbot>=0.34.0'))

It is indeed certbot 0.31.0 that I have installed. But where does this "Requirement.parse('certbot>=0.34.0')" come from and why is it still here? I am guessing this is a leftover from a previous certbot installation that was not properly removed/purged when I uninstalled it.

morten · July 11, 2019, 3:40pm

I searched for the "0.34.0" string in /usr and it revealed the following:

root@stage:/usr# grep -r '0.34.0' .
./local/lib/python2.7/dist-packages/certbot_dns_cloudflare-0.35.1.dist-info/METADATA:Requires-Dist: certbot (>=0.34.0)
./local/lib/python3.5/dist-packages/certbot_dns_cloudflare-0.35.1.dist-info/METADATA:Requires-Dist: certbot (>=0.34.0)

This seems to be a leftover from my previous certbot-auto installation, which was (certbot) version 0.35.1.

Not sure if this is what is breaking my current certbot 0.31.0 installation, though.

Phil · July 11, 2019, 3:51pm

@morten,

Since it’s already broken, let’s try getting rid of those certbot packages too so we can start fresh.

apt purge certbot

morten · July 11, 2019, 4:41pm

Purging and reinstalling doesn't help.

root@stage:~# apt purge certbot
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following package was automatically installed and is no longer required:
python3-pyparsing
Use 'apt autoremove' to remove it.
The following packages will be REMOVED:
certbot* python-certbot-nginx* python3-certbot-nginx*
0 upgraded, 0 newly installed, 3 to remove and 1 not upgraded.
After this operation, 340 kB disk space will be freed.
Do you want to continue? [Y/n] Y
(Reading database ... 122559 files and directories currently installed.)
Removing python-certbot-nginx (0.31.0-1+ubuntu16.04.1+certbot+1) ...
Removing python3-certbot-nginx (0.31.0-1+ubuntu16.04.1+certbot+1) ...
Removing certbot (0.31.0-1+ubuntu16.04.1+certbot+1) ...
Purging configuration files for certbot (0.31.0-1+ubuntu16.04.1+certbot+1) ...
Processing triggers for man-db (2.7.5-1) ...
root@stage:~# apt-get install python-certbot-nginx
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
certbot python3-certbot-nginx
Suggested packages:
python3-certbot-apache python-certbot-doc python-certbot-nginx-doc
The following NEW packages will be installed:
certbot python-certbot-nginx python3-certbot-nginx
0 upgraded, 3 newly installed, 0 to remove and 1 not upgraded.
Need to get 0 B/64.2 kB of archives.
After this operation, 340 kB of additional disk space will be used.
Do you want to continue? [Y/n] Y
Selecting previously unselected package certbot.
(Reading database ... 122510 files and directories currently installed.)
Preparing to unpack .../certbot_0.31.0-1+ubuntu16.04.1+certbot+1_all.deb ...
Unpacking certbot (0.31.0-1+ubuntu16.04.1+certbot+1) ...
Selecting previously unselected package python3-certbot-nginx.
Preparing to unpack .../python3-certbot-nginx_0.31.0-1+ubuntu16.04.1+certbot+1_all.deb ...
Unpacking python3-certbot-nginx (0.31.0-1+ubuntu16.04.1+certbot+1) ...
Selecting previously unselected package python-certbot-nginx.
Preparing to unpack .../python-certbot-nginx_0.31.0-1+ubuntu16.04.1+certbot+1_all.deb ...
Unpacking python-certbot-nginx (0.31.0-1+ubuntu16.04.1+certbot+1) ...
Processing triggers for man-db (2.7.5-1) ...
Setting up certbot (0.31.0-1+ubuntu16.04.1+certbot+1) ...
certbot.service is a disabled or a static unit, not starting it.
Setting up python3-certbot-nginx (0.31.0-1+ubuntu16.04.1+certbot+1) ...
Setting up python-certbot-nginx (0.31.0-1+ubuntu16.04.1+certbot+1) ...
root@stage:~# certbot
An unexpected error occurred:
pkg_resources.VersionConflict: (certbot 0.31.0 (/usr/lib/python3/dist-packages), Requirement.parse('certbot>=0.34.0'))
Please see the logfile '/tmp/tmpwmteb61w' for more details.

bmw · July 11, 2019, 5:33pm

The problem here is the certbot-dns-cloudflare installation left over from pip. You can tell it’s from pip instead of certbot-auto or apt because it’s in /usr/local.

When Certbot runs, it’s searching your Python environment for any Certbot plugins, finding the Cloudflare plugin, and trying to load it which fails because it has unsatisfied dependencies.

Removing this plugin through pip or manually deleting these files should solve your problem.

morten · July 11, 2019, 6:07pm

It did

I apparently had two “left-over” packages from the pip certbot-dns-cloudflare installation:

certbot-dns-cloudflare (for Python 2.7)
certbot-dns-cloudflare (for Python 3.5)

The first I could remove properly via “pip uninstall”. The second I removed with “rm -rf”.

And then certbot worked again.

Thanks for the help!

Phil · July 11, 2019, 7:24pm

Thanks for the help @bmw!

system · August 10, 2019, 7:34pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.