Certbot Python error on Ubuntu 16.04: ImportError: No module named ordered_dict

[hunterbuchanan]

I ran this command: certbot

It produced this output:

Traceback (most recent call last): File "/usr/bin/certbot", line 9, in <module> load_entry_point('certbot==0.27.0', 'console_scripts', 'certbot')() File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 542, in load_entry_point return get_distribution(dist).load_entry_point(group, name) File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2569, in load_entry_point return ep.load() File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2229, in load return self.resolve() File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2235, in resolve module = __import__(self.module_name, fromlist=['__name__'], level=0) File "/usr/lib/python2.7/dist-packages/certbot/main.py", line 18, in <module> from certbot import account File "/usr/lib/python2.7/dist-packages/certbot/account.py", line 18, in <module> from acme import messages File "/usr/lib/python2.7/dist-packages/acme/messages.py", line 11, in <module> from acme import challenges File "/usr/lib/python2.7/dist-packages/acme/challenges.py", line 12, in <module> import requests File "/usr/lib/python2.7/dist-packages/requests/__init__.py", line 58, in <module> from . import utils File "/usr/lib/python2.7/dist-packages/requests/utils.py", line 26, in <module> from .compat import parse_http_list as _parse_list_header File "/usr/lib/python2.7/dist-packages/requests/compat.py", line 42, in <module> from .packages.urllib3.packages.ordered_dict import OrderedDict ImportError: No module named ordered_dict

My web server is (include version): Apache 2.4.46

The operating system my web server runs on is (include version): Ubuntu 16.04

I can login to a root shell on my machine (yes or no, or I don't know): yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): Can't run certbot --version but the Apt package version is 0.27.0-1~ubuntu16.04.1

Pretty straightforward. I seem to have all dependencies properly installed, but I can't run certbot at all without running into this python module error. It works fine on another server with a very similar setup and the same software versions. Not sure how to troubleshoot further.

Edit: Removed domain, it's irrelevant to the issue.

[_az]

I would guess that somebody installed an older version of urllib3 into the system Python directory using pip.

Try:

pip3 remove urllib3
apt install --reinstall python3-urllib3

For what it's worth, the Certbot snap package does a great job at preventing these types of dependency issues, as well as giving you a dramatically newer version.

[hunterbuchanan]

Thanks in advance. This is on one of our production web servers, we don't have snap installed at all and don't intend to install it for now. But it's good to know it might be an easier solution here.

Actually, it looks like we may have a bigger python issue here. Certbot is using Python 2, but whether I use pip or pip3 to try to remove urllib3, I run into problems:

$ sudo pip remove urllib3
Traceback (most recent call last):
  File "/usr/local/bin/pip", line 9, in <module>
    load_entry_point('pip==21.0.1', 'console_scripts', 'pip')()
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 542, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2569, in load_entry_point
    return ep.load()
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2229, in load
    return self.resolve()
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2235, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/local/lib/python2.7/dist-packages/pip/_internal/cli/main.py", line 60
    sys.stderr.write(f"ERROR: {exc}")
                                   ^
SyntaxError: invalid syntax

$ sudo pip3 remove urllib3
Traceback (most recent call last):
  File "/usr/local/bin/pip3", line 7, in <module>
    from pip._internal import main
ImportError: No module named 'pip._internal'

[_az]

@hunterbuchanan

Certbot is using Python 2

Oops! I my tired eyes yesterday missed that, sorry.

That explains why I couldn't reproduce the exact same error when I tried. But with Python 2, it's reproducible by breaking the packages with with pip install --upgrade urllib3.

I think the same advice basically applies:

sudo pip uninstall urllib3
sudo apt install --reinstall python-urllib3

[hunterbuchanan]

To clarify, I get this exact error message when I try to do anything at all with pip, whether it's sudo pip uninstall urllib3 or sudo pip or just pip:

Traceback (most recent call last):
  File "/usr/local/bin/pip", line 9, in <module>
    load_entry_point('pip==21.0.1', 'console_scripts', 'pip')()
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 542, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2569, in load_entry_point
    return ep.load()
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2229, in load
    return self.resolve()
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2235, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/local/lib/python2.7/dist-packages/pip/_internal/cli/main.py", line 60
    sys.stderr.write(f"ERROR: {exc}")
                                   ^
SyntaxError: invalid syntax

It looks like I have a bigger problem with pip/python on this server.

[_az]

@hunterbuchanan

"/usr/local/bin/pip"

Try calling /usr/bin/pip directly instead.

/usr/local/bin/pip is an external installation of pip and probably won't help you.

[hunterbuchanan]

OK that was a great catch! The bad news is, I get nearly the same error message from running /usr/bin/pip (with any combination of commands), just a bit shorter:

$ /usr/bin/pip
Traceback (most recent call last):
  File "/usr/bin/pip", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/dist-packages/pip/__init__.py", line 16, in main
    from pip._internal.utils.entrypoints import _wrapper
  File "/usr/local/lib/python2.7/dist-packages/pip/_internal/utils/entrypoints.py", line 3, in <module>
    from pip._internal.cli.main import main
  File "/usr/local/lib/python2.7/dist-packages/pip/_internal/cli/main.py", line 60
    sys.stderr.write(f"ERROR: {exc}")
                                   ^
SyntaxError: invalid syntax

[_az]

Grr I'm not actually sure how to repair your system.

You need to downgrade Python 2's pip back to a version (pip==20.3) that supports Python 2 , because the one you have installed in /usr/local/lib/python2.7/dist-packages is too new.

However, you can't easily downgrade it when pip itself is broken.

The only thing that comes to mind is to remove the pip directories from that /usr/local/lib/python2.7/dist-packages/, which will basically remove the externally-installed pip. But this is just off-the-top-of-my-head advice, I wouldn't trust it.

There's also some other advice here: pip install on Python 2 fails with SyntaxError: sys.stderr.write(f"ERROR: {exc}") · Issue #9500 · pypa/pip · GitHub

[hunterbuchanan]

You were right! I moved /usr/local/bin/pip and /usr/local/lib/python2.7/dist-packages/pip/ out of their respective locations, and the correct version of pip allowed me to remove the wrong version of urllib3. Then reinstalling it via apt did the trick. I'm sure at some point I used pip to update too many python2 packages willy-nilly.

Thanks so much for the time and the help! Much appreciated.