ModuleNotFoundError

Hi! I'm trying to develop a certbot plugin to automate the process of getting wildcard certificates for domains provided by DreamHost.

I have already wrote the plugin and released it to pypi:

I guess that my problem must be in setup.py, but I can't understand whats wrong.

I have based my plugin on others such as:

I have installed certbot using snapd:

sudo snap install core; sudo snap refresh core
sudo snap install --classic certbot
sudo ln -sf /snap/bin/certbot /usr/bin/certbot

Then I installed my package using sudo pip install certbot-dns-dreamhost

Until this point I think I made no mistakes in the instalation, but when I run sudo certbot plugins to check if my plugin has installed correctly I get the error:

An unexpected error occurred:
ModuleNotFoundError: No module named 'certbot_dns_dreamhost'
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /tmp/certbot-log-g7d5c__k/log or re-run Certbot with -v for more details.

And if I check the content from the file with cat /tmp/certbot-log-g7d5c__k/log

2022-11-18 20:44:38,355:DEBUG:certbot._internal.log:Exiting abnormally:
Traceback (most recent call last):
  File "/usr/local/bin/certbot", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.10/dist-packages/certbot/main.py", line 19, in main
    return internal_main.main(cli_args)
  File "/usr/local/lib/python3.10/dist-packages/certbot/_internal/main.py", line 1705, in main
    plugins = plugins_disco.PluginsRegistry.find_all()
  File "/usr/local/lib/python3.10/dist-packages/certbot/_internal/plugins/disco.py", line 241, in find_all
    plugin_ep = cls._load_entry_point(entry_point, plugins, with_prefix=False)
  File "/usr/local/lib/python3.10/dist-packages/certbot/_internal/plugins/disco.py", line 261, in _load_entry_point
    plugin_ep = PluginEntryPoint(entry_point, with_prefix)
  File "/usr/local/lib/python3.10/dist-packages/certbot/_internal/plugins/disco.py", line 60, in __init__
    self.plugin_cls: Type[interfaces.Plugin] = entry_point.load()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2465, in load
    return self.resolve()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2471, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
ModuleNotFoundError: No module named 'certbot_dns_dreamhost'
2022-11-18 20:44:38,355:ERROR:certbot._internal.log:An unexpected error occurred:

I think the problem is in the setup.py file:

from setuptools import setup
from setuptools import find_packages

VERSION = "0.2.2"

install_requires = [
    "acme>=0.29.0",
    "certbot>=0.34.0",
    "setuptools",
    "requests",
    "mock",
    "requests-mock",
]

setup(
    name="certbot-dns-dreamhost",
    version=VERSION,
    description="DreamHost DNS Authenticator plugin for Certbot",
    url="https://github.com/goncalo-leal/certbot-dns-dreamhost",
    author="Gonçalo Leal",
    author_email="goncalolealsilva@ua.pt",
    license="Apache License 2.0",
    package="src/certbot_dns_dreamhost",
    include_package_data=True,
    install_requires=install_requires,
    entry_points={
        "certbot.plugins": [
            "dns-dreamhost = certbot_dns_dreamhost.dns_dreamhost:Authenticator"
        ]
    },
)

image

Maybe my question is stupid, but I really can't understand what I'm doing wrong.
Hope someone can help me :slight_smile:

Maybe I'm stupid, but your thread doesn't actually mention the issue? Sure, it mentions "ModuleNotFoundError" in the thread title, but your openings post doesn't elaborate on that at all. You're probably going to get more responses if you'd provide a detailed post with the context around the error with debug info.

4 Likes

Sorry, I will update my question. Thanks for the hint

1 Like

It's not possible to use pip-installed plugins with the Certbot snap. You would need to publish a plugin snap in order to achieve that. Best to probably leave that, though, until you have the plugin working normally.

Based on your errors:

you are not actually using the Certbot snap. /usr/local/bin/certbot is what you get when you try to install Certbot via the system's global pip command. It's probably stealing precedence from /usr/bin/certbot.

Best thing to try is to using your plugin with Certbot via pip (using a virtual environment rather than globally), following these instructions. I think it's worth seeing if you still get the error when you use this method. Once you get it working this way, you can look at other distribution methods.

I have a DNS plugin here which supports DreamHost, you could give that a go as well.

8 Likes

Your pip files (source as wel as wheel) do not contain actual Python code.. Well, except for the source package, it contains setup.py but nothing else. See:

(venv) osiris@desktop pip $ unzip certbot_dns_dreamhost-0.2.2-py2.py3-none-any.whl 
Archive:  certbot_dns_dreamhost-0.2.2-py2.py3-none-any.whl
  inflating: certbot_dns_dreamhost-0.2.2.dist-info/LICENSE.md  
  inflating: certbot_dns_dreamhost-0.2.2.dist-info/METADATA  
  inflating: certbot_dns_dreamhost-0.2.2.dist-info/WHEEL  
  inflating: certbot_dns_dreamhost-0.2.2.dist-info/entry_points.txt  
  inflating: certbot_dns_dreamhost-0.2.2.dist-info/top_level.txt  
  inflating: certbot_dns_dreamhost-0.2.2.dist-info/RECORD  
(venv) osiris@desktop pip $ tar xvf certbot-dns-dreamhost-0.2.2.tar.gz 
certbot-dns-dreamhost-0.2.2/
certbot-dns-dreamhost-0.2.2/LICENSE.md
certbot-dns-dreamhost-0.2.2/PKG-INFO
certbot-dns-dreamhost-0.2.2/README.md
certbot-dns-dreamhost-0.2.2/certbot_dns_dreamhost.egg-info/
certbot-dns-dreamhost-0.2.2/certbot_dns_dreamhost.egg-info/PKG-INFO
certbot-dns-dreamhost-0.2.2/certbot_dns_dreamhost.egg-info/SOURCES.txt
certbot-dns-dreamhost-0.2.2/certbot_dns_dreamhost.egg-info/dependency_links.txt
certbot-dns-dreamhost-0.2.2/certbot_dns_dreamhost.egg-info/entry_points.txt
certbot-dns-dreamhost-0.2.2/certbot_dns_dreamhost.egg-info/requires.txt
certbot-dns-dreamhost-0.2.2/certbot_dns_dreamhost.egg-info/top_level.txt
certbot-dns-dreamhost-0.2.2/setup.cfg
certbot-dns-dreamhost-0.2.2/setup.py
(venv) osiris@desktop pip $ 

I suspect your release process isn't functioning properly.

If I manually copy over certbot_dns_dreamhost into venv/lib/python3.9/site-packages/ it can find the plugin. However, it then complains about a missing zope module due to import zope.interface in dns_dreamhost.py. Note that Certbot is moving away from zope and thus it's probably better to not use it yourself. See Remove zope from Certbot by adferrand · Pull Request #9161 · certbot/certbot · GitHub for examples. (Edit:) this probably still works when using 1.32.0, I tested on the master branch which is 2.0.0 and removes the zope dependency.

Also, why is it pulling in the mock dep? It's not even used in your code :slight_smile:

4 Likes

Thank you and @_az too. I have followed your tips and I think I have successfully build my plugin.

After running python -m pip install certbot-dns-dreamhost==0.3.0 (on a python virtual env) if I cd ./venv/lib/python3.10/site-packages/ && ls I see (along with other directories):

certbot
certbot-1.32.0.dist-info

If I ls certbot_dns_dreamhost I get

dns_dreamhost.py  __init__.py  __pycache__

I f I run certbot plugins I get:```


  • dns-dreamhost
    Description: Obtain certificates using a DNS .txt record
    Interfaces: Authenticator, Plugin
    Entry point: dns-dreamhost = certbot_dns_dreamhost.dns_dreamhost:Authenticator

  • standalone
    Description: Spin up a temporary webserver
    Interfaces: Authenticator, Plugin
    Entry point: standalone = certbot._internal.plugins.standalone:Authenticator

  • webroot
    Description: Place files in webroot directory
    Interfaces: Authenticator, Plugin
    Entry point: webroot = certbot._internal.plugins.webroot:Authenticator



However, if I run `sudo certbot certonly --authenticator certbot-dns-dreamhost:dns-dreamhost --certbot-dns-dreamhost:dns-dreamhost-credentials ./credentials.ini -d '*.<domain>.com'` I get:

usage:
certbot [SUBCOMMAND] [options] [-d DOMAIN] [-d DOMAIN] ...

Certbot can obtain and install HTTPS/TLS/SSL certificates. By default,
it will attempt to use a webserver both for obtaining and installing the
certificate.
certbot: error: unrecognized arguments: --certbot-dns-dreamhost:dns-dreamhost-credentials ./credentials.ini


Any idea why this is happening?

You probably need to use just dns-dreamhost as plugin name on the command line.

4 Likes

How? I tried but still get the same error

(venv) $ sudo certbot certonly --authenticator certbot-dns-dreamhost:dns-dreamhost --dns-dreamhost:dns-dreamhost-credentials ./credentials.ini -d '*.domain.com'

usage: 
  certbot [SUBCOMMAND] [options] [-d DOMAIN] [-d DOMAIN] ...

Certbot can obtain and install HTTPS/TLS/SSL certificates.  By default,
it will attempt to use a webserver both for obtaining and installing the
certificate. 
certbot: error: unrecognized arguments: --dns-dreamhost:dns-dreamhost-credentials ./credentials.ini

(venv) $ sudo certbot certonly --authenticator dns-dreamhost:dns-dreamhost --dns-dreamhost:dns-dreamhost-credentials ./credentials.ini -d '*.domain.com'

usage: 
  certbot [SUBCOMMAND] [options] [-d DOMAIN] [-d DOMAIN] ...

Certbot can obtain and install HTTPS/TLS/SSL certificates.  By default,
it will attempt to use a webserver both for obtaining and installing the
certificate. 
certbot: error: unrecognized arguments: --dns-dreamhost:dns-dreamhost-credentials ./credentials.ini

Just like this:

(venv) $ certbot certonly --authenticator dns-dreamhost --dns-dreamhost-credentials ./credentials.ini -d example.com --config-dir . --work-dir . --logs-dir .
Saving debug log to /tmp/dreamhost/letsencrypt.log
An unexpected error occurred:
TypeError: Can't instantiate abstract class Authenticator with abstract method _cleanup
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /tmp/dreamhost/letsencrypt.log or re-run Certbot with -v for more details.
(venv) $ 

The whole name:othername isn't necessary any longer for some time now.

With regard to the new error: you've got _perform() defined twice. I'm thinking one of them needs to be _cleanup().

4 Likes

It is running now. Thanks a lot for your time!

5 Likes

You are welcome to get your plugin listed on User Guide — Certbot 1.32.0 documentation by opening a PR against certbot/using.rst at master · certbot/certbot · GitHub.

5 Likes

IMO it might be better to wait for production level code :slight_smile:

It looks like even a simple python -m pylint dns_dreamhost.py hasn't been done which would have revealed at least one easily fixed bug already.

4 Likes

Yes, the code isn't yet production ready. However, this is my first time working with certbot and building a plugin of any type. I'll work on my code and in a good read me file. Any other tip in order to build a good plugin?

1 Like

That's al fine of course :slight_smile: Just remember _az's post when you think your plugin is ready for the public at large :slight_smile:

While I don't have any tips for building plugins, it's often a good idea to have pylint or similar linting tools pass. That would let you detect the erroneous copy/paste in the middle of something else at https://github.com/goncalo-leal/certbot-dns-dreamhost/blob/5a15c1dad83657c6eba4ecb3834dc8cbdba03ca7/src/certbot_dns_dreamhost/dns_dreamhost.py#L105

You might also want to look at typing (and mypy) although that's probably more a "nice to have".

5 Likes

Hi, I tried to follow your tips and I think my plugin is now ok. In the near future I will try to make available through snap and rpm because those options are better than pipy since I'm working mainly with servers with CentOS and some debian distros.
Thank you and @_az for your time and help!

2 Likes

You might want to consider checking your typing with mypy. E.g., from within a virtual environment:

pip install mypy
python -m mypy dns_dreamhost.py
3 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.