Unexpected error: module 'idna' has no attribute 'core'. Skipping

Came to renew my certificate and have encountered a new error that I’m unsure how to resolve.

I found a similar thread suggesting it might be down to an outdated certbot plugin (which in this case would be for nginx) so have included the version for that too in the information below, it seems the most up-to-date are installed (and I force reinstallation just in case). Explicity included the plugin to be used too.

My domain is: neils-snaps.co.uk

I ran this command:

certbot --nginx renew

It produced this output:

Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
Processing /etc/letsencrypt/renewal/neils-snaps.co.uk.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
Cert is due for renewal, auto-renewing...
Plugins selected: Authenticator nginx, Installer nginx
Attempting to renew cert (neils-snaps.co.uk) from /etc/letsencrypt/renewal/neils-snaps.co.uk.conf produced an unexpected error: module 'idna' has no attribute 'core'. Skipping.

All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/neils-snaps.co.uk/fullchain.pem (failure)

My web server is (include version):

nginx -V
nginx version: nginx/1.16.1
built with OpenSSL 1.1.1d  10 Sep 2019 (running with OpenSSL 1.1.1f  31 Mar 2020)
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --conf-path=/etc/nginx/nginx.conf --sbin-path=/usr/bin/nginx --pid-path=/run/nginx.pid --lock-path=/run/lock/nginx.loc
k --user=http --group=http --http-log-path=/var/log/nginx/access.log --error-log-path=stderr --http-client-body-temp-path=/var/lib/nginx/client-body --http-pro
xy-temp-path=/var/lib/nginx/proxy --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/ngi
nx/uwsgi --with-cc-opt='-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -D_FORTIFY_SOURCE=2' --with-ld-opt=-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now --
with-compat --with-debug --with-file-aio --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_degradation_module --wi
th-http_flv_module --with-http_geoip_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_realip_module --with-ht
tp_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail -
-with-mail_ssl_module --with-pcre-jit --with-stream --with-stream_geoip_module --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-threads

The operating system my web server runs on is (include version):

Arch Linux

My hosting provider, if applicable, is: VPS hosted by OVH

I can login to a root shell on my machine (yes or no, or I don’t know): Yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel): No

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you’re using Certbot):

certbot --version
certbot 1.3.0

Looking at the debug log-file the error appears to be coming from urllib3 so I reinstalled it and (as well as requests and pyopenssl just to be sure), but the problem persists…

Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/urllib3/contrib/pyopenssl.py", line 192, in idna_encode
    return idna.encode(name)
AttributeError: module 'idna' has no attribute 'encode'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/certbot/_internal/renewal.py", line 448, in handle_renewal_request
    main.renew_cert(lineage_config, plugins, renewal_candidate)
  File "/usr/lib/python3.8/site-packages/certbot/_internal/main.py", line 1174, in renew_cert
    le_client = _init_le_client(config, auth, installer)
  File "/usr/lib/python3.8/site-packages/certbot/_internal/main.py", line 610, in _init_le_client
    return client.Client(config, acc, authenticator, installer, acme=acme)
  File "/usr/lib/python3.8/site-packages/certbot/_internal/client.py", line 257, in __init__
    acme = acme_from_config_key(config, self.account.key, self.account.regr)
  File "/usr/lib/python3.8/site-packages/certbot/_internal/client.py", line 44, in acme_from_config_key
    return acme_client.BackwardsCompatibleClientV2(net, key, config.server)
  File "/usr/lib/python3.8/site-packages/acme/client.py", line 808, in __init__
    directory = messages.Directory.from_json(net.get(server).json())
  File "/usr/lib/python3.8/site-packages/acme/client.py", line 1141, in get
    self._send_request('GET', url, **kwargs), content_type=content_type)
  File "/usr/lib/python3.8/site-packages/acme/client.py", line 1091, in _send_request
    response = self.session.request(method, url, *args, **kwargs)
  File "/usr/lib/python3.8/site-packages/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python3.8/site-packages/requests/sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python3.8/site-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(  File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 670, in urlopen
    httplib_response = self._make_request(
  File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 381, in _make_request
    self._validate_conn(conn)
  File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 976, in _validate_conn
    conn.connect()
  File "/usr/lib/python3.8/site-packages/urllib3/connection.py", line 385, in connect
    cert = self.sock.getpeercert()
  File "/usr/lib/python3.8/site-packages/urllib3/contrib/pyopenssl.py", line 381, in getpeercert
    "subjectAltName": get_subj_alt_name(x509),
  File "/usr/lib/python3.8/site-packages/urllib3/contrib/pyopenssl.py", line 250, in get_subj_alt_name
    names = [
  File "/usr/lib/python3.8/site-packages/urllib3/contrib/pyopenssl.py", line 250, in <listcomp>
    names = [
  File "/usr/lib/python3.8/site-packages/urllib3/contrib/pyopenssl.py", line 200, in _dnsname_to_stdlib
    name = idna_encode(name)
  File "/usr/lib/python3.8/site-packages/urllib3/contrib/pyopenssl.py", line 193, in idna_encode
    except idna.core.IDNAError:

Any suggestions are welcome.

1 Like

I don’t know how to fix that problem.
[looks like python upgrade to 3.8 broke something…]

But as an alternative you could also try using:
certbot-auto
acme.sh
or other ACME clients.

1 Like

Thanks for the alternative options, will give them a whirl.

1 Like

Can you try reinstalling the python3-idna package?

Edit:

What Python installation is being used? Are you logged in as root or running Certbot with sudo? “head -1 $(which certbot)” or “head -1 $(sudo which certbot)” might show it.

If e.g. it’s /usr/bin/python3.8 and you’re using sudo, what does “sudo /usr/bin/python3.8 -c 'import idna; print(idna)'” show?

1 Like

Hi,

I’m running certbot as root rather than with sudo.

head -1 $(which certbot)
#!/usr/bin/python
/usr/bin/python --version
Python 3.8.2
ls -l /usr/bin/python
lrwxrwxrwx 1 root root 7 Apr  8 16:31 /usr/bin/python -> python3
/usr/bin/python -c 'import idna; print(idna)'
<module 'idna' from '/usr/lib/python3.8/site-packages/idna/__init__.py'> 

There is no python3-idna package available under ArchLinux but I tried reinstalling, and whilst pacman recognised it was already installed and then proceeded to re-install the reinstalation failed…

pacman -Syu python-idna
:: Synchronising package databases...
 core is up to date
 extra is up to date
 community is up to date
warning: python-idna-2.9-1 is up to date -- reinstalling
:: Starting full system upgrade...
resolving dependencies...
looking for conflicting packages...
                                                                                                                                                                
Packages (4) iana-etc-20200401-1  mlocate-0.26.git.20170220-3  s-nail-14.9.18-1  python-idna-2.9-1
                                                                                                                                                                
Total Installed Size:  5.86 MiB
Net Upgrade Size:      0.67 MiB
                                                                                                                                                                
:: Proceed with installation? [Y/n] y
(4/4) checking keys in keyring                                                                   [########################################################] 100%
(4/4) checking package integrity                                                                 [########################################################] 100%
(4/4) loading package files                                                                      [########################################################] 100%
(4/4) checking for file conflicts                                                                [########################################################] 100%
error: failed to commit transaction (conflicting files)                                                             
python-idna: /usr/lib/python3.8/site-packages/idna-2.9-py3.8.egg-info/PKG-INFO exists in filesystem                                                             
python-idna: /usr/lib/python3.8/site-packages/idna-2.9-py3.8.egg-info/SOURCES.txt exists in filesystem                                                          
python-idna: /usr/lib/python3.8/site-packages/idna-2.9-py3.8.egg-info/dependency_links.txt exists in filesystem
python-idna: /usr/lib/python3.8/site-packages/idna-2.9-py3.8.egg-info/top_level.txt exists in filesystem                                                        
python-idna: /usr/lib/python3.8/site-packages/idna/__init__.py exists in filesystem
python-idna: /usr/lib/python3.8/site-packages/idna/__pycache__/__init__.cpython-38.opt-1.pyc exists in filesystem
python-idna: /usr/lib/python3.8/site-packages/idna/__pycache__/__init__.cpython-38.pyc exists in filesystem
python-idna: /usr/lib/python3.8/site-packages/idna/__pycache__/codec.cpython-38.opt-1.pyc exists in filesystem
python-idna: /usr/lib/python3.8/site-packages/idna/__pycache__/codec.cpython-38.pyc exists in filesystem
python-idna: /usr/lib/python3.8/site-packages/idna/__pycache__/compat.cpython-38.opt-1.pyc exists in filesystem
python-idna: /usr/lib/python3.8/site-packages/idna/__pycache__/compat.cpython-38.pyc exists in filesystem
python-idna: /usr/lib/python3.8/site-packages/idna/__pycache__/core.cpython-38.opt-1.pyc exists in filesystem
python-idna: /usr/lib/python3.8/site-packages/idna/__pycache__/core.cpython-38.pyc exists in filesystem
python-idna: /usr/lib/python3.8/site-packages/idna/__pycache__/idnadata.cpython-38.opt-1.pyc exists in filesystem
python-idna: /usr/lib/python3.8/site-packages/idna/__pycache__/idnadata.cpython-38.pyc exists in filesystem
python-idna: /usr/lib/python3.8/site-packages/idna/__pycache__/intranges.cpython-38.opt-1.pyc exists in filesystem
python-idna: /usr/lib/python3.8/site-packages/idna/__pycache__/uts46data.cpython-38.pyc exists in filesystem
python-idna: /usr/lib/python3.8/site-packages/idna/codec.py exists in filesystem                            
python-idna: /usr/lib/python3.8/site-packages/idna/compat.py exists in filesystem                           
python-idna: /usr/lib/python3.8/site-packages/idna/core.py exists in filesystem                             
python-idna: /usr/lib/python3.8/site-packages/idna/idnadata.py exists in filesystem                         
python-idna: /usr/lib/python3.8/site-packages/idna/intranges.py exists in filesystem                        
python-idna: /usr/lib/python3.8/site-packages/idna/package_data.py exists in filesystem                     
python-idna: /usr/lib/python3.8/site-packages/idna/uts46data.py exists in filesystem                        
python-idna: /usr/share/licenses/python-idna/LICENSE.rst exists in filesystem                               
Errors occurred, no packages were upgraded.                                                                 

Which whilst mildly annoying (and something I need to work out) doesn’t, if I understand this correctly, explain why the currently installed idna which can be imported by the version python that certbot calls isn’t being imported?

EDIT :

Ok, I resolved this by forcing reinstalation of python-idna to work around the above issue of file conflicts I moved them from their location to /tmp/ (as recommended here and could then reinstall python-idna

mv /usr/lib/python3.8/site-packages/idna* /usr/share/licenses/python-idna /tmp/.
certbot renew                                      
Saving debug log to /var/log/letsencrypt/letsencrypt.log                       
                                                                               
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/neils-snaps.co.uk.conf                     
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cert is due for renewal, auto-renewing...
Plugins selected: Authenticator nginx, Installer nginx
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for neils-snaps.co.uk 
2020/04/21 18:16:48 [notice] 211405#211405: signal process started
Waiting for verification...
Cleaning up challenges
2020/04/21 18:16:51 [notice] 211408#211408: signal process started
2020/04/21 18:16:54 [notice] 211413#211413: signal process started

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
new certificate deployed with reload of nginx server; fullchain is
/etc/letsencrypt/live/neils-snaps.co.uk/fullchain.pem
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Congratulations, all renewals succeeded. The following certs have been renewed:
  /etc/letsencrypt/live/neils-snaps.co.uk/fullchain.pem (success)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Thanks for the suggestion @mnordhoff :+1:

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.