SSLCertVerificationError of acme-v02.api.letsencrypt.org causes certbot renewal failure

[rg305]

@mdbooth

the python requests module does not use the default system certificate root store.

Can the one in use be updated?

[mdbooth]

Yes. It's deployed in the certifi module, which I have updated.

[rg305]

Then I suspect there are more than one copy of certifi in your system.
[perhaps in different environments]

[mdbooth]

I have now installed certbot from snap:

snap run --shell certbot
$ python -c "import requests; print(requests.get(url = 'https://valid-isrgrootx1.letsencrypt.org').text)"
...snip...
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get issuer certificate (_ssl.c:1129)

So it doesn't even work from the snap

[rg305]

You could try another ACME client ...

[mdbooth]

This isn't using an ACME client: it's a (hopefully!) local python ssl configuration issue affecting the letsencrypt servers specifically.

[rg305]

@mdbooth

I ran this command:
certbot -v --work-dir $(pwd)/work --logs-dir $(pwd)/logs --config-dir $(pwd)/config renew --dns-godaddy-credentials credentials.ini

That ["certbot"] is an ACME client.
Try another one.

[mdbooth]

I meant: the issue reproduces with the following python one-liner:

python -c "import requests; print(requests.get(url = 'https://valid-isrgrootx1.letsencrypt.org').text)"

It's not an ACME client issue. It's a python issue.

[mdbooth]

Thanks for this! I will switch to using this when I have fixed the python ssl issue as it looks better maintained.

[rg305]

I never said it was a problem with the ACME client itself.

Listen closely:

• The ACME client uses python.
• Another ACME client might NOT use python.

[Osiris]

Maybe something is intercepting your connection to the ACME server? I haven't found a way to somehow view the certificate send by the server. One would hope the SSLError exception would somehow incorporate that?

[mdbooth]

I think I've ruled that out because curl works fine. I'm currently trying to get python requests to use the system ca store.

If nobody else is hitting this it's going to be something bizarre.

[rg305]

Configuring python to use a specific ca store isn't really anything to do with this community.
Sure, it is using python to connect to LE.
But that can be said for anything on the net:

• my python can't connect to LE
• my browser can't connect to LE
• curl can't connect to LE
• I wrote a Java app and it can't connect to LE
• etc.

Each of which could likely find much better help elsewhere.

[mdbooth]

If we can find a solution to the problem, though, and it turns out not to be 100% local, e.g. a bad update to package X on distro Y, documenting the solution here is likely to help others. I'm also hopeful that in this case I might find others who have already found the solution, as whatever the problem is it affects certbot.

[rg305]

True.
Also true:

@rg305

could likely find much better help elsewhere.

Not sure this is the place ALL such affected persons will coming looking for help.
[likely only the ones using certbot]

[Osiris]

@mdbooth

I think I've ruled that out because curl works fine.

True

[Osiris]

@mdbooth Could you perhaps in a Python shell, with the non-functional requests, check certifi? Probably not the case for you, but when I checked on my Gentoo system, certifi was using my system root store (due to the fact Gentoo uses a certifi hack):

osiris@erazer ~ $ python
Python 3.11.4 (main, Jun 30 2023, 20:11:33) [GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import certifi
>>> certifi.where()
'/etc/ssl/certs/ca-certificates.crt'
>>>

Maybe good to just check.