SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed

I'm trying to renew my certificates and facing the following error

My domain is: hiddendomain.com

I ran this command:
certbot --nginx certonly --cert-name hiddendomain.com -d hiddendomain.com,www.hiddendomain.com

It produced this output:

[root@ip-172-30-0-86 ~]# certbot certonly --cert-name hiddendomain.com -d hiddendomain.com,www.hiddendomain.com
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Found credentials in shared credentials file: ~/.aws/credentials

How would you like to authenticate with the ACME CA?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: Obtain certificates using a DNS TXT record (if you are using AWS Route53 for
DNS). (dns-route53)
2: Nginx Web Server plugin (nginx)
3: Spin up a temporary webserver (standalone)
4: Place files in webroot directory (webroot)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate number [1-4] then [enter] (press 'c' to cancel): 2
Plugins selected: Authenticator nginx, Installer None

Please choose an account
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: ip-172-30-0-86@2018-05-30T14:41:31Z (f816)
2: ip-172-30-0-86@2018-06-05T10:08:39Z (11dd)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate number [1-2] then [enter] (press 'c' to cancel): 2
Starting new HTTPS connection (1): acme-v02.api.letsencrypt.org
An unexpected error occurred:
SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:765)
Please see the logfiles in /var/log/letsencrypt for more details.

My web server is: Nginx

The operating system my web server runs on is: Centos 7

My hosting provider, if applicable, is: AWS

I can login to a root shell on my machine: Yes, already running as root

The version of my client is 1.11.0

Log File:

2021-10-11 04:48:05,890:DEBUG:certbot._internal.main:certbot version: 1.11.0
2021-10-11 04:48:05,890:DEBUG:certbot._internal.main:Location of certbot entry point: /usr/bin/certbot
2021-10-11 04:48:05,890:DEBUG:certbot._internal.main:Arguments: ['--nginx', '--cert-name', 'hiddendomain.com', '-d', 'hiddendomain.com,www.hiddendomain.com']
2021-10-11 04:48:05,890:DEBUG:certbot._internal.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#certbot-route53:auth,PluginEntryPoint#dns-route53,PluginEntryPoint#manual,PluginEntryPoint#nginx,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
2021-10-11 04:48:05,906:DEBUG:certbot._internal.log:Root logging level set at 20
2021-10-11 04:48:05,906:INFO:certbot._internal.log:Saving debug log to /var/log/letsencrypt/letsencrypt.log
2021-10-11 04:48:05,907:DEBUG:certbot._internal.plugins.selection:Requested authenticator nginx and installer nginx
2021-10-11 04:48:08,179:DEBUG:certbot._internal.plugins.selection:Single candidate plugin: * nginx
Description: Nginx Web Server plugin
Interfaces: IAuthenticator, IInstaller, IPlugin
Entry point: nginx = certbot_nginx._internal.configurator:NginxConfigurator
Initialized: <certbot_nginx._internal.configurator.NginxConfigurator object at 0x27bda10>
Prep: True
2021-10-11 04:48:08,180:DEBUG:certbot._internal.plugins.selection:Single candidate plugin: * nginx
Description: Nginx Web Server plugin
Interfaces: IAuthenticator, IInstaller, IPlugin
Entry point: nginx = certbot_nginx._internal.configurator:NginxConfigurator
Initialized: <certbot_nginx._internal.configurator.NginxConfigurator object at 0x27bda10>
Prep: True
2021-10-11 04:48:08,180:DEBUG:certbot._internal.plugins.selection:Selected authenticator <certbot_nginx._internal.configurator.NginxConfigurator object at 0x27bda10> and installer <certbot_nginx._internal.configurator.NginxConfigurator object at 0x27bda10>
2021-10-11 04:48:08,180:INFO:certbot._internal.plugins.selection:Plugins selected: Authenticator nginx, Installer nginx
2021-10-11 04:48:09,184:DEBUG:certbot._internal.main:Picked account: <Account(RegistrationResource(body=Registration(status=u'valid', terms_of_service_agreed=None, agreement=None, only_return_existing=None, contact=(u'mailto:jon@example.com',), key=JWKRSA(key=<ComparableRSAKey(<cryptography.hazmat.backends.openssl.rsa._RSAPublicKey object at 0x15e5510>)>), external_account_binding=None), uri=u'https://acme-v02.api.letsencrypt.org/acme/acct/35816188', new_authzr_uri=None, terms_of_service=u'https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf'), f81617d0360c7cfb84c917a3147397e2, Meta(creation_host=u'ip-172-30-0-86', register_to_eff=None, creation_dt=datetime.datetime(2018, 5, 30, 14, 41, 31, tzinfo=<UTC>)))>
2021-10-11 04:48:09,186:DEBUG:acme.client:Sending GET request to https://acme-v02.api.letsencrypt.org/directory.
2021-10-11 04:48:09,193:INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): acme-v02.api.letsencrypt.org
2021-10-11 04:48:09,509:DEBUG:certbot._internal.log:Exiting abnormally:
Traceback (most recent call last):
  File "/usr/bin/certbot", line 9, in <module>
    load_entry_point('certbot==1.11.0', 'console_scripts', 'certbot')()
  File "/usr/lib/python2.7/site-packages/certbot/main.py", line 15, in main
    return internal_main.main(cli_args)
  File "/usr/lib/python2.7/site-packages/certbot/_internal/main.py", line 1421, in main
    return config.func(config, plugins)
  File "/usr/lib/python2.7/site-packages/certbot/_internal/main.py", line 1277, in certonly
    le_client = _init_le_client(config, auth, installer)
  File "/usr/lib/python2.7/site-packages/certbot/_internal/main.py", line 659, in _init_le_client
    return client.Client(config, acc, authenticator, installer, acme=acme)
  File "/usr/lib/python2.7/site-packages/certbot/_internal/client.py", line 255, in __init__
    acme = acme_from_config_key(config, self.account.key, self.account.regr)
  File "/usr/lib/python2.7/site-packages/certbot/_internal/client.py", line 43, in acme_from_config_key
    return acme_client.BackwardsCompatibleClientV2(net, key, config.server)
  File "/usr/lib/python2.7/site-packages/acme/client.py", line 831, in __init__
    directory = messages.Directory.from_json(net.get(server).json())
  File "/usr/lib/python2.7/site-packages/acme/client.py", line 1168, in get
    self._send_request('GET', url, **kwargs), content_type=content_type)
  File "/usr/lib/python2.7/site-packages/acme/client.py", line 1118, in _send_request
    response = self.session.request(method, url, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 464, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 576, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 431, in send
    raise SSLError(e, request=request)
SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:765)
2021-10-11 04:48:09,511:ERROR:certbot._internal.log:An unexpected error occurred:
2021-10-11 04:48:09,511:ERROR:certbot._internal.log:SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:765)

I would try uninstalling certbot 1.11.0 and installing a newer version.
Then show us the version:
certbot --version

Then we will go from there.

Output of certbot --version

[root@ip-172-30-0-86 ~]# certbot --version
certbot 1.11.0

Tried running

certbot 1.11.0
[root@ip-172-30-0-86 ~]# yum install certbot python2-certbot-nginx
Loaded plugins: fastestmirror
https://us-east.repo.webtatic.com/yum/el7/x86_64/repodata/repomd.xml: [Errno 14] curl#60 - "The certificate issuer's certificate has expired.  Check your system date and time."
Trying other mirror.
It was impossible to connect to the CentOS servers.
This could mean a connectivity issue in your environment, such as the requirement to configure a proxy,
or a transparent proxy that tampers with TLS security, or an incorrect system clock.
You can try to solve this issue by using the instructions on https://wiki.centos.org/yum-errors
If above article doesn't help to resolve this issue please use https://bugs.centos.org/.

https://uk.repo.webtatic.com/yum/el7/x86_64/repodata/repomd.xml: [Errno 14] curl#60 - "The certificate issuer's certificate has expired.  Check your system date and time."
Trying other mirror.
Loading mirror speeds from cached hostfile
 * base: download.cf.centos.org
 * epel: d2lzkl7pfhq30w.cloudfront.net
 * extras: download.cf.centos.org
 * remi-php72: mirror.netweaver.uk
 * remi-safe: mirror.netweaver.uk
 * updates: download.cf.centos.org
 * webtatic: uk.repo.webtatic.com
Package certbot-1.11.0-1.el7.noarch already installed and latest version
Package python2-certbot-nginx-1.11.0-1.el7.noarch already installed and latest version
Nothing to do

That site uses LE via default trust path:

echo | openssl s_client -connect us-east.repo.webtatic.com:443 -servername us-east.repo.webtatic.com | head
depth=2 C = US, O = Internet Security Research Group, CN = ISRG Root X1
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = R3
verify return:1
depth=0 CN = webtatic.com
verify return:1
CONNECTED(00000005)
---
Certificate chain
DONE
 0 s:CN = webtatic.com
   i:C = US, O = Let's Encrypt, CN = R3
 1 s:C = US, O = Let's Encrypt, CN = R3
   i:C = US, O = Internet Security Research Group, CN = ISRG Root X1
 2 s:C = US, O = Internet Security Research Group, CN = ISRG Root X1
   i:O = Digital Signature Trust Co., CN = DST Root CA X3
---

This likely means that your server doesn't have the "ISRG Root X1" in the trusted root store.

Compare outputs:
curl -Iki https://us-east.repo.webtatic.com/yum/el7/x86_64/repodata/repomd.xml
curl -Ii https://us-east.repo.webtatic.com/yum/el7/x86_64/repodata/repomd.xml

[root@ip-172-30-0-86 ~]# curl -Iki https://us-east.repo.webtatic.com/yum/el7/x86_64/repodata/repomd.xml
HTTP/1.1 200 OK
Server: nginx
Date: Mon, 11 Oct 2021 05:43:26 GMT
Content-Type: text/xml
Content-Length: 3691
Last-Modified: Sat, 03 Oct 2020 14:20:55 GMT
Connection: keep-alive
ETag: "5f7888c7-e6b"
X-Frame-Options: DENY
Accept-Ranges: bytes

[root@ip-172-30-0-86 ~]# curl -Ii https://us-east.repo.webtatic.com/yum/el7/x86_64/repodata/repomd.xml
curl: (60) The certificate issuer's certificate has expired.  Check your system date and time.
More details here: http://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a "bundle"
 of Certificate Authority (CA) public keys (CA certs). If the default
 bundle file isn't adequate, you can specify an alternate file
 using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
 the bundle, the certificate verification probably failed due to a
 problem with the certificate (it might be expired, or the name might
 not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
 the -k (or --insecure) option.
[root@ip-172-30-0-86 ~]#

That confirms my suspicions...
curl can get the content - but it doesn't trust the cert path being served.

Also please show output of:
openssl version

2 Likes
[root@ip-172-30-0-86 ~]# openssl version
OpenSSL 1.0.2k-fips  26 Jan 2017

Please see:
https://www.openssl.org/blog/blog/2021/09/13/LetsEncryptRootCertExpire/

2 Likes

I ran the following command. This did it for me.

yum install ca-certificates openssl

Thanks a lot!

6 Likes

To buy us some time before running and testing this update via yum (we have OpenSSL version 1.0.2k-fips on CentOS 7), we used the --no-verify-ssl flag on the certbot renew command.

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.