Cert renewal attempts failed

Help
1

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. https://crt.sh/?q=example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is:
https://the1000kmchallenge.com

I ran this command:
certbot renew --apache

It produced this output:
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/www.the1000kmchallenge.com.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cert is due for renewal, auto-renewing…
Plugins selected: Authenticator apache, Installer apache
Starting new HTTPS connection (1): acme-v02.api.letsencrypt.org
Attempting to renew cert (www.the1000kmchallenge.com) from /etc/letsencrypt/renewal/www.the1000kmchallenge.com.conf produced an unexpected error: [Errno 2] No such file or directory. Skipping.
All renewal attempts failed. The following certs could not be renewed:
** /etc/letsencrypt/live/www.the1000kmchallenge.com/fullchain.pem (failure)**

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

All renewal attempts failed. The following certs could not be renewed:
** /etc/letsencrypt/live/www.the1000kmchallenge.com/fullchain.pem (failure)**
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1 renew failure(s), 0 parse failure(s)

My web server is (include version):
rpm -q httpd
httpd-2.4.6-89.el7.centos.x86_64

The operating system my web server runs on is (include version):
CentOS 7

My hosting provider, if applicable, is:
DigitalOcean

I can login to a root shell on my machine (yes or no, or I don’t know):
Yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel):
No

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you’re using Certbot):
certbot 0.31.0

Thanks for all the help!

2

Can you post the output of “sudo ls -alR /etc/letsencrypt” and the traceback from /var/log/letsencrypt/letsencrypt.log?

3

Thanks for your quick response mnordhoff! there you go:

[root@tempnew conf.d]# ls -alR /etc/letsencrypt
/etc/letsencrypt:
total 28
drwxr-xr-x 9 root root 190 Jul 27 20:15 .
drwxr-xr-x. 82 root root 8192 May 7 18:47 …
drwx------ 4 root root 86 Jul 27 19:38 accounts
drwx------ 3 root root 40 May 7 19:10 archive
drwxr-xr-x 2 root root 4096 May 7 19:11 csr
drwx------ 2 root root 4096 May 7 19:11 keys
drwx------ 3 root root 54 May 7 19:10 live
-rw-r–r-- 1 root root 1591 Apr 30 10:27 options-ssl-apache.conf
drwxr-xr-x 2 root root 89 Jul 27 20:03 renewal
drwxr-xr-x 5 root root 43 Apr 30 10:27 renewal-hooks
-rw-r–r-- 1 root root 64 Apr 30 10:27 .updated-options-ssl-apache-conf-digest.txt

/etc/letsencrypt/accounts:
total 0
drwx------ 4 root root 86 Jul 27 19:38 .
drwxr-xr-x 9 root root 190 Jul 27 20:15 …
drwx------ 3 root root 23 Jul 27 19:38 acme-staging-v02.api.letsencrypt.org
drwx------ 3 root root 23 Apr 30 10:27 acme-v02.api.letsencrypt.org

/etc/letsencrypt/accounts/acme-staging-v02.api.letsencrypt.org:
total 0
drwx------ 3 root root 23 Jul 27 19:38 .
drwx------ 4 root root 86 Jul 27 19:38 …
drwx------ 2 root root 6 Jul 27 19:38 directory

/etc/letsencrypt/accounts/acme-staging-v02.api.letsencrypt.org/directory:
total 0
drwx------ 2 root root 6 Jul 27 19:38 .
drwx------ 3 root root 23 Jul 27 19:38 …

/etc/letsencrypt/accounts/acme-v02.api.letsencrypt.org:
total 0
drwx------ 3 root root 23 Apr 30 10:27 .
drwx------ 4 root root 86 Jul 27 19:38 …
drwx------ 3 root root 46 May 1 17:59 directory

/etc/letsencrypt/accounts/acme-v02.api.letsencrypt.org/directory:
total 0
drwx------ 3 root root 46 May 1 17:59 .
drwx------ 3 root root 23 Apr 30 10:27 …
drwx------ 2 root root 64 May 1 17:59 6ff7661297857e0d13da59ea18217eac

/etc/letsencrypt/accounts/acme-v02.api.letsencrypt.org/directory/6ff7661297857e0d13da59ea18217eac:
total 12
drwx------ 2 root root 64 May 1 17:59 .
drwx------ 3 root root 46 May 1 17:59 …
-rw-r–r-- 1 root root 78 May 1 17:59 meta.json
-r-------- 1 root root 1632 May 1 17:59 private_key.json
-rw-r–r-- 1 root root 78 May 1 17:59 regr.json

/etc/letsencrypt/archive:
total 0
drwx------ 3 root root 40 May 7 19:10 .
drwxr-xr-x 9 root root 190 Jul 27 20:15 …
drwxr-xr-x 2 root root 160 May 7 19:47 www.the1000kmchallenge.com

/etc/letsencrypt/archive/www.the1000kmchallenge.com:
total 32
drwxr-xr-x 2 root root 160 May 7 19:47 .
drwx------ 3 root root 40 May 7 19:10 …
-rw-r–r-- 1 root root 1939 May 7 19:10 cert1.pem
-rw-r–r-- 1 root root 1939 May 7 19:11 cert2.pem
-rw-r–r-- 1 root root 1647 May 7 19:10 chain1.pem
-rw-r–r-- 1 root root 1647 May 7 19:11 chain2.pem
-rw-r–r-- 1 root root 3586 May 7 19:10 fullchain1.pem
-rw-r–r-- 1 root root 3586 May 7 19:11 fullchain2.pem
-rw------- 1 root root 1708 May 7 19:10 privkey1.pem
-rw------- 1 root root 1704 May 7 19:11 privkey2.pem

/etc/letsencrypt/csr:
total 72
drwxr-xr-x 2 root root 4096 May 7 19:11 .
drwxr-xr-x 9 root root 190 Jul 27 20:15 …
-rw-r–r-- 1 root root 952 May 1 18:00 0000_csr-certbot.pem
-rw-r–r-- 1 root root 952 May 1 18:07 0001_csr-certbot.pem
-rw-r–r-- 1 root root 952 May 1 18:08 0002_csr-certbot.pem
-rw-r–r-- 1 root root 952 May 1 18:11 0003_csr-certbot.pem
-rw-r–r-- 1 root root 952 May 1 18:16 0004_csr-certbot.pem
-rw-r–r-- 1 root root 932 May 1 18:18 0005_csr-certbot.pem
-rw-r–r-- 1 root root 932 May 1 18:55 0006_csr-certbot.pem
-rw-r–r-- 1 root root 932 May 1 19:13 0007_csr-certbot.pem
-rw-r–r-- 1 root root 932 May 2 04:25 0008_csr-certbot.pem
-rw-r–r-- 1 root root 932 May 2 04:49 0009_csr-certbot.pem
-rw-r–r-- 1 root root 932 May 2 05:24 0010_csr-certbot.pem
-rw-r–r-- 1 root root 932 May 2 05:28 0011_csr-certbot.pem
-rw-r–r-- 1 root root 932 May 2 05:31 0012_csr-certbot.pem
-rw-r–r-- 1 root root 924 May 2 06:09 0013_csr-certbot.pem
-rw-r–r-- 1 root root 940 May 7 19:06 0014_csr-certbot.pem
-rw-r–r-- 1 root root 940 May 7 19:10 0015_csr-certbot.pem
-rw-r–r-- 1 root root 940 May 7 19:11 0016_csr-certbot.pem

/etc/letsencrypt/keys:
total 72
drwx------ 2 root root 4096 May 7 19:11 .
drwxr-xr-x 9 root root 190 Jul 27 20:15 …
-rw------- 1 root root 1704 May 1 18:00 0000_key-certbot.pem
-rw------- 1 root root 1704 May 1 18:07 0001_key-certbot.pem
-rw------- 1 root root 1704 May 1 18:08 0002_key-certbot.pem
-rw------- 1 root root 1704 May 1 18:11 0003_key-certbot.pem
-rw------- 1 root root 1704 May 1 18:16 0004_key-certbot.pem
-rw------- 1 root root 1708 May 1 18:18 0005_key-certbot.pem
-rw------- 1 root root 1708 May 1 18:55 0006_key-certbot.pem
-rw------- 1 root root 1704 May 1 19:13 0007_key-certbot.pem
-rw------- 1 root root 1704 May 2 04:25 0008_key-certbot.pem
-rw------- 1 root root 1704 May 2 04:49 0009_key-certbot.pem
-rw------- 1 root root 1704 May 2 05:24 0010_key-certbot.pem
-rw------- 1 root root 1704 May 2 05:28 0011_key-certbot.pem
-rw------- 1 root root 1704 May 2 05:31 0012_key-certbot.pem
-rw------- 1 root root 1704 May 2 06:09 0013_key-certbot.pem
-rw------- 1 root root 1704 May 7 19:06 0014_key-certbot.pem
-rw------- 1 root root 1708 May 7 19:10 0015_key-certbot.pem
-rw------- 1 root root 1704 May 7 19:11 0016_key-certbot.pem

/etc/letsencrypt/live:
total 4
drwx------ 3 root root 54 May 7 19:10 .
drwxr-xr-x 9 root root 190 Jul 27 20:15 …
-rw-r–r-- 1 root root 740 May 2 05:31 README
drwxr-xr-x 2 root root 93 Jul 27 18:44 www.the1000kmchallenge.com

/etc/letsencrypt/live/www.the1000kmchallenge.com:
total 4
drwxr-xr-x 2 root root 93 Jul 27 18:44 .
drwx------ 3 root root 54 May 7 19:10 …
lrwxrwxrwx 1 root root 50 May 7 19:11 cert.pem -> …/…/archive/www.the1000kmchallenge.com/cert2.pem
lrwxrwxrwx 1 root root 51 May 7 19:11 chain.pem -> …/…/archive/www.the1000kmchallenge.com/chain2.pem
lrwxrwxrwx 1 root root 55 May 7 19:11 fullchain.pem -> …/…/archive/www.the1000kmchallenge.com/fullchain2.pem
lrwxrwxrwx 1 root root 53 May 7 19:11 privkey.pem -> …/…/archive/www.the1000kmchallenge.com/privkey2.pem
-rw-r–r-- 1 root root 692 May 7 19:10 README

/etc/letsencrypt/renewal:
total 16
drwxr-xr-x 2 root root 89 Jul 27 20:03 .
drwxr-xr-x 9 root root 190 Jul 27 20:15 …
-rw-r–r-- 1 root root 589 May 7 19:11 www.the1000kmchallenge.com.conf
-rw-r–r-- 1 root root 12288 Jul 27 20:03 .www.the1000kmchallenge.com.conf.swp

/etc/letsencrypt/renewal-hooks:
total 0
drwxr-xr-x 5 root root 43 Apr 30 10:27 .
drwxr-xr-x 9 root root 190 Jul 27 20:15 …
drwxr-xr-x 2 root root 6 Apr 30 10:27 deploy
drwxr-xr-x 2 root root 6 Apr 30 10:27 post
drwxr-xr-x 2 root root 6 Apr 30 10:27 pre

/etc/letsencrypt/renewal-hooks/deploy:
total 0
drwxr-xr-x 2 root root 6 Apr 30 10:27 .
drwxr-xr-x 5 root root 43 Apr 30 10:27 …

/etc/letsencrypt/renewal-hooks/post:
total 0
drwxr-xr-x 2 root root 6 Apr 30 10:27 .
drwxr-xr-x 5 root root 43 Apr 30 10:27 …

/etc/letsencrypt/renewal-hooks/pre:
total 0
drwxr-xr-x 2 root root 6 Apr 30 10:27 .
drwxr-xr-x 5 root root 43 Apr 30 10:27 …

[root@tempnew conf.d]# tail -50 /var/log/letsencrypt/letsencrypt.log
Description: Apache Web Server plugin
Interfaces: IAuthenticator, IInstaller, IPlugin
Entry point: apache = certbot_apache.entrypoint:ENTRYPOINT
Initialized: <certbot_apache.override_centos.CentOSConfigurator object at 0x7f9835d76d10>
Prep: True
2019-07-27 20:15:50,778:DEBUG:certbot.plugins.selection:Selected authenticator <certbot_apache.override_centos.CentOSConfigurator object at 0x7f9835d76d10> and installer <certbot_apache.override_centos.CentOSConfigurator object at 0x7f9835d76d10>
2019-07-27 20:15:50,778:INFO:certbot.plugins.selection:Plugins selected: Authenticator apache, Installer apache
2019-07-27 20:15:50,797:DEBUG:certbot.main:Picked account: <Account(RegistrationResource(body=Registration(status=None, terms_of_service_agreed=None, agreement=None, only_return_existing=None, contact=(), key=None, external_account_binding=None), uri=u’https://acme-v02.api.letsencrypt.org/acme/acct/56239394’, new_authzr_uri=None, terms_of_service=None), 6ff7661297857e0d13da59ea18217eac, Meta(creation_host=u’pasem-t1kc-tempnew’, creation_dt=datetime.datetime(2019, 5, 1, 17, 59, 26, tzinfo=)))>
2019-07-27 20:15:50,804:DEBUG:acme.client:Sending GET request to https://acme-v02.api.letsencrypt.org/directory.
2019-07-27 20:15:50,807:INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): acme-v02.api.letsencrypt.org
2019-07-27 20:15:50,813:WARNING:certbot.renewal:Attempting to renew cert (www.the1000kmchallenge.com) from /etc/letsencrypt/renewal/www.the1000kmchallenge.com.conf produced an unexpected error: [Errno 2] No such file or directory. Skipping.
2019-07-27 20:15:50,815:DEBUG:certbot.renewal:Traceback was:
Traceback (most recent call last):
File “/usr/lib/python2.7/site-packages/certbot/renewal.py”, line 452, in handle_renewal_request
main.renew_cert(lineage_config, plugins, renewal_candidate)
File “/usr/lib/python2.7/site-packages/certbot/main.py”, line 1191, in renew_cert
le_client = _init_le_client(config, auth, installer)
File “/usr/lib/python2.7/site-packages/certbot/main.py”, line 612, in _init_le_client
return client.Client(config, acc, authenticator, installer, acme=acme)
File “/usr/lib/python2.7/site-packages/certbot/client.py”, line 266, in init
acme = acme_from_config_key(config, self.account.key, self.account.regr)
File “/usr/lib/python2.7/site-packages/certbot/client.py”, line 51, in acme_from_config_key
return acme_client.BackwardsCompatibleClientV2(net, key, config.server)
File “/usr/lib/python2.7/site-packages/acme/client.py”, line 814, in init
directory = messages.Directory.from_json(net.get(server).json())
File “/usr/lib/python2.7/site-packages/acme/client.py”, line 1152, in get
self._send_request(‘GET’, url, **kwargs), content_type=content_type)
File “/usr/lib/python2.7/site-packages/acme/client.py”, line 1101, in _send_request
response = self.session.request(method, url, *args, **kwargs)
File “/usr/lib/python2.7/site-packages/requests/sessions.py”, line 464, in request
resp = self.send(prep, **send_kwargs)
File “/usr/lib/python2.7/site-packages/requests/sessions.py”, line 576, in send
r = adapter.send(request, **kwargs)
File “/usr/lib/python2.7/site-packages/requests/adapters.py”, line 431, in send
raise SSLError(e, request=request)
SSLError: [Errno 2] No such file or directory

2019-07-27 20:15:50,815:ERROR:certbot.renewal:All renewal attempts failed. The following certs could not be renewed:
2019-07-27 20:15:50,816:ERROR:certbot.renewal: /etc/letsencrypt/live/www.the1000kmchallenge.com/fullchain.pem (failure)
2019-07-27 20:15:50,816:DEBUG:certbot.log:Exiting abnormally:
Traceback (most recent call last):
File “/usr/bin/certbot”, line 9, in
load_entry_point(‘certbot==0.31.0’, ‘console_scripts’, ‘certbot’)()
File “/usr/lib/python2.7/site-packages/certbot/main.py”, line 1365, in main
return config.func(config, plugins)
File “/usr/lib/python2.7/site-packages/certbot/main.py”, line 1272, in renew
renewal.handle_renewal_request(config)
File “/usr/lib/python2.7/site-packages/certbot/renewal.py”, line 477, in handle_renewal_request
len(renew_failures), len(parse_failures)))
Error: 1 renew failure(s), 0 parse failure(s)

4

I had guessed that something was wrong with your /etc/letsencrypt/, but apparently not.

This seems to mean that the urllib3 library, which is used by the requests library, can't find your system's CA certificate bundle.

I'm not certain where it's supposed to be on CentOS, but can you post "ls -l /etc/ssl/certs/*.crt"?

5

I have no clue at all~ and yes sir, there you go::sweat_smile:

[root@tempnew conf.d]# ls -l /etc/ssl/certs/*.crt
-rw------- 1 root root 1452 Apr 28 14:51 /etc/ssl/certs/localhost.crt

6

Can you try “ls -l /etc/pki/tls/certs/*.crt”?

I’m sorry, I don’t have much experience with CentOS.

7

no worry, same here… :sweat_smile:

[root@tempnew conf.d]# ls -l /etc/pki/tls/certs/*.crt
-rw------- 1 root root 1452 Apr 28 14:51 /etc/pki/tls/certs/localhost.crt

8

Is the ca-certificates package installed?

My understanding is that there should be a file or two named “ca-bundle.crt” or similar in one of those two directories.

9

yes, already installed:
yum list installed | grep ca-certificates
ca-certificates.noarch 2018.2.22-70.0.el7_5 @base

under these 2 directory, I do remember I saw it somewhere, but nowhere to found inside the 2 locations…

[root@tempnew conf.d]# pwd
/etc/pki/tls/certs
[root@tempnew conf.d]# ls -la
total 28
drwxr-xr-x. 3 root root 156 Jul 27 20:47 .
drwxr-xr-x. 5 root root 88 May 7 18:18 …
drwxr-xr-x 2 root root 6 May 7 19:34 back
-rw-r–r-- 1 root root 1939 May 7 19:49 cert.pem
-rw-r–r-- 1 root root 1647 May 7 19:49 chain.pem
-rw-r–r-- 1 root root 3586 May 7 19:49 fullchain.pem
-rw------- 1 root root 1452 Apr 28 14:51 localhost.crt
-rwxr-xr-x 1 root root 610 Mar 12 10:12 make-dummy-cert
-rw-r–r-- 1 root root 2516 Mar 12 10:12 Makefile
-rwxr-xr-x 1 root root 829 Mar 12 10:12 renew-dummy-cert

[root@tempnew conf.d]# pwd
/etc/ssl/certs
[root@tempnew conf.d]# ls -la
total 28
drwxr-xr-x. 3 root root 156 Jul 27 20:47 .
drwxr-xr-x. 5 root root 88 May 7 18:18 …
drwxr-xr-x 2 root root 6 May 7 19:34 back
-rw-r–r-- 1 root root 1939 May 7 19:49 cert.pem
-rw-r–r-- 1 root root 1647 May 7 19:49 chain.pem
-rw-r–r-- 1 root root 3586 May 7 19:49 fullchain.pem
-rw------- 1 root root 1452 Apr 28 14:51 localhost.crt
-rwxr-xr-x 1 root root 610 Mar 12 10:12 make-dummy-cert
-rw-r–r-- 1 root root 2516 Mar 12 10:12 Makefile
-rwxr-xr-x 1 root root 829 Mar 12 10:12 renew-dummy-cert

10

Maybe the file(s) got moved or deleted?

Or a script to generate them didn’t work?

What’s in the “back” directory?

11

the “back” is just empty folder created sometime ago…going to remove it.

anyway, could the issue be causing by this missing crt file? I guess it has been deleted or moved to somewhere already, let me see if I can find it or regenerate the file…

thanks for your help!

12

Hi mnordhoff,

I just wanted to update the progress of my issue yesterday, it has been solved by reinstalling the ca-certificates…the old files could be removed by the previous guy.

[root@tempnew conf.d]# yum reinstall ca-certificates
[root@tempnew conf.d]# certbot renew
…
…
Congratulations, all renewals succeeded. The following certs have been renewed:
** /etc/letsencrypt/live/www.the1000kmchallenge.com/fullchain.pem (success)**
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
[root@tempnew conf.d]#

Thank you for taking the time to help me and giving me the clue to the solutions, have a good day ahead! :smile:

2 Likes
13

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.