Crontab Certificate Renewal is missed

4m3ndy · August 9, 2017, 3:19pm

Hello Everyone,
I'd like to discuss my problem with you and I have been facing this on multiple servers.

Certificate Renewal is being missed and all the configurations are correct, and when I run the dry run test, it runs successfully, however when I trace the let's encrypt log I get no renewal, and the certificate is due to renewal as it's 30 days before expiration date.

My Environment:

OS: CentOS 7
WebServer: nginx

This is how I obtained the certificate:

certbot certonly --webroot -w /var/www/html -d jobs.espace.com.eg -d jobs.espace-technologies.com

I do the certificate renewal through crontab

0 4 * * 0 /bin/certbot renew --post-hook "nginx -s reload" >> /var/log/letsencrypt-renew.log

and Here's the certificate renewal history of the certificate:
crt.sh | jobs.espace.com.eg

The second renewal done successfully which is crt.sh | 132764979, while the third renewal was not attempted crt.sh | 182708178, so I had to renew the certificate myself manually after the expiration of the certificate.

Here's my configurations

version = 0.8.1
cert = /etc/letsencrypt/live/jobs.espace-technologies.com/cert.pem
privkey = /etc/letsencrypt/live/jobs.espace-technologies.com/privkey.pem
chain = /etc/letsencrypt/live/jobs.espace-technologies.com/chain.pem
fullchain = /etc/letsencrypt/live/jobs.espace-technologies.com/fullchain.pem
[renewalparams]
authenticator = webroot
installer = None

and this is the output of the --dry-run test:

Processing /etc/letsencrypt/renewal/jobs.espace-technologies.com.conf
** DRY RUN: simulating 'certbot renew' close to cert expiry
** (The test certificates below have not been saved.)

Congratulations, all renewals succeeded. The following certs have been renewed:
/etc/letsencrypt/live/jobs.espace-technologies.com/fullchain.pem (success)
** DRY RUN: simulating 'certbot renew' close to cert expiry
** (The test certificates above have not been saved.)

jared.m · August 9, 2017, 3:48pm

Could you post the logs where it’s failing?

4m3ndy · August 9, 2017, 3:59pm

No Failures happened, and this is the when list the files the /var/log/letsencrypt directory

There's no log file created within jun 16 and 2 august, which means that the cronjob did not run within this period

The Jun 16 log file is an attempt that was not due to renewal
The Aug 2 log file is my manual attempt to renew the certificate

drwx------. 2 root root 4.0K Aug 9 17:35 .
drwxr-xr-x. 11 root root 4.0K Aug 6 03:39 ..
-rw-r--r--. 1 root root 4.7K Aug 9 17:36 letsencrypt.log
-rw-r--r--. 1 root root 70K Aug 9 17:08 letsencrypt.log.1
-rw-r--r--. 1 root root 4.6K May 14 04:00 letsencrypt.log.10
-rw-r--r--. 1 root root 64K Aug 2 15:27 letsencrypt.log.2
-rw-r--r--. 1 root root 64K Aug 2 15:21 letsencrypt.log.3
-rw-r--r--. 1 root root 4.6K Jun 16 03:20 letsencrypt.log.4
-rw-r--r--. 1 root root 64K Jun 16 03:20 letsencrypt.log.5
-rw-r--r--. 1 root root 4.6K Jun 11 04:00 letsencrypt.log.6
-rw-r--r--. 1 root root 4.6K Jun 4 04:00 letsencrypt.log.7
-rw-r--r--. 1 root root 4.6K May 28 04:00 letsencrypt.log.8
-rw-r--r--. 1 root root 4.6K May 21 04:00 letsencrypt.log.9

jared.m · August 9, 2017, 4:06pm

First note is that it’s recommended to run renewal twice daily, no weekly. Second, if it’s not actually running, you should probably try and figure out why your cron isn’t executing commands as expected. What’s the August 9th logs? Are those also only manual renewals? You should see log entries from certbot every time it runs, indicating that a renewal was or was not attempted and the outcome if it was attempted.

rg305 · August 9, 2017, 4:13pm

I would try adding webroot to the renewal process:
/bin/certbot renew --webroot -w /var/www/html --post-hook “nginx -s reload” >> /var/log/letsencrypt-renew.log

Also, webroot uses http and, unless you are bypassing the acme-challenges, one of the sites is re-re-re-directing the http (to https then http then https):
wget jobs.espace.com.eg
–2017-08-09 12:14:47-- http://jobs.espace.com.eg/
Resolving jobs.espace.com.eg (jobs.espace.com.eg)… 138.201.180.85
Connecting to jobs.espace.com.eg (jobs.espace.com.eg)|138.201.180.85|:80… connected.
HTTP request sent, awaiting response… 301 Moved Permanently
Location: https://jobs.espace.com.eg/ [following]
–2017-08-09 12:14:48-- https://jobs.espace.com.eg/
Connecting to jobs.espace.com.eg (jobs.espace.com.eg)|138.201.180.85|:443… connected.
HTTP request sent, awaiting response… 302 Moved Temporarily
Location: http://espace.com.eg/company/jobs [following]
–2017-08-09 12:14:48-- http://espace.com.eg/company/jobs
Resolving espace.com.eg (espace.com.eg)… 97.107.136.109
Connecting to espace.com.eg (espace.com.eg)|97.107.136.109|:80… connected.
HTTP request sent, awaiting response… 302 Moved Temporarily
Location: https://espace.com.eg/company/jobs [following]
–2017-08-09 12:14:48-- https://espace.com.eg/company/jobs
Connecting to espace.com.eg (espace.com.eg)|97.107.136.109|:443… connected.
HTTP request sent, awaiting response… 200 OK

jared.m · August 9, 2017, 4:19pm

Adding --webroot is unnecessary and not best practice in general. Renewal uses the same authorization method in use for issuance - this would be webroot in this case.

Webroot can follow https redirects - that’s not an issue.

rg305 · August 9, 2017, 4:22pm

But the redirect leads to a completely different domain (and IP).
Could be problematic if not considered in the design.

jared.m · August 9, 2017, 4:23pm

Good point - noticed the domains, but the IP may be problematic.

rg305 · August 9, 2017, 4:26pm

Something has changed since the last working renewal.
Maybe the redirections.
Maybe the webroot folder was moved.
Maybe the acme-challenge alias was changed.
…

schoen · August 9, 2017, 4:27pm

I want to point out by way of agreement with this concern that certbot renew is designed to work in environments where you have perhaps dozens of different certificates which are obtained by all different authentication methods. In that case, specifying authentication information related to just one of the certificates would be a recipe for disaster (failed renewals) because it would be used for various certificates that it didn't actually apply to.

The intended design is indeed to specify the authentication parameters with certonly or run, and then have Certbot remember them for the future and supply them automatically with renew. If you need to change them, the recommended way is via a single (potentially --force-renewal) renewal with certonly and appropriate parameters, which should be saved.

4m3ndy · August 9, 2017, 8:29pm

Here's my nginx configuration:

server {
listen 80 default_server;
server_name jobs.espace-technologies.com jobs.espace.com.eg;
return 301 https://$host$request_uri;
}
server {
listen 443 default_server;
server_name jobs.espace-technologies.com jobs.espace.com.eg;
ssl on;
ssl_certificate /etc/letsencrypt/live/jobs.espace-technologies.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/jobs.espace-technologies.com/privkey.pem;
    location =/{
    rewrite (.*) http://espace.com.eg/company/jobs redirect;
    }

    location ~ /.well-known {
            root /var/www/html;
            allow all;
    }
}

according to these configuration, we want to redirect the root location to go to espace.com.eg/jobs, but when it requires .well-known locations, it servers over https.

The webroot location wasn't changed, and the as you can see the configurations are correct.

4m3ndy · August 9, 2017, 8:31pm

so This mean that It’s better to use certbot renew with out --webroot, did I get this right ?

4m3ndy · August 9, 2017, 8:37pm

Yes do those a --dry-run test by me, just to make sure the configuration was right, and when I traced the log /var/log/letsencrypt/letsencrypt.log, the renewal dry-run was successful.

I'll try to update my cronjob with this, but the execution of the renewal cron during the 30 days before expiration wasn't unreasonable, that's why I'm trying to debug the environment, and I can see all the configurations are right till now.

rg305 · August 9, 2017, 8:58pm

try placing a test file at:
/var/www/html/acme-challenge/test.txt
and see if it is accessible via the Internet

schoen · August 9, 2017, 9:13pm

Maybe also /var/www/html/.well-known/acme-challenge/test.txt for greater realism.

rg305 · August 9, 2017, 9:18pm

Doesn't that essentially remove the "/.well-known" portion form the URL?
mapping
http://fqdn/.well-known
to
/var/www/html
and
http://fqdn/.well-known/acme-challenge/test.txt
to
/var/www/html/acme-challenge/test.txt

schoen · August 9, 2017, 10:03pm

We’ve definitely had people have trouble with that issue before. It might be more correct with root /var/www/html/.well-known in this case.

4m3ndy · August 10, 2017, 12:34am

When /.well-known is requested by the acme challenge, it's created, and I can access secret.txt, so I think the root is fine

I have another question, If the post-hook did not successfully run, will the certificate be issued ?
as I got this in the log file

Congratulations, all renewals succeeded. The following certs have been renewed:
/etc/letsencrypt/live/jobs.espace-technologies.com/fullchain.pem (success)
Unable to find post-hook command nginx in the PATH.
(PATH is /usr/bin:/bin)

rg305 · August 10, 2017, 12:41am

Yes, the cert was issued (renewed) and can be found at

The post-hook simply restarts nginx to ensure the latest cert is in use.

I see that you have managed to get and use the latest cert:

system · September 9, 2017, 12:41am

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
I am not getting it - Crontab Certbot Renew Server	3	18021	May 11, 2019
Cert not renewing again Help	14	2144	February 8, 2017
CRON[3889]: Authentication token is no longer valid; new one required Help	2	2498	January 13, 2017
Unable auto renew by crontab under centos / nginx Help	12	3637	November 5, 2016
Failed to autorenew from cron - Could not find a usable 'nginx' binary Help	3	5812	October 24, 2019

Crontab Certificate Renewal is missed

Related topics