Can't tell if cron jobs for letsencrypt run

Help
1

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. https://crt.sh/?q=example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: a-view.org

I ran this command:

It produced this output:

My web server is (include version): nginx

The operating system my web server runs on is (include version): Ubuntu 16.04.5 LTS

My hosting provider, if applicable, is:

I can login to a root shell on my machine (yes or no, or I don’t know):

I’m using a control panel to manage my site (no, or provide the name and version of the control panel):

There is SO much contradictory information about cron on the web. Full of inconsistencies.

I seem to have two commands that are running cron jobs, but I can’t tell if they have ever run successfully.

I created a new one with crontab -e that looks like this as shown by crontab - l:

43 6 * * * certbot renew --post-hook "service nginx restart"

It is hidden away at /var/spool/cron/crontabs in the file root.

But, there is no evidence it ever runs. It never gets moved into the /etc/cron.* directories. So, does that mean it will never ever run?

There is another cron job in /etc/cron.d in a file called certbot that looks like this:

# /etc/cron.d/certbot: crontab entries for the certbot package
#
# Upstream recommends attempting renewal twice a day
#
# Eventually, this will be an opportunity to validate certificates
# haven't been revoked, etc. Renewal will only occur if expiration
# is within 30 days.
SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
0 */12 * * * root test -x /usr/bin/certbot -a \! -d /run/systemd/system && perl -e 'sleep int(rand(43200))' && certbot -q renew

It seems like the above just runs a test and doesn’t run the command itself. Sheesh.

These are QUITE different. Here is a log grep for past cron jobs:

grep CRON /var/log/syslog
Sep 15 07:17:01 ghost-trial CRON[28175]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Sep 15 08:17:01 ghost-trial CRON[28326]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Sep 15 09:17:01 ghost-trial CRON[28491]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Sep 15 10:17:01 ghost-trial CRON[28635]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Sep 15 11:17:01 ghost-trial CRON[28846]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Sep 15 12:00:01 ghost-trial CRON[28924]: (root) CMD (test -x /usr/bin/certbot -a \! -d /run/systemd/system && perl -e 'sleep int(rand(43200))' && certbot -q renew)
Sep 15 12:17:01 ghost-trial CRON[28942]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Sep 15 13:17:01 ghost-trial CRON[29007]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Sep 15 14:17:01 ghost-trial CRON[29070]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Sep 15 15:17:01 ghost-trial CRON[29092]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Sep 15 16:17:01 ghost-trial CRON[29138]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Sep 15 17:17:01 ghost-trial CRON[29193]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Sep 15 18:17:01 ghost-trial CRON[29242]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Sep 15 19:17:01 ghost-trial CRON[29508]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Sep 15 19:46:36 ghost-trial cron[29810]: (CRON) INFO (pidfile fd = 3)
Sep 15 19:46:36 ghost-trial cron[29810]: (CRON) INFO (Skipping @reboot jobs -- not system startup)

Why is there no evidence that the crontab job created with crontab -e has ever run? What is the point of that command? Do things created that way just go into la-la land?

This just looks like something created by a bum script (like most linux scripts–full of errors and vulnerabilities).

2

So, “RTFM”:

sudo certbot renew --dry-run
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/a-view.org.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cert not due for renewal, but simulating renewal for dry run
Plugins selected: Authenticator nginx, Installer nginx
Starting new HTTPS connection (1): acme-staging-v02.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for a-view.org
http-01 challenge for lnotes.a-view.org
http-01 challenge for www.a-view.org
Waiting for verification...
Cleaning up challenges

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
new certificate deployed with reload of nginx server; fullchain is
/etc/letsencrypt/live/a-view.org/fullchain.pem
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
** DRY RUN: simulating 'certbot renew' close to cert expiry
**          (The test certificates below have not been saved.)

Congratulations, all renewals succeeded. The following certs have been renewed:
  /etc/letsencrypt/live/a-view.org/fullchain.pem (success)
** DRY RUN: simulating 'certbot renew' close to cert expiry
**          (The test certificates above have not been saved.)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

IMPORTANT NOTES:
 - Your account credentials have been saved in your Certbot
   configuration directory at /etc/letsencrypt. You should make a
   secure backup of this folder now. This configuration directory will
   also contain certificates and private keys obtained by Certbot so
   making regular backups of this folder is ideal.

So, the renewal dry run shows that the renew config and the rest of letsencrypt is healthy. But, this has nothing to do with cron.

The documentation on actually automating renewal is sparse (as in, not useful at all):

If you are not sure whether or not your system has this already automated, refer to your distribution’s documentation, or check your system’s crontab (typically in /etc/crontab/ and /etc/cron.*/* and systemd timers (systemctl list-timers).

I realize its not your job to document everything about cron, but you could at least have correct examples for the cron line and where it should go.

The “why only 90 days” article makes clear that short expiration forces the use of automation and short-lived certs reduce exposure to possible bad credentials. So, I guess this is important to get right.

3

No, it wouldn't. It goes into the root user's crontab.

No, it first checks to see if the command is there, and if so, it runs the renewal after a random delay. It does that twice a day.

To see if it's running, check in /var/log/letsencrypt/.

4

Unless there's a systemd timer :wink: So yes, a lot of checks, but in the end, if all checks out, certbot will be run!

5

@lewisl @danb35 @Osiris

It it true.

And it might be a systemd timer...
I see this in the above output:

Starting new HTTPS connection (1): acme-staging-v02.api.letsencrypt.org

I experienced a similar issue as a result of one line in "cli.ini"

find /etc/ -type f -name cli.ini
cat /etc/certbot/cli.ini

Verify the server is NOT the staging (test) server...
# server = https://acme-staging.api.letsencrypt.org/directory

But is actually :

The production server.

server = https://acme-v01.api.letsencrypt.org/directory

Learn how cli.ini works:
https://certbot.eff.org/docs/using.html#configuration-file

Might help a bit!
Rip

6

I have been looking at the user guide but it says very little about automating renewal except to run the dry-run test.

Are you saying I need to change cli.ini? Of course, if it is wrong it is because ghost-cli installs it wrong because I never did anything manually.

When will we get past “optimistic” scripts that just copy or create a bunch of things in various directories and hope it all works. Scripts that install complex software should do a substantial scan for all previous instances and all of the various conf files and ini files that scattered across the entire file system and resolve all of it. Course, that’s a bit much… …but it is the only way installer scripts can ensure correct configuration.

7

Something definitely runs a couple of times a day. Impossible for me to understand what it actually does, but something happens that creates a lot of output.

The simple one I created seems to have never run.

My cli.ini contains a comment:

cat cli.ini
# Because we are using logrotate for greater flexibility, disable the
# internal certbot logrotation.

Here is my renew.conf file:

cat a-view.org.conf
# renew_before_expiry = 30 days
version = 0.26.1
archive_dir = /etc/letsencrypt/archive/a-view.org
cert = /etc/letsencrypt/live/a-view.org/cert.pem
privkey = /etc/letsencrypt/live/a-view.org/privkey.pem
chain = /etc/letsencrypt/live/a-view.org/chain.pem
fullchain = /etc/letsencrypt/live/a-view.org/fullchain.pem

# Options used in the renewal process
[renewalparams]
authenticator = nginx
account = caa2787c9669139339ece18a72952cf9
server = https://acme-v02.api.letsencrypt.org/directory
installer = nginx

I have never edited this file. But, it looks like I should uncomment the first line?

8

No, 30 days is the default. If you wanted to change it, you'd need to uncomment that line.

9

Three observations:

(1) Putting crontab lines into a single per-user “crontab” file (e.g. in /var/spool/cron/crontabs) is the traditional way of using cron. This is what crontab -e does.

The /etc/cron.d mechanism is an alternative (to give OS packages a convenient way to add and remove cron jobs when they’re installed or uninstalled), but the old style is still supported as well.

(2) This command

0 */12 * * * root test -x /usr/bin/certbot -a \! -d /run/systemd/system && perl -e 'sleep int(rand(43200))' && certbot -q renew

does run the renewal if the test succeeds—note the certbot -q renew at the very end. The && syntax tells the shell to run the following command if the previous command succeeded.

The purpose of this test is to inhibit the cron job if systemd is installed, because the OS package that creates this cron job also creates a systemd timer which would be redundant with the cron job if systemd is installed on your system (which is most likely the case in modern OSes). Note that your manually-created crontab line is also redundant with this OS-package-created cron job (although it doesn’t specify a post-hook, which would be saved in /etc/letsencrypt/renewal/a-view.org.conf if you specified it when obtaining or manually renewing the certificate, or which you could also add by editing that file).

(3) Another option for confirming that certbot renew is getting run periodically is to check the Certbot log files in /var/log/letsencrypt, which should have a new log file for every time that Certbot was run, hopefully ending in a line like

2018-09-16 01:37:32,518:DEBUG:certbot.renewal:no renewal failures

This line would be present even if no renewals were attempted due to all of the installed certificates having enough remaining validity.

10

It looks like you used your operating system's packages to install Certbot since

# /etc/cron.d/certbot: crontab entries for the certbot package

is a file created by an operating system package. In this case, the operating system packager has assumed responsibility for making sure that the installation method will work properly in the context of your operating system's facilities. If not, that is a bug in the OS package and the packager will be expected to fix it as a bug in the operating system (or a bug in the PPA and the PPA packager will be expected to fix it, if you're using the PPA version).

While we've sometimes struggled to ensure that Certbot packages were up-to-date enough to include various important fixes, I think the locus of responsibility here is clear overall and we haven't seen any evidence that the system isn't working in your case. (In particular, if you installed Certbot from your OS packages, please don't use tutorials that assume a different installation method.)

11

I did install certbot with apt-get install as the version that ghost-cli installed (you guys have no responsibility for what they do) was the older letsencrypt or even acme version.

12

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.