Inconsistent behavior (and empty certbot logs) between cron and manual

#1

Hey folks! Appreciate your time. Any theories why these two results are not the same? Background:

  • I’ve never gotten background auto-renewal to work
  • I get the email expiry notifications and have to log in to renew
  • Running manually works perfectly first time
  • The logs are being written, but “empty” (no output between “processing” and “failure”)
  • Ubuntu 14.04.5 LTS, Python 2.7.6, nginx/1.12.1
> sudo crontab -e
0 * * * * /home/mike/certbot-auto renew --nginx >> /var/log/certbot.log
> tail /var/log/certbot.log
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/<removed>.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
All renewal attempts failed. The following certs could not be renewed:
/etc/letsencrypt/live/<removed>/fullchain.pem (failure)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> sudo -i
> /home/mike/certbot-auto renew --nginx
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/<removed>.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cert is due for renewal, auto-renewing...
Plugins selected: Authenticator nginx, Installer nginx
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for <removed>
Waiting for verification...
Cleaning up challenges
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
new certificate deployed with reload of nginx server; fullchain is
/etc/letsencrypt/live/<removed>/fullchain.pem
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Congratulations, all renewals succeeded. The following certs have been renewed:
  /etc/letsencrypt/live/<removed>/fullchain.pem (success)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
#2

It seems like you may need quotes in the cron job.
[just taking a stab at it]

try:
0 * * * * "/home/mike/certbot-auto renew --nginx" >> /var/log/certbot.log

#3

I disagree with this; quotes aren’t appropriate to the syntax of the cron job.

#4

Could you take a look at /var/log/letsencrypt? The built-in logging features in Certbot are supposed to log to files there. They might provide more details about what’s happening.

Also, you could try adding 2>&1 at the very end of the command line in the hope that your own log files will include the command’s stderr (not just stdout).

#5

Ah thanks @schoen that helps a lot. In /var/log/letsencrypt I found:

019-03-12 19:04:49,752:DEBUG:certbot.plugins.disco:No installation (PluginEntryPoint#nginx): Could not find a usable 'nginx' binary. Ensure nginx exists, the binary is executable, and your PATH is set correctly.
Traceback (most recent call last):
  File "/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/certbot/plugins/disco.py", line 132, in prepare
    self._initialized.prepare()
  File "/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/certbot_nginx/configurator.py", line 146, in prepare
    "Could not find a usable 'nginx' binary. Ensure nginx exists, "
NoInstallationError: Could not find a usable 'nginx' binary. Ensure nginx exists, the binary is executable, and your PATH is set correctly.

Which leads me to other forum threads to solve this issue.

#6

Thanks! This makes me think that the PATH variable is set differently between your root shell and your cron job. If so, it might be as easy as changing the PATH in crontab.

Maybe try echo $PATH and which nginx from the root shell and see if it’s in a moderately unusual or specific location?

#7

If you want to know your default PATH in the cron job, a simple option is to add a new cron job like

* * * * * echo $PATH | tee /tmp/path

and then wait one minute and look in the file /tmp/path for the answer. (Don’t forgot to remove this cron job later so that it doesn’t keep running in the future!)

#8

That’s great advice. Using your last comment, I discovered my $PATH during cron is:

> cat /tmp/path
/usr/bin:/bin

while…

> which nginx
/usr/sbin/nginx
> sudu -i
> which nginx
/usr/sbin/nginx

To address, I created a symlink:

> sudo ln -s /usr/sbin/nginx /usr/bin/nginx
> /usr/bin/nginx -v
nginx version: nginx/1.12.1
#9

You could also add /usr/sbin to your PATH in the crontab. I think that most versions of cron allow you to explicitly say something like

PATH=/usr/bin:/bin:/usr/sbin

as a line toward the top of the crontab file.

closed #10

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.