Manual update works, cron based update does not

Help
1

I am able to perform a certbot renewal using the commands below, no issues. But when the same commands are executed using cron, they generate an error. Details follow:

  1. Manual commands that work successfully:
    admin@4D-M1-Prod NGINX % nginx -s stop
    admin@4D-M1-Prod NGINX % sudo /opt/homebrew/bin/certbot renew
    Password:
    Saving debug log to /var/log/letsencrypt/letsencrypt.log

Processing /etc/letsencrypt/renewal/prod.mbmdb.net.conf

Renewing an existing certificate for prod.mbmdb.net

Congratulations, all renewals succeeded:
/etc/letsencrypt/live/prod.mbmdb.net/fullchain.pem (success)

admin@4D-M1-Prod NGINX % nginx -p '' -c nginx.conf
admin@4D-M1-Prod NGINX %

  1. Root level crontab runs the following script:

renew Letsencrypt certificate for NGINX web server

run from root crontab

CURRENTDATE=$(date +%c)
echo "\nChecking Letsencrypt cert renewal status at "$CURRENTDATE"\n" >> /Users/admin/Desktop/certbotRenew.txt
cd /Users/admin/Documents/NGINX
nginx -s stop >> /Users/admin/Desktop/certbotRenew.txt
sleep 3
/opt/homebrew/bin/certbot renew >> /Users/admin/Desktop/certbotRenew.txt
sleep 3
nginx -p '' -c nginx.conf >> /Users/admin/Desktop/certbotRenew.txt

The output from the above script is:

Checking Letsencrypt cert renewal status at Wed Apr 10 23:45:00 2024

Processing /etc/letsencrypt/renewal/prod.mbmdb.net.conf

Renewing an existing certificate for prod.mbmdb.net

Certbot failed to authenticate some domains (authenticator: standalone). The Certificate Authority reported these problems:
Domain: prod.mbmdb.net
Type: unauthorized
Detail: 24.247.206.182: Invalid response from MBM Giving "<html lang="en"><link rel="preconnect" href="https://fonts.gstatic.com" crossorigin="">\n <meta charset="ut"

Hint: The Certificate Authority failed to download the challenge files from the temporary standalone webserver started by Certbot on port 80. Ensure that the listed domains point to this machine and that it can accept inbound connections from the internet.

QUESTION: The scheduled cron run script is run from the root level cron. Why does it produce different results compared to using sudo to manually run the same command? The nginx server is running on port 80, but as you can see, it is stopped ahead of running the cert renewal, so there should not be a conflict when/if your process starts up a temporary server on port 80. I put sleep commands in to provide a delay but this did not address it.

My domain is: prod.mbmdb.net
Server version:
nginx/1.25.3
The operating system my web server runs on is (include version): MacOS Sonoma 14.4.1
My hosting provider, if applicable, is: N/A
I can login to a root shell on my machine (yes or no, or I don't know): yes
I'm using a control panel to manage my site (no, or provide the name and version of the control panel): No
The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): certbot 2.7.4

2

The HTML shown looks like part of your website.

Usually one only stops their webserver for usage with the standalone authenticator. However, the webserver spun up by the Certbot standalone plugin would not return such HTML: it would return the challenge token or simply close the connection if it would encounter something unexpected.. Not HTML.

So your nginx is probably not actually stopped? Because why would there be that HTML that looks exactly like the first line of your website? That said, why wouldn't the standalone plugin error out when port 80 was already used by nginx? Very weird.

Also, is there a specific reason why you're using the standalone plugin to begin with? Usually the --nginx authenticator is more userfriendly and if you don't want anything messing with your nginx configuration, you could usually simply use the --webroot authenticator.

2 Likes
3

Possibly I need to supply the full path for the nginx command when it is run from root, and that may be the cause of nginx not stopping; I will check that.

I did not pursue using the --nginx authenticator because the basic certbot renew command worked without issues. In our case, stopping nginx for a minute overnight is not an issue. But I have no issue evaluating the nginx approach to authentication, and will evaluate that.

Most importantly, thank you for your reply, it is quite helpful to get experienced eyes on this.

Kirk

4

My preference is to use the standalone authenticator, but on a higher port such as --http-01-port=8080, then have nginx proxy the /.well-known/acme-challenge directory onto that port.

I recommend that approach for a few reasons:
1- certbot does not touch nginx, which minimizes issues
2- setup and troubleshooting only require being able to configure and test a proxypass.

Here's a python script i wrote that can be used to test a proxypass onto /.well-known/acme-challenge/{challenge}: peter_sslers/tools/fake_server.py at main · aptise/peter_sslers · GitHub

There is nothing wrong with using Certbot and nginx (unless you have 100+ sites or a dynamic configuration via openresty). I just find the proxypass method much easier to setup, troubleshoot and manage as it has minimal initial downtime and will have 100% uptime once it's going as Nginx supports a graceful restart on certificate rotation.

3 Likes
5

@jvanasco I agree the --nginx authenticator might not be suitable in certain situations (large configurations et c.), but what's wrong with the webroot authenticator? It doesn't modify the nginx configuration and you could even use a certain special location for the /.well-known/acme-challenge/ path without the added difficulty of proxying.

1 Like
6

In my experience, the webroot is often more difficult to correctly set up in Apache and Nginx due to existing directory configuration and aliasing rules. People also often have issues correctly configuring Certbot to write the correct directory. If the apache/nginx block is a static website with a single virtualhost and document root - it is easy and straightforward. Once there are other directives in place though, it can be nightmarish.

I've found proxypass to be much simpler as there is no configuration for Certbot, and the proxypass avoids invoking the various configuration directives that (needlessly over)complicate configuring Nginx/Apache.

IMHO, on a complexity scale of 1-10, weboot will either be a 1 or a 10 based on the existing config, while proxypass will reliably be a 2.

2 Likes