Failed certificate renewal - nginx

Help
1

Hi all,

I've recently ventured into the world of self-hosting, and am essentially starting from a low knowledge base. I have set up a PC at home where I'm running three hosting services - Plex, Audiobookshelf, and Calibreweb. Plex is relatively easy to run, as it comes with its own app etc.

For both Calibreweb and Audiobookshelf, I have:

  • installed docker and created containers for them to both run in
  • installed NGINX as a reverse proxy manager, within docker
  • registered a domain
  • created a DNS entry to point from my domain to each my PC
  • set up NGINX to receive the URL, and point it to my PCs IP and port.

my domains are:
calibreweb.drumm.one
audiobookshelf.drumm.one

In setting up NGINX, I had it create a certificate for each of the services. I got my email recently saying the certificates needed renewal. I logged into my NGINX dashboard, and attempted to do the renewal, and got an "internal error". I attended to test server reachability, and got the error "communication with the API failed, is NPM running correctly?"

I have opened up my NGINX docker container, and checked the log files, and see the following (I believe there are two attempts in there - one is the automated auto-renew, and the other was my forced attempt):

2023-09-03 09:17:47 [9/2/2023] [11:17:47 PM] [SSL ] › :heavy_multiplication_x: error Error: Command failed: certbot renew --non-interactive --quiet --config "/etc/letsencrypt.ini" --work-dir "/tmp/letsencrypt-lib" --logs-dir "/tmp/letsencrypt-log" --preferred-challenges "dns,http" --disable-hook-validation
2023-09-03 09:17:47 Failed to renew certificate npm-5 with error: Some challenges have failed.
2023-09-03 09:17:47 Failed to renew certificate npm-6 with error: Some challenges have failed.
2023-09-03 09:17:47 All renewals failed. The following certificates could not be renewed:
2023-09-03 09:17:47 /etc/letsencrypt/live/npm-5/fullchain.pem (failure)
2023-09-03 09:17:47 /etc/letsencrypt/live/npm-6/fullchain.pem (failure)
2023-09-03 09:17:47 2 renew failure(s), 0 parse failure(s)
2023-09-03 09:17:47
2023-09-03 09:17:47 at ChildProcess.exithandler (node:child_process:402:12)
2023-09-03 09:17:47 at ChildProcess.emit (node:events:513:28)
2023-09-03 09:17:47 at maybeClose (node:internal/child_process:1100:16)
2023-09-03 09:17:47 at Process.ChildProcess._handle.onexit (node:internal/child_process:304:5)
2023-09-03 10:12:02 [9/3/2023] [12:12:02 AM] [SSL ] › :information_source: info Renewing SSL certs close to expiry...
2023-09-03 10:15:51 [9/3/2023] [12:15:51 AM] [SSL ] › :heavy_multiplication_x: error Error: Command failed: certbot renew --non-interactive --quiet --config "/etc/letsencrypt.ini" --work-dir "/tmp/letsencrypt-lib" --logs-dir "/tmp/letsencrypt-log" --preferred-challenges "dns,http" --disable-hook-validation
2023-09-03 10:15:51 Failed to renew certificate npm-5 with error: Some challenges have failed.
2023-09-03 10:15:51 Failed to renew certificate npm-6 with error: Some challenges have failed.
2023-09-03 10:15:51 All renewals failed. The following certificates could not be renewed:
2023-09-03 10:15:51 /etc/letsencrypt/live/npm-5/fullchain.pem (failure)
2023-09-03 10:15:51 /etc/letsencrypt/live/npm-6/fullchain.pem (failure)
2023-09-03 10:15:51 2 renew failure(s), 0 parse failure(s)
2023-09-03 10:15:51
2023-09-03 10:15:51 at ChildProcess.exithandler (node:child_process:402:12)
2023-09-03 10:15:51 at ChildProcess.emit (node:events:513:28)
2023-09-03 10:15:51 at maybeClose (node:internal/child_process:1100:16)
2023-09-03 10:15:51 at Process.ChildProcess._handle.onexit (node:internal/child_process:304:5)
2023-09-03 11:12:02 [9/3/2023] [1:12:02 AM] [SSL ] › :information_source: info Renewing SSL certs close to expiry...
2023-09-03 11:16:28 [9/3/2023] [1:16:28 AM] [SSL ] › :information_source: info Renewing Let'sEncrypt certificates for Cert #5: audiobookshelf.drumm.one
2023-09-03 11:16:28 [9/3/2023] [1:16:28 AM] [SSL ] › :information_source: info Command: certbot renew --force-renewal --config "/etc/letsencrypt.ini" --work-dir "/tmp/letsencrypt-lib" --logs-dir "/tmp/letsencrypt-log" --cert-name "npm-5" --preferred-challenges "dns,http" --no-random-sleep-on-renew --disable-hook-validation
2023-09-03 11:16:29 [9/3/2023] [1:16:29 AM] [Express ] › :warning: warning Command failed: certbot renew --force-renewal --config "/etc/letsencrypt.ini" --work-dir "/tmp/letsencrypt-lib" --logs-dir "/tmp/letsencrypt-log" --cert-name "npm-5" --preferred-challenges "dns,http" --no-random-sleep-on-renew --disable-hook-validation
2023-09-03 11:16:29 Another instance of Certbot is already running.
2023-09-03 11:16:29 Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /tmp/certbot-log-6palt_ay/log or re-run Certbot with -v for more details.
2023-09-03 11:16:29
2023-09-03 11:16:46 [9/3/2023] [1:16:46 AM] [SSL ] › :information_source: info Testing http challenge for audiobookshelf.drumm.one
2023-09-03 11:16:47 Uncaught SyntaxError: Unexpected end of JSON input
2023-09-03 11:16:47
2023-09-03 11:16:47 FROM
2023-09-03 11:16:47 bash: line 1: 15131 Trace/breakpoint trap node --abort_on_uncaught_exception --max_old_space_size=250 index.js
2023-09-03 11:16:48 ❯ Starting backend ...
2023-09-03 11:16:48 [9/3/2023] [1:16:48 AM] [Global ] › :information_source: info Using Sqlite: /data/database.sqlite
2023-09-03 11:16:49 [9/3/2023] [1:16:49 AM] [Migrate ] › :information_source: info Current database version: none
2023-09-03 11:16:49 [9/3/2023] [1:16:49 AM] [Setup ] › :information_source: info Logrotate Timer initialized
2023-09-03 11:16:49 [9/3/2023] [1:16:49 AM] [Setup ] › :information_source: info Logrotate completed.
2023-09-03 11:16:49 [9/3/2023] [1:16:49 AM] [IP Ranges] › :information_source: info Fetching IP Ranges from online services...
2023-09-03 11:16:49 [9/3/2023] [1:16:49 AM] [IP Ranges] › :information_source: info Fetching https://ip-ranges.amazonaws.com/ip-ranges.json
2023-09-03 11:16:51 [9/3/2023] [1:16:51 AM] [IP Ranges] › :information_source: info Fetching https://www.cloudflare.com/ips-v4
2023-09-03 11:16:51 [9/3/2023] [1:16:51 AM] [IP Ranges] › :information_source: info Fetching https://www.cloudflare.com/ips-v6
2023-09-03 11:16:51 [9/3/2023] [1:16:51 AM] [SSL ] › :information_source: info Let's Encrypt Renewal Timer initialized
2023-09-03 11:16:51 [9/3/2023] [1:16:51 AM] [SSL ] › :information_source: info Renewing SSL certs close to expiry...
2023-09-03 11:16:51 [9/3/2023] [1:16:51 AM] [IP Ranges] › :information_source: info IP Ranges Renewal Timer initialized
2023-09-03 11:16:51 [9/3/2023] [1:16:51 AM] [Global ] › :information_source: info Backend PID 1433 listening on port 3000 ...
2023-09-03 11:16:52 [9/3/2023] [1:16:52 AM] [SSL ] › :heavy_multiplication_x: error Error: Command failed: certbot renew --non-interactive --quiet --config "/etc/letsencrypt.ini" --work-dir "/tmp/letsencrypt-lib" --logs-dir "/tmp/letsencrypt-log" --preferred-challenges "dns,http" --disable-hook-validation
2023-09-03 11:16:52 Another instance of Certbot is already running.
2023-09-03 11:16:52
2023-09-03 11:16:52 at ChildProcess.exithandler (node:child_process:402:12)
2023-09-03 11:16:52 at ChildProcess.emit (node:events:513:28)
2023-09-03 11:16:52 at maybeClose (node:internal/child_process:1100:16)
2023-09-03 11:16:52 at Process.ChildProcess._handle.onexit (node:internal/child_process:304:5)
2023-09-03 11:17:29 [9/3/2023] [1:17:29 AM] [SSL ] › :information_source: info Renewing Let'sEncrypt certificates for Cert #5: audiobookshelf.drumm.one
2023-09-03 11:17:29 [9/3/2023] [1:17:29 AM] [SSL ] › :information_source: info Command: certbot renew --force-renewal --config "/etc/letsencrypt.ini" --work-dir "/tmp/letsencrypt-lib" --logs-dir "/tmp/letsencrypt-log" --cert-name "npm-5" --preferred-challenges "dns,http" --no-random-sleep-on-renew --disable-hook-validation
2023-09-03 11:17:30 [9/3/2023] [1:17:30 AM] [Express ] › :warning: warning Command failed: certbot renew --force-renewal --config "/etc/letsencrypt.ini" --work-dir "/tmp/letsencrypt-lib" --logs-dir "/tmp/letsencrypt-log" --cert-name "npm-5" --preferred-challenges "dns,http" --no-random-sleep-on-renew --disable-hook-validation
2023-09-03 11:17:30 Another instance of Certbot is already running.
2023-09-03 11:17:30 Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /tmp/certbot-log-jhxy4jqd/log or re-run Certbot with -v for more details.
2023-09-03 11:17:30

I've attempted restarting docker and the machine several times to no effect. Any help that anyone can offer would be appreciated. While a novice, I am happy to follow advice and instruction!

Thanks
BD

1 Like
2

Hi @brettdrummond, and welcome to the LE community forum :slight_smile:

That may need a reboot to clear.

And Nginx Proxy Manager is not a fan favorite in this forum.
If you can avoid using it, I would recommend that you do that.
At a bare minimum, remove:

What shows?:
certbot certificates

3 Likes
3

Do you know if you have NPM doing an HTTP or a DNS challenge? Because for http challenge you need to be able to be reached on port 80 with a public IP address. I cannot explain why you get the error about certbot still running but an HTTP challenge won't work right now until you fix your DNS

3 Likes
4

Thanks both.

@rg305 I'll look for a replacement for NGINX. What is Caddy like?

@MikeMcQ - I honestly couldn't tell you, and wouldn't know how to find out. My domain name registry has a DNS A record to point to my public IP from the URL. Does that help?

1 Like
5

It most definitely does not point to a public IP. It points to your private IP which means no one on the public internet can reach your web services. The Let's Debug links I provided showed these DNS settings:

audiobookshelf.drumm.one. 0 IN A 192.168.0.247
calibreweb.drumm.one. 0 IN A 192.168.0.247

nginx is perfectly fine. It is my personal favorite web server. The Nginx Proxy Manager is an entirely different thing.

Caddy seems well-liked by those that use it.

4 Likes
6

Ah. That's my learning curve. I've changed the IP to the public (not private IP), and run the Let's Debug page. It shows no errors.

7

Does NPM work now? That might have been the cause of the first error in that series you showed.

3 Likes
8

You beat me to it. I was just able to renew certificates without errors. Thanks for the assist (in overcoming my own stupidity!!).

2 Likes
9

Your server is still using the old cert. Is NPM supposed to reload your openresty server automatically?

Use a site like this to see the cert your server is using

3 Likes
10

I've rebooted NPM and that seemed to trigger the use of the new certificates. Thx

2 Likes