Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.
That describes an nginx config problem. First, is nginx running prior to running certbot renew ? Because it must be running before using Certbot with --nginx plugin like you do. Certbot will start a copy of nginx outside of systemd if it is not and this can cause problems.
Assuming so, what does this show?
sudo nginx -t
And, your Ubuntu version should easily support the snap install of Certbot. Your 0.40 version is very very old. See below for install instructions. Just follow them carefully.
This worked like a charm with no downtime. Now the renew --dry-run command works.
In our nginx server config, we left these lines:
ssl_certificate /etc/letsencrypt/live/opendata-stage.schleswig-holstein.de/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/opendata-stage.schleswig-holstein.de/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
We also have a basic auth active in our main server block, this is why we have this block as well:
Ubuntu 20 has snap pre-installed so not sure why you did that step. But, the rest look correct. Just to confirm, what does this show: sudo certbot --version
This may be a problem. It would not cause an error about the pid but might interfere with how Certbot --nginx handles the challenge.
With --nginx Certbot makes a temp change to your server block such that the challenge is replied directly by a "return" statement and is not processed by your location block setting auth_basic off.
This wouldn't cause a "pid" error but perhaps a validation error. But, your first post shows getting a cert so for whatever reason that much seems fine.
What is the result now with a current Certbot?
sudo certbot renew --dry-run
You didn't use sudo in your first post. Don't you need that for controlling nginx since you use --nginx option?
I don't see how that would matter when using the --nginx plugin. The way that works is by inserting a location block that uses an equal sign match. This takes precedence over all other location blocks. An example is shown below.
Did you change to --webroot method? Would you show contents of this: /etc/letsencrypt/renewal/opendata-stage.schleswig-holstein.de.conf
Sample Certbot --nginx plugin changes. The last line is the highest priority location block for nginx.
server {rewrite ^(/.well-known/acme-challenge/.*) $1 break; # managed by Certbot
listen 80;
listen [::]:80;
server_name example.com www.example.com;
root /var/www/html;
location = /.well-known/acme-challenge/MxkxRxkxzxkxgxHxvxkxOxpxkxVxkx8xkx4xFxExuxg {default_type text/plain;return 200 MxkxRxkxzxkxgxHxvxkxOxpxkxVxkx8xkx4xFxExuxg.AwQwgwgwzwgwJwgw4wew3wrwxwiw-wpwvwqwxwBwswT;} # managed by Certbot
}
Perhaps the issue stems from my ACME challenge location block, which was also active before, but without disabling basic auth. This may have led to conflicts between the location blocks.
Yes, I can confirm that completely removing that location block also allows the renewal process to work. This confirms that ACME challenges function even with basic auth enabled, as the Let's Encrypt-inserted location block takes precedence over the basic auth configuration.
Apologies for the misleading information, and thank you for resolving my issue.