Nginx plugin starts working after standalone command

[lono0778]

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is:
nb.dc-smarter.com

I ran this command:
certbot --nginx --non-interactive --agree-tos -m support@dc-smarter.com --domains test.nb.dc-smarter.com --cert-name test.nb.dc-smarter.com

It produced this output:
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Requesting a certificate for image.nb.dc-smarter.com

Certbot failed to authenticate some domains (authenticator: nginx). The Certificate Authority reported these problems:
Domain: test.nb.dc-smarter.com
Type: unauthorized
Detail: 4.184.221.233: Invalid response from https://image.nb.dc-smarter.com/.well-known/acme-challenge/ApYjx0cd8jRYnJIIR1mdtJazagUyCW-TMKOuEiOdjis: 404

Hint: The Certificate Authority failed to verify the temporary nginx configuration changes made by Certbot. Ensure the listed domains point to this nginx server and that it is accessible from the internet.

My web server is (include version): nginx version: nginx/1.18.0 (Ubuntu)

The operating system my web server runs on is (include version):
Canonical:0001-com-ubuntu-server-focal:20_04-lts-gen2:latest

My hosting provider, if applicable, is:

I can login to a root shell on my machine (yes or no, or I don't know):
yes

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):
certbot 2.9.0

The most interesting thing is, when i execute only and initially this command:
certbot --nginx --non-interactive --agree-tos -m support@dc-smarter.com --domains test.nb.dc-smarter.com --cert-name test.nb.dc-smarter.com

it fails, but if i execute first:

1. certbot certonly --domain test.nb.dc-smarter.com --cert-name test.nb.dc-smarter.com --key-type rsa --authenticator standalone --pre-hook "systemctl stop nginx" --post-hook "systemctl start nginx" --non-interactive --agree-tos -m support@dc-smarter.com

then

2. certbot --nginx --non-interactive --agree-tos -m support@dc-smarter.com --domains test.nb.dc-smarter.com --cert-name test.nb.dc-smarter.com

all is working.

I would like to know why and how i can achieve that nginx plugin will work without the previous standalone command.

Thanks and regards

Timo

[Osiris]

@lono0778

I would like to know why and how i can achieve that nginx plugin will work without the previous standalone command.

Valid authorizations are cached for 30 days, so it looks like the second time you run the --nginx command, it just re-uses the previous valid authz.

[lono0778]

Hi Osiris,

thank you for fast response on this. Is there any way to get some valid authz in place without the standalone command in between?

What i need to say is, that nginx is configured to redirect http to https and i am wondering if this could cause the --nginx command failing, but should not, according to:

Challenge Types - Let's Encrypt

" It only accepts redirects to “http:” or “https:”, and only to ports 80 or 443".

So, im actually clueless here...

[rg305]

Please show the nginx config to better understand the problem therein, with:

nginx -T

From the outside looking in, I don't see a problem:
HTTP redirects to HTTPS:

curl -Ii http://test.nb.dc-smarter.com/.well-known/acme-challenge/Test_File-1234
HTTP/1.1 301 Moved Permanently
Server: nginx/1.18.0 (Ubuntu)
Date: Tue, 27 Feb 2024 13:16:15 GMT
Content-Type: text/html
Content-Length: 178
Connection: keep-alive
Location: https://test.nb.dc-smarter.com/.well-known/acme-challenge/Test_File-1234

HTTPS request returns 404 as expected:

curl -Ii https://test.nb.dc-smarter.com/.well-known/acme-challenge/Test_File-1234
HTTP/1.1 404 Not Found
Server: nginx/1.18.0 (Ubuntu)
Date: Tue, 27 Feb 2024 13:16:22 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 11112
Connection: keep-alive
Vary: HX-Request, Cookie, origin
X-Request-ID: 8c1d6d71-59e3-4071-bcbc-3ec2de1d5e0d
X-Content-Type-Options: nosniff
Referrer-Policy: same-origin
Cross-Origin-Opener-Policy: same-origin
X-Frame-Options: SAMEORIGIN
Set-Cookie: csrftoken=0KQK6bYJVdSnM6Iq9ToESfOSwcq95OiP; expires=Tue, 25 Feb 2025 13:16:22 GMT; Max-Age=31449600; Path=/; SameSite=Lax

[lono0778]

@Osiris

--nginx

No problem at all currently, it works all es expected. The issue i would llike to get rid of, is the error i get when i try to get certificates installed by executing initially "certbot --nginx..." - command. This will fail, until i execute "certbot certonly..." - command. So, to get --nginx command working and config maintained by certbot, i need to execute the certonly command previously. I would like to avoid this, if possible...

[rg305]

You contradict yourself, with these opposing statements:

@lono0778

No problem at all currently, it works all es expected.

@lono0778

the error i get when i try to get certificates installed by executing initially "certbot --nginx..." - command. This will fail,

It either "works" or "it fails".
Which is it?

[lono0778]

sorry for the confusion:

when i execute initally this command:

certbot --nginx --non-interactive --agree-tos -m support@dc-smarter.com --domains test.nb.dc-smarter.com --cert-name test.nb.dc-smarter.com

it fails.

When i execute this comand first:

certbot certonly --domain test.nb.dc-smarter.com --cert-name test.nb.dc-smarter.com --key-type rsa --authenticator standalone --pre-hook "systemctl stop nginx" --post-hook "systemctl start nginx" --non-interactive --agree-tos -m support@dc-smarter.com

and then

certbot --nginx --non-interactive --agree-tos -m support@dc-smarter.com --domains test.nb.dc-smarter.com --cert-name test.nb.dc-smarter.com

It will be successful.

I am wondering if i can get it working without executing the standalone command first...

[rg305]

@lono0778

it fails.

Please show the logs for that failure.

[one step at a time]

[jvanasco]

Most people with nginx issues are due to certbot not being able to parse the config files or handle them correctly. That doesn't seem to be the case here - those errors are typically raised before authorization is attempted.

IMHOW, the issue is likely due to your nginx config for the http and https sections. if you can share as much of those blocks as possible, someone may see the issue.

If you can't get this config running, an alternative approach is to do the following:

1- configure nginx to proxypass the acme-challenge directory to an alternate port, like 8080
2- run the certbot certonly standalone option but bind it to the above identified port, e.g. --http-01-port=8080

That can be easier to setup and troubleshoot, because you just need to set up a test service on 8080 that can serve an acme challenge. Here is my fake server that I use to test this: peter_sslers/tools/fake_server.py at main · aptise/peter_sslers · GitHub

If you run that server on 8080, then you can work on the nginx configuration until you're able to correctly reach the acme-challenge paths on port 80 through the proxypass.

[lono0778]

Hello,

i simply followed the netbox documentation:

HTTP Server Installation

Option A: nginx

Begin by installing nginx:

sudo apt install -y nginx

Once nginx is installed, copy the nginx configuration file provided by NetBox to /etc/nginx/sites-available/netbox. Be sure to replace netbox.example.com with the domain name or IP address of your installation. (This should match the value configured for ALLOWED_HOSTS in configuration.py.)

sudo cp /opt/netbox/contrib/nginx.conf /etc/nginx/sites-available/netbox

Then, delete /etc/nginx/sites-enabled/default and create a symlink in the sites-enabled directory to the configuration file you just created.

sudo rm /etc/nginx/sites-enabled/default
sudo ln -s /etc/nginx/sites-available/netbox /etc/nginx/sites-enabled/netbox

Finally, restart the nginx service to use the new configuration.

sudo systemctl restart nginx

The config looks like:

server {
listen [::]:443 ssl ipv6only=off;

# CHANGE THIS TO YOUR SERVER'S NAME
server_name test.nb.dc-smarter.com;

ssl_certificate /etc/ssl/certs/netbox.crt;
ssl_certificate_key /etc/ssl/private/netbox.key;

client_max_body_size 25m;

location /static/ {
    alias /opt/netbox/netbox/static/;
}

location / {
    proxy_pass http://127.0.0.1:8001;
    proxy_set_header X-Forwarded-Host $http_host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-Proto $scheme;
}

}

server {
# Redirect HTTP traffic to HTTPS
listen [::]:80 ipv6only=off;
server_name _;
return 301 https://$host$request_uri;
}

[lono0778]

i gave up and finally ended in executing always both commands in sequence, as this will work for me.

1. certbot certonly --domain image.example.com --cert-name image.example.com --key-type rsa --authenticator standalone --pre-hook "systemctl stop nginx" --post-hook "systemctl start nginx" --non-interactive --agree-tos -m support@example.com

2. certbot --nginx --non-interactive --agree-tos -m support@example.com --domains image.example.com --cert-name image.example.com

And all is fine...

[MikeMcQ]

@lono0778

i gave up and finally ended in executing always both commands in sequence

That is highly unusual to need to run certonly with pre/post hooks before --nginx

When you tried running command #2 with --nginx earlier was nginx running?

There is an unusual bug that if nginx is not running before certbot --nginx is used then Certbot will start a copy of nginx that is not compatible with systemctl. That can lead to odd problems.

The fix is just to be sure nginx is running.

Your #1 command has a post-hook so leaves nginx running in a good state before trying command #2. Maybe this explains why that "fixes" it?