Can't renew certificate nginx reverse-proxy

Help
1

My domain is: cbe30c15fb7f.sn.mynetname.net

I ran this command: certbot renew

It produced this output:

Saving debug log to /var/log/letsencrypt/letsencrypt.log

Processing /etc/letsencrypt/renewal/cbe30c15fb7f.sn.mynetname.net.conf

Renewing an existing certificate for cbe30c15fb7f.sn.mynetname.net

Certbot failed to authenticate some domains (authenticator: nginx). The Certificate Authority reported these problems:
Domain: cbe30c15fb7f.sn.mynetname.net
Type: unauthorized
Detail: 188.243.62.66: Invalid response from http://cbe30c15fb7f.sn.mynetname.net/.well-known/acme-challenge/2NAwvuIJlZLTnNNXG-YkwSyI1NEt0zx7UnvObysLwyg: 404

Hint: The Certificate Authority failed to verify the temporary nginx configuration changes made by Certbot. Ensure the listed domains point to this nginx server and that it is accessible from the internet.

Failed to renew certificate cbe30c15fb7f.sn.mynetname.net with error: Some challenges have failed.

All renewals failed. The following certificates could not be renewed:
/etc/letsencrypt/live/cbe30c15fb7f.sn.mynetname.net/fullchain.pem (failure)

1 renew failure(s), 0 parse failure(s)
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.

My web server is (include version): nginx/1.20.1

The operating system my web server runs on is (include version): Linux centos7.loc 3.10.0-1160.88.1.el7.x86_64

My hosting provider, if applicable, is: no

I can login to a root shell on my machine (yes or no, or I don't know): yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): 2.10.0

Hi community! I want to adjust the automatic renewal certificate for my home server.
I have server centos and I run there docker container for my webapp.
This web application (container) uses HTTP on the TCP port 2342. On the host system, I configured NGINX as a reverse proxy.
I use the free DDNS service provided to Mikrotik users.

You can see attachments for detail
letsencrypt.log.5.txt (25.4 KB)
nginx.conf.txt (2.3 KB)
cbe30c15fb7f.sn.mynetname.txt (3.6 KB)
...
Please help.

1 Like
2

Hello @nitro, welcome to the Let's Encrypt community. :slightly_smiling_face:

Try running sudo certbot renew instead.

2 Likes
3 
    server {
        listen       80;
        listen       [::]:80;
        server_name  _;
        root         /usr/share/nginx/html;

        # Load configuration files for the default server block.
        include /etc/nginx/default.d/*.conf;     <<<<<<<<<<<<<<<<<<<< ?????

        error_page 404 /404.html;
        location = /404.html {
        }

        error_page 500 502 503 504 /50x.html;
        location = /50x.html {
        }
    }

Can we see what might be loaded from there?;
ls -l /etc/nginx/default.d/*.conf

3 Likes
4

Certbot wouldn't have gotten so far without being run with the proper rights, so OP either already used sudo or ran Certbot as root.

2 Likes
6

It seems like you converted the HTTP server block to HTTPS:

server {
    #listen 80; # If you really need HTTP (unsecure) remove the "#" on the beginning. Not recommended!
    # listen [::]:80; # HTTP IPv6

    listen 443 ssl http2; # Listen on port 443 and enable ssl and HTTP/2
    listen [::]:443 ssl http2; # Same for IPv6

    # Put your domain name in here.
    server_name  cbe30c15fb7f.sn.mynetname.net;

But the renewal still needs an HTTP server block to handle the ACME challenge requests.
Let's confirm by reviewing the contents of the renewal config file:
cat /etc/letsencrypt/renewal/cbe30c15fb7f.sn.mynetname.net.conf

3 Likes
7

The directory is empty

[root@centos7 ~]# ls -l /etc/nginx/default.d/
total 0
[root@centos7 ~]#

1 Like
8

Here you are

[root@centos7 ~]# cat /etc/letsencrypt/renewal/cbe30c15fb7f.sn.mynetname.net.conf
# renew_before_expiry = 30 days
version = 2.8.0
archive_dir = /etc/letsencrypt/archive/cbe30c15fb7f.sn.mynetname.net
cert = /etc/letsencrypt/live/cbe30c15fb7f.sn.mynetname.net/cert.pem
privkey = /etc/letsencrypt/live/cbe30c15fb7f.sn.mynetname.net/privkey.pem
chain = /etc/letsencrypt/live/cbe30c15fb7f.sn.mynetname.net/chain.pem
fullchain = /etc/letsencrypt/live/cbe30c15fb7f.sn.mynetname.net/fullchain.pem

# Options used in the renewal process
[renewalparams]
authenticator = nginx
installer = nginx
account = 17386d133bb09d6efba259fbfaf13b4e
server = https://acme-v02.api.letsencrypt.org/directory
key_type = rsa
9

For HTTP server block you can find out config in main nginx conf file nginx.conf[.txt]

11

It is difficult to break the nginx plugin [not impossible].
And you may have succeeded at that by converting the vhost that covers that name from HTTP to HTTPS (only).

That vhost does not use the name requested by the cert renewal:
server_name _;

I think you can fix this problem in one of three ways:

  • adjust that "default" vhost server_name to match/include the cert name
    [not a great idea]

  • replace the converted HTTP vhost server block back with (enough of) what was there before it was modified to HTTPS
    [having both (HTTP and HTTPS) vhosts for that name should resolve the issue]

  • replace the authentication method from nignx to webroot
    [without a dedicated HTTP vhost, this solution is at the mercy of the settings in the "default" vhost]

So... my recommendation is the middle choice: To use a dedicated HTTP vhost for the name(s) in the cert.

2 Likes
12

Am I reading this right for the middle suggestion?
I should comment out the HTTP section in the main nginx conf file, then I should add the HTTP section to the /etc/nginx/sites-enabled/cbe30c15fb7f.sn.mynetname.net configuration. Right?

13

You don't need to do anything to the default HTTP server block in the main config file.
You can add a vhost server block within just about any existing file OR create a new file for it.
[where you put the new server block is not relevant - what is relevant is the content of that server block]

3 Likes
14

Like this?
/etc/nginx/sites-enabled/cbe30c15fb7f.sn.mynetname.net

server {
        listen 80;
        #listen [::]:80;
        server_name  cbe30c15fb7f.sn.mynetname.net;        
    }

server {
    #listen 80; # If you really need HTTP (unsecure) remove the "#" on the beginning. Not recommended!
    # listen [::]:80; # HTTP IPv6

    listen 443 ssl http2; # Listen on port 443 and enable ssl and HTTP/2
    listen [::]:443 ssl http2; # Same for IPv6

    # Put your domain name in here.
    server_name  cbe30c15fb7f.sn.mynetname.net;

    # - - - - - - - - - -
    # SSL security
    # - - - - - - - - - -
    ssl_certificate /etc/letsencrypt/live/cbe30c15fb7f.sn.mynetname.net/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/cbe30c15fb7f.sn.mynetname.net/privkey.pem; # managed by Certbot

    # Since the PP API is also used on Android, we have to keep TLS1.2 in here for a while.
    # A lot of the older Android devices do not support TLS1.3 yet :/
    ssl_protocols            TLSv1.3;
...
17

Well... almost.
I'd add two things:

  • Ideally it would include a root location
    [preferably a new and unique location to be used only for ACME renewals]
    Like: root /acme-challenges;
    OR: root /var/tmp/acme-challenges;
    [remember to create the folder too (with the proper permissions)]

  • and you might also want to redirect all the non ACME challenge requests to HTTPS.

2 Likes
18

Thanks!
Which directory should I specify in the HTTP server section If my web service (who answers for the requests from the Internet) containerized? Does it matter?

About "redirect all the non ACME challenge", could you please give me an example?

20

The proxy answers HTTP and HTTPS.
The HTTPS requests are proxied to the container [via HTTP port 2342].
The HTTP requests are answered locally by the proxy; Currently by the default server block in the main config file.
Once you restart/reload nginx, it will have an HTTP server block for that name and it will do what that block requires.
So...

[if you mean root statement]
I gave two possible examples of locations:

As for

You can add two locations blocks [to cover each possibility]:

        location / {
            return 301 https://$host$request_uri; 
        }
        location /.well-known/acme-challenge {
            root /var/tmp/acme-challenges;
            try_files $uri =404;
        }

Remember to create whatever folder location you choose.

3 Likes
21

Unfortunately,
cbe30c15fb7f.sn.mynetname.net.txt (3.9 KB)
letsencrypt.log.txt (27.5 KB)
same result...

[root@centos7 ~]# ls -l /var/tmp/
total 0
drwxr-xr-x. 2 root root  6 May 10 10:34 acme-challenges

new config of the nginx conf file (/etc/nginx/sites-enabled/cbe30c15fb7f.sn.mynetname.net) in attach, please see
new logs also in attach
P.S. after edited nginx conf file I reload ngix by command "systemctl reload nginx.service"

1 Like
22

Despite the failure, we are making progress.
The added server block is being used.

Generic HTTP requests are being redirected to HTTPS:

curl -Ii cbe30c15fb7f.sn.mynetname.net
HTTP/1.1 301 Moved Permanently
Server: nginx/1.20.1
Date: Fri, 10 May 2024 07:56:25 GMT
Content-Type: text/html
Content-Length: 169
Connection: keep-alive
Location: https://cbe30c15fb7f.sn.mynetname.net/

ACME challenge requests are being looked for locally:
[in this case not found (404) is expected - because that file name doesn't exists]

curl -Ii cbe30c15fb7f.sn.mynetname.net/.well-known/acme-challenge/Test_File-1234
HTTP/1.1 404 Not Found
Server: nginx/1.20.1
Date: Fri, 10 May 2024 07:56:45 GMT
Content-Type: text/html
Content-Length: 153
Connection: keep-alive
2 Likes
25

Something is preventing the nginx plugin from working correctly.
Let's try switching to webroot.
Try:

certbot certonly \
--webroot -w /var/tmp/acme-challenges \
--cert-name cbe30c15fb7f.sn.mynetname.net
2 Likes
26

[root@centos7 ~]# certbot certonly
--webroot -w /var/tmp/acme-challenges
--cert-name cbe30c15fb7f.sn.mynetname.net
Saving debug log to /var/log/letsencrypt/letsencrypt.log

An RSA certificate named cbe30c15fb7f.sn.mynetname.net already exists. Do you
want to update its key type to ECDSA?

(U)pdate key type/(K)eep existing key type: K
Renewing an existing certificate for cbe30c15fb7f.sn.mynetname.net

Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
Domain: cbe30c15fb7f.sn.mynetname.net
Type: unauthorized
Detail: 188.243.62.66: Invalid response from http://cbe30c15fb7f.sn.mynetname.net/.well-known/acme-challenge/Ky012U5rONOtEdAeLhj2B-LMKDzIaCz7hbcYYDUFlMI: 404

Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded from the internet.

Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.
letsencrypt.log.1.txt (18.8 KB)

27

Very strange!

Let's try placing a file in that location:
echo "test" > /var/tmp/acme-challenges/Test_File-1234

Then it should be accessible via:
http://cbe30c15fb7f.sn.mynetname.net/.well-known/acme-challenge/Test_File-1234

If not, then we need to see the web server logs [access and error].
If yes, then... we need to... think about what just happened [this should NOT be the case].

3 Likes