Can't renew certificate nginx reverse-proxy

Help
28

Hi! I created file...

[root@centos7 ~]# ls -l /var/tmp/acme-challenges/
total 8
-rw-r--r--. 1 root root 3650 May 12 18:00 404.html
-rwxr-xr-x. 1 root root    5 May 12 12:39 Test_File-1234

Access seems to be granted, please check...
I checked the URL from the IP address 80.254.50.127 by command
curl -Ii cbe30c15fb7f.sn.mynetname.net/.well-known/acme-challenge/Test_File-1234

HTTP/1.1 404 Not Found
Server: nginx/1.20.1
Date: Sun, 12 May 2024 14:49:16 GMT
Content-Type: text/html
Content-Length: 153
Connection: keep-alive

Tail of access.log

# grep "80.254.50.127" /var/log/nginx/access.log
80.254.50.127 - - [12/May/2024:12:43:20 +0300] "HEAD /.well-known/acme-challenge/Test_File-1234 HTTP/1.1" 404 0 "-" "curl/7.74.0" "-"
80.254.50.127 - - [12/May/2024:12:43:44 +0300] "HEAD /acme-challenge/Test_File-1234 HTTP/1.1" 301 0 "-" "curl/7.74.0" "-"
80.254.50.127 - - [12/May/2024:12:45:48 +0300] "HEAD /.well-known/acme-challenge/Test_File-1234 HTTP/1.1" 404 0 "-" "curl/7.74.0" "-"
80.254.50.127 - - [12/May/2024:12:46:05 +0300] "HEAD /acme-challenge/Test_File-1234 HTTP/1.1" 301 0 "-" "curl/7.74.0" "-"
80.254.50.127 - - [12/May/2024:12:48:28 +0300] "HEAD /.well-known/acme-challenge/Test_File-1234 HTTP/1.1" 404 0 "-" "curl/7.74.0" "-"
80.254.50.127 - - [12/May/2024:17:49:16 +0300] "HEAD /.well-known/acme-challenge/Test_File-1234 HTTP/1.1" 404 0 "-" "curl/7.74.0" "-"
80.254.50.127 - - [12/May/2024:17:50:49 +0300] "GET / HTTP/1.1" 301 169 "-" "curl/7.74.0" "-"
80.254.50.127 - - [12/May/2024:17:51:12 +0300] "GET /.well-known/acme-challenge/Test_File-1234 HTTP/1.1" 404 153 "-" "curl/7.74.0" "-"
80.254.50.127 - - [12/May/2024:18:00:28 +0300] "GET /.well-known/acme-challenge/404.html HTTP/1.1" 404 153 "-" "curl/7.74.0" "-"

Error log
tail /var/log/nginx/error.log

2024/05/12 13:31:11 [error] 13387#13387: OCSP response not successful (6: unauthorized) while requesting certificate status, responder: r3.o.lencr.org, peer: 95.101.75.42:80, certificate: "/etc/letsencrypt/live/cbe30c15fb7f.sn.mynetname.net/fullchain.pem"
2024/05/12 14:15:18 [error] 13386#13386: OCSP response not successful (6: unauthorized) while requesting certificate status, responder: r3.o.lencr.org, peer: 95.101.75.47:80, certificate: "/etc/letsencrypt/live/cbe30c15fb7f.sn.mynetname.net/fullchain.pem"
2024/05/12 14:15:34 [error] 13387#13387: OCSP response not successful (6: unauthorized) while requesting certificate status, responder: r3.o.lencr.org, peer: 2.16.21.67:80, certificate: "/etc/letsencrypt/live/cbe30c15fb7f.sn.mynetname.net/fullchain.pem"
2024/05/12 15:35:55 [error] 13386#13386: OCSP response not successful (6: unauthorized) while requesting certificate status, responder: r3.o.lencr.org, peer: 2.16.21.72:80, certificate: "/etc/letsencrypt/live/cbe30c15fb7f.sn.mynetname.net/fullchain.pem"
2024/05/12 15:35:56 [error] 13387#13387: OCSP response not successful (6: unauthorized) while requesting certificate status, responder: r3.o.lencr.org, peer: 184.51.252.197:80, certificate: "/etc/letsencrypt/live/cbe30c15fb7f.sn.mynetname.net/fullchain.pem"
2024/05/12 16:08:27 [error] 13387#13387: OCSP response not successful (6: unauthorized) while requesting certificate status, responder: r3.o.lencr.org, peer: 95.101.75.42:80, certificate: "/etc/letsencrypt/live/cbe30c15fb7f.sn.mynetname.net/fullchain.pem"
2024/05/12 16:08:33 [error] 13386#13386: OCSP response not successful (6: unauthorized) while requesting certificate status, responder: r3.o.lencr.org, peer: 80.239.138.161:80, certificate: "/etc/letsencrypt/live/cbe30c15fb7f.sn.mynetname.net/fullchain.pem"
2024/05/12 17:14:02 [error] 13387#13387: OCSP response not successful (6: unauthorized) while requesting certificate status, responder: r3.o.lencr.org, peer: 23.196.236.32:80, certificate: "/etc/letsencrypt/live/cbe30c15fb7f.sn.mynetname.net/fullchain.pem"
2024/05/12 17:41:12 [error] 13387#13387: OCSP response not successful (6: unauthorized) while requesting certificate status, responder: r3.o.lencr.org, peer: 184.51.252.188:80, certificate: "/etc/letsencrypt/live/cbe30c15fb7f.sn.mynetname.net/fullchain.pem"
2024/05/12 17:48:11 [notice] 31930#31930: signal process started
29

To me that would indicate that this section is doesn't get taken for
http://<YOUR_DOMAIN>/.well-known/acme-challenge/<TOKEN> .
In this example to be specific
http://cbe30c15fb7f.sn.mynetname.net/.well-known/acme-challenge/Test_File-1234

And I get the same results you showed

$ curl -Ii http://cbe30c15fb7f.sn.mynetname.net/.well-known/acme-challenge/Test_File-1234
HTTP/1.1 404 Not Found
Server: nginx/1.20.1
Date: Sun, 12 May 2024 15:26:27 GMT
Content-Type: text/html
Content-Length: 153
Connection: keep-alive
1 Like
30

I don't know what can I do next.
Probably the "location" section doesn't take correctly by nginx.
Because, if you try this URL (cbe30c15fb7f.sn.mynetname.net/Test_File-1234) it would be working correctly.

$ curl -Ii cbe30c15fb7f.sn.mynetname.net/Test_File-1234
HTTP/1.1 200 OK
Server: nginx/1.20.1
Date: Mon, 13 May 2024 06:23:12 GMT
Content-Type: application/octet-stream
Content-Length: 5
Last-Modified: Sun, 12 May 2024 16:01:08 GMT
Connection: keep-alive
ETag: "6640e7c4-5"
Accept-Ranges: bytes

It's default configuration. I placed server_name value in the main nginx conf file and I deleted HTTP server section in the vhost conf file.

    server {
        listen       80;
        #listen       [::]:80;
        #server_name  _;
        server_name  cbe30c15fb7f.sn.mynetname.net;
        root         /usr/share/nginx/html;

        # Load configuration files for the default server block.
        include /etc/nginx/default.d/*.conf;

        error_page 404 /404.html;
        location = /404.html {
        }

        error_page 500 502 503 504 /50x.html;
        location = /50x.html {
        }
    }

I tried to put "location" parameter to the nginx main conf file - but nginx still out error 404 as before.
Even though I use default nginx root directory (/usr/share/nginx/html)

31

Can you show us your entire active nginx config? You can do that by running this command and posting the result. Or redirect output to a file and upload it like you did earlier.

An upper case T is essential

sudo nginx -T >config.txt

My HTTP requests to your domain right now do not get the same response as you show in your most recent post. I get redirected to HTTPS but there is no redirect in your most recent server block.

1 Like
32

For general nginx information you might find nginx documentation and https://forum.nginx.org/ helpful.

33

Please remove the section:

        # Load configuration files for the default server block.
        include /etc/nginx/default.d/*.conf;

It does NOT belong there [within another server block].

2 Likes
34

Sorry for the late answer.
Config in attach.

# ls -l /usr/share/nginx/html
total 16
-rw-r--r--. 1 root root 3650 Nov 10  2022 404.html
-rw-r--r--. 1 root root 3693 Nov 10  2022 50x.html
lrwxrwxrwx. 1 root root   20 May 22  2023 en-US -> ../../doc/HTML/en-US
drwxr-xr-x. 2 root root   27 May 22  2023 icons
lrwxrwxrwx. 1 root root   18 May 22  2023 img -> ../../doc/HTML/img
lrwxrwxrwx. 1 root root   25 May 22  2023 index.html -> ../../doc/HTML/index.html
-rw-r--r--. 1 root root  368 Nov 10  2022 nginx-logo.png
lrwxrwxrwx. 1 root root   14 May 22  2023 poweredby.png -> nginx-logo.png
-rwxr-xr-x. 1 root root    5 May 12 19:01 Test_File-1234

My attempts

curl -Ii cbe30c15fb7f.sn.mynetname.net/acme-challenge/Test_File-1234
HTTP/1.1 **404 Not Found**
Server: nginx/1.20.1
Date: Wed, 15 May 2024 12:55:36 GMT
Content-Type: text/html
Content-Length: 3650
Connection: keep-alive
ETag: "636d2d22-e42"

curl -Ii cbe30c15fb7f.sn.mynetname.net/acme-challenge/404.html
HTTP/1.1 **404 Not Found**
Server: nginx/1.20.1
Date: Wed, 15 May 2024 12:55:52 GMT
Content-Type: text/html
Content-Length: 3650
Connection: keep-alive
ETag: "636d2d22-e42"

curl -Ii cbe30c15fb7f.sn.mynetname.net/404.html
HTTP/1.1 **200 OK**
Server: nginx/1.20.1
Date: Wed, 15 May 2024 12:56:02 GMT
Content-Type: text/html
Content-Length: 3650
Last-Modified: Thu, 10 Nov 2022 16:56:02 GMT
Connection: keep-alive
ETag: "636d2d22-e42"
Accept-Ranges: bytes

curl -Ii cbe30c15fb7f.sn.mynetname.net/Test_File-1234
HTTP/1.1 **200 OK**
Server: nginx/1.20.1
Date: Wed, 15 May 2024 12:56:12 GMT
Content-Type: application/octet-stream
Content-Length: 5
Last-Modified: Sun, 12 May 2024 16:01:08 GMT
Connection: keep-alive
ETag: "6640e7c4-5"
Accept-Ranges: bytes

config.txt (12.2 KB)

35

Hi! I removed... but not helped...

36

What does this do?

sudo certbot certonly --dry-run --webroot -w /usr/share/nginx/html -d cbe30c15fb7f.sn.mynetname.net
3 Likes
37

sudo certbot certonly --dry-run --webroot -w /usr/share/nginx/html -d cbe30c15fb7f.sn.mynetname.net

Saving debug log to /var/log/letsencrypt/letsencrypt.log

An RSA certificate named cbe30c15fb7f.sn.mynetname.net already exists. Do you
want to update its key type to ECDSA?

(U)pdate key type/(K)eep existing key type: K
Simulating renewal of an existing certificate for cbe30c15fb7f.sn.mynetname.net
The dry run was successful.

38

Okay good. What is contents of this file

/etc/letsencrypt/renewal/cbe30c15fb7f.sn.mynetname.net.conf
3 Likes
39 
# cat /etc/letsencrypt/renewal/cbe30c15fb7f.sn.mynetname.net.conf
# renew_before_expiry = 30 days
version = 2.8.0
archive_dir = /etc/letsencrypt/archive/cbe30c15fb7f.sn.mynetname.net
cert = /etc/letsencrypt/live/cbe30c15fb7f.sn.mynetname.net/cert.pem
privkey = /etc/letsencrypt/live/cbe30c15fb7f.sn.mynetname.net/privkey.pem
chain = /etc/letsencrypt/live/cbe30c15fb7f.sn.mynetname.net/chain.pem
fullchain = /etc/letsencrypt/live/cbe30c15fb7f.sn.mynetname.net/fullchain.pem

# Options used in the renewal process
[renewalparams]
authenticator = nginx
installer = nginx
account = 17386d133bb09d6efba259fbfaf13b4e
server = https://acme-v02.api.letsencrypt.org/directory
key_type = rsa
40

Okay. What does this do. Sorry if this is repeating stuff but just want fresh results after various changes.

sudo certbot certonly --dry-run --nginx -d cbe30c15fb7f.sn.mynetname.net
3 Likes
41

Don't worry!) I want to solve the problem

# sudo certbot certonly --dry-run --nginx -d cbe30c15fb7f.sn.mynetname.net
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
An RSA certificate named cbe30c15fb7f.sn.mynetname.net already exists. Do you
want to update its key type to ECDSA?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(U)pdate key type/(K)eep existing key type: K
Simulating renewal of an existing certificate for cbe30c15fb7f.sn.mynetname.net

Certbot failed to authenticate some domains (authenticator: nginx). The Certificate Authority reported these problems:
  Domain: cbe30c15fb7f.sn.mynetname.net
  Type:   unauthorized
  Detail: 188.243.62.66: Invalid response from http://cbe30c15fb7f.sn.mynetname.net/.well-known/acme-challenge/TDfJ-nJJDVtyCb7TUNydN0UPq-t4WHrn9J8R6njaed0: 404

Hint: The Certificate Authority failed to verify the temporary nginx configuration changes made by Certbot. Ensure the listed domains point to this nginx server and that it is accessible from the internet.

Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.
42

Well, the quickest way is to switch to --webroot method from --nginx. Usually --nginx works very well and I do not see anything in your nginx config that would cause trouble.

Issue the webroot command from earlier but without --dry-run to get the cert and reset the renewal config file

Also, do you really need RSA cert? ECDSA is now the default with Certbot and has been for some time.

sudo certbot certonly --webroot -w /usr/share/nginx/html -d cbe30c15fb7f.sn.mynetname.net

Show result of above and then we'll test the renew. Let's do just one step at a time

4 Likes
43 
# sudo certbot certonly --webroot -w /usr/share/nginx/html -d cbe30c15fb7f.sn.mynetname.net
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
An RSA certificate named cbe30c15fb7f.sn.mynetname.net already exists. Do you
want to update its key type to ECDSA?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(U)pdate key type/(K)eep existing key type: U
Renewing an existing certificate for cbe30c15fb7f.sn.mynetname.net

Successfully received certificate.
Certificate is saved at: /etc/letsencrypt/live/cbe30c15fb7f.sn.mynetname.net/fullchain.pem
Key is saved at:         /etc/letsencrypt/live/cbe30c15fb7f.sn.mynetname.net/privkey.pem
This certificate expires on 2024-08-13.
These files will be updated when the certificate renews.
Certbot has set up a scheduled task to automatically renew this certificate in the background.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
If you like Certbot, please consider supporting our work by:
 * Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
 * Donating to EFF:                    https://eff.org/donate-le
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
44

Excellent. Now reload nginx and you should be using the new cert

I do this on Ubuntu but use service or whatever on your system. A restart is not required just a reload

sudo systemctl reload nginx
3 Likes
45

Thank you so much!
Should I configure something else to make the next certificate reissue fully automatic?

1 Like
46

Excellent. I see nginx using the new cert.

Two more steps. Testing renew and then adding a deploy-hook to reload nginx after get fresh cert. I should have added that to earlier command but we can do it easily

Let's test renew. what does this do

sudo certbot renew --dry-run
3 Likes
47

Successfully!

# sudo certbot renew --dry-run
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/cbe30c15fb7f.sn.mynetname.net.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Simulating renewal of an existing certificate for cbe30c15fb7f.sn.mynetname.net

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Congratulations, all simulated renewals succeeded:
  /etc/letsencrypt/live/cbe30c15fb7f.sn.mynetname.net/fullchain.pem (success)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -