Certbot Renew Failure: Invalid Response

Help
1

My domain is: saferesponsetoolkit.com.au

I ran this command: sudo certbot renew

It produced this output:

Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/saferesponsetoolkit.com.au.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Renewing an existing certificate for saferesponsetoolkit.com.au and 2 more domains

Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
  Domain: saferesponsetoolkit.com
  Type:   unauthorized
  Detail: 3.27.82.110: Invalid response from http://saferesponsetoolkit.com/.well-known/acme-challenge/xiA-VY4nQ4RoHc_lFTmJ8KqZ_eT5CTp47sLTNLBbQFc: 404

Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded from the internet.

Failed to renew certificate saferesponsetoolkit.com.au with error: Some challenges have failed.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
All renewals failed. The following certificates could not be renewed:
  /etc/letsencrypt/live/saferesponsetoolkit.com.au/fullchain.pem (failure)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Hook 'post-hook' ran with output:
 Testing NGINX configuration..
 Reloading NGINX to make the new certificates available
 Reloading nginx configuration (via systemctl): nginx.service.
Hook 'post-hook' ran with error output:
 nginx: [warn] the "listen ... http2" directive is deprecated, use the "http2" directive instead in /etc/nginx/ploi/saferesponsetoolkit.com.au/before/ssl-redirect.conf:11
 nginx: [warn] the "listen ... http2" directive is deprecated, use the "http2" directive instead in /etc/nginx/ploi/saferesponsetoolkit.com.au/before/ssl-redirect.conf:12
 nginx: [warn] the "listen ... http2" directive is deprecated, use the "http2" directive instead in /etc/nginx/ssl/saferesponsetoolkit.com.au:1
 nginx: [warn] the "listen ... http2" directive is deprecated, use the "http2" directive instead in /etc/nginx/ssl/saferesponsetoolkit.com.au:2
 nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
 nginx: configuration file /etc/nginx/nginx.conf test is successful
1 renew failure(s), 0 parse failure(s)
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.

My web server is (include version): nginx/1.26.1

The operating system my web server runs on is (include version): Ubuntu 22.04

My hosting provider, if applicable, is: AWS EC2

I can login to a root shell on my machine (yes or no, or I don't know): yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): certbot 3.1.0

Thank you so much for your help!

2

Welcome @MaggieMeow

The "404" error code is an HTTP Not Found. With webroot authenticator that usually means the webroot-path used when you first got that cert is not the same as theroot folder in your nginx anymore.

Would you show contents of this file

/etc/letsencrypt/renewal/saferesponsetoolkit.com.au.conf

And, show the server block for these domains from your nginx conf (probably in /sites/enabled)

2 Likes
3

Hi @MikeMcQ thanks for tha reply

here's the content of /etc/letsencrypt/renewal/saferesponsetoolkit.com.au.conf

ubuntu@the-stop-campaign:~$ cat /etc/letsencrypt/renewal/saferesponsetoolkit.com.au.conf
# renew_before_expiry = 30 days
version = 2.11.0
archive_dir = /etc/letsencrypt/archive/saferesponsetoolkit.com.au
cert = /etc/letsencrypt/live/saferesponsetoolkit.com.au/cert.pem
privkey = /etc/letsencrypt/live/saferesponsetoolkit.com.au/privkey.pem
chain = /etc/letsencrypt/live/saferesponsetoolkit.com.au/chain.pem
fullchain = /etc/letsencrypt/live/saferesponsetoolkit.com.au/fullchain.pem

# Options used in the renewal process
[renewalparams]
account = 7a55a2854dd774ff5a929a2fece853d2
authenticator = webroot
webroot_path = /home/ploi/saferesponsetoolkit.com.au/public
server = https://acme-v02.api.letsencrypt.org/directory
key_type = ecdsa
[[webroot_map]]
saferesponsetoolkit.com = /home/ploi/saferesponsetoolkit.com.au/public
saferesponsetoolkit.com.au = /home/ploi/saferesponsetoolkit.com.au/public

here's my nginx config

server {
    server_name _;
    listen       80  default_server;
    return       404;
}

server {
    listen 443 ssl;
    server_name _;
    ssl_certificate /etc/nginx/ploi/default/nginx.crt;
    ssl_certificate_key /etc/nginx/ploi/default/nginx.key;
    return       404;
}ubuntu@the-stop-campaign:/etc/nginx/sites-available$ cat 
catch-all                   default                     saferesponsetoolkit.com.au  
ubuntu@the-stop-campaign:/etc/nginx/sites-available$ cat 
catch-all                   default                     saferesponsetoolkit.com.au  
ubuntu@the-stop-campaign:/etc/nginx/sites-available$ cat saferesponsetoolkit.com.au 
# Ploi Webserver Configuration, do not remove!
include /etc/nginx/ploi/saferesponsetoolkit.com.au/before/*;

server {
    #listen 80;
    #listen [::]:80;

    root /home/ploi/saferesponsetoolkit.com.au/public;
    server_name saferesponsetoolkit.com.au saferesponsetoolkit.com 3.27.82.110;    

    include /etc/nginx/ssl/saferesponsetoolkit.com.au;

    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers '[redacted]';
    ssl_prefer_server_ciphers on;
    ssl_dhparam /etc/nginx/dhparams.pem;

    index index.php index.html;

    add_header X-Frame-Options "SAMEORIGIN";
    add_header X-XSS-Protection "1; mode=block";
    add_header X-Content-Type-Options "nosniff";

    charset utf-8;

    # Ploi Configuration, do not remove!
    include /etc/nginx/ploi/saferesponsetoolkit.com.au/server/*;

    location / {
        try_files $uri $uri/ /index.php?$query_string;
    }

    access_log off;
    error_log  /var/log/nginx/saferesponsetoolkit.com.au-error.log error;

    location = /favicon.ico { access_log off; log_not_found off; }
    location = /robots.txt  { access_log off; log_not_found off; }

    error_page 404 /index.php;

    location ~ \.php$ {
        try_files $uri /index.php =404;
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
        fastcgi_pass unix:/run/php/php8.3-fpm.sock;
        fastcgi_buffers 16 16k;
        fastcgi_index index.php;
        fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name;
        fastcgi_param DOCUMENT_ROOT $realpath_root;
        include fastcgi_params;
    }

     location ~ /\.(?!well-known).* {
         deny all;
     }
}

# Ploi Webserver Configuration, do not remove!
include /etc/nginx/ploi/saferesponsetoolkit.com.au/after/*;

I've made sure a manually created file under .well-known/acme-challenge can be accessed with HTTP 200.

Thanks for the help!

1 Like
4

Above is the webroot-path you used when getting the cert originally. Certbot uses that same path to place the challenge token file for its renewal.

Yet, the below server block is the only one listening on port 80. There is no root statement in this server block so uses the default nginx value. I am certain it is not the same as above :slight_smile:

What was the full path you used for that file and what URL did you try. It needs to be found from an HTTP://(domain)/.well-known/acme-challange/(token file) format of URL (not HTTPS or other path components)

What happened to the server block for port 80 for these domains? While it is technically possible to use a default server block for this purpose that is usually not done.

2 Likes
5

There's actually another server block below the first one

server {
    #listen 80;
    #listen [::]:80;

    root /home/ploi/saferesponsetoolkit.com.au/public;
    server_name saferesponsetoolkit.com.au saferesponsetoolkit.com 3.27.82.110;    

......

What was the full path you used for that file and what URL did you try. It needs to be found from an HTTP://(domain)/.well-known/acme-challange/(token file) format of URL (not HTTPS or other path components)

I manually created /home/ploi/saferesponsetoolkit.com.au/public/.well-known/acme-challenge/testfile
and you can see the test file i created is accessible here: http://saferesponsetoolkit.com.au/.well-known/acme-challenge/testfile

Before I manually created that file, the ./well-known/acme-challenge/ directory did not exist. I suspected that certbot somehow did not create the file at the correct directory. What do you think?

6

Yes, but it does not appear to be listening on port 80. Note those listen statements are commented out. Besides, that server block is setup for HTTPS (see the ssl certificate and related config). Usually best to not mix port 80 and 443 in the same server block. I didn't see any listen statements for port 443 in that block. I assume they were in the include for that.

When I try your test file I get redirected from HTTP to HTTPS. While that is supported by the HTTP Challenge I don't see any redirects in the nginx server config you showed. In fact, your only port 80 server block replies with 404 to all requests. Which is what we saw in your cert request.

We need to understand why your nginx responds as it does. It is not credible that Certbot would not place the challenge token file in the folder from its renewal config file. We would be flooded with complaints if such a popular ACME Client had such a fundamental problem. There is definitely something happening with your nginx that is not explained by what you have shown so far.

Would you post the full contents of this

sudo nginx -T

An upper case T is essential. The output will be long.

2 Likes
7

I uncommented #listen 80; #listen [::]:80; and the renewal was successful.
Thank you so much for your kind help!

3 Likes
8

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.