Certbot Renew Error - Invalid Response 404 during secondary validation

Help
1

My domain is: pzidgrabber.com

I ran this command: certbot renew --dry-run -v

It produced this output:

Processing /etc/letsencrypt/renewal/pzidgrabber.com.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Certificate is due for renewal, auto-renewing...
Plugins selected: Authenticator nginx, Installer nginx
Simulating renewal of an existing certificate for pzidgrabber.com and 2 more domains
Performing the following challenges:
http-01 challenge for api.pzidgrabber.com
http-01 challenge for pzidgrabber.com
http-01 challenge for www.pzidgrabber.com
Waiting for verification...
Challenge failed for domain api.pzidgrabber.com
http-01 challenge for api.pzidgrabber.com

Certbot failed to authenticate some domains (authenticator: nginx). The Certificate Authority reported these problems:
  Domain: api.pzidgrabber.com
  Type:   unauthorized
  Detail: 2600:3c02::f03c:93ff:fe78:9ff6: Invalid response from http://api.pzidgrabber.com/.well-known/acme-challenge/LokhR1JOjvWDHveKrZSGnXgqhzEMsircKEVEHtil2ek: 404

Hint: The Certificate Authority failed to verify the temporary nginx configuration changes made by Certbot. Ensure the listed domains point to this nginx server and that it is accessible from the internet.

Cleaning up challenges
Failed to renew certificate pzidgrabber.com with error: Some challenges have failed.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
All simulated renewals failed. The following certificates could not be renewed:
  /etc/letsencrypt/live/pzidgrabber.com/fullchain.pem (failure)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1 renew failure(s), 0 parse failure(s)
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.
root@localhost:/etc/nginx/conf.d# nano pzidgrabber.com.conf
root@localhost:/etc/nginx/conf.d# nano pzidgrabber.com.conf
root@localhost:/etc/nginx/conf.d# sudo service nginx restart
root@localhost:/etc/nginx/conf.d# certbot renew --dry-run -v
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/pzidgrabber.com.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Certificate is due for renewal, auto-renewing...
Plugins selected: Authenticator nginx, Installer nginx
Simulating renewal of an existing certificate for pzidgrabber.com and 2 more domains
Performing the following challenges:
http-01 challenge for api.pzidgrabber.com
http-01 challenge for pzidgrabber.com
http-01 challenge for www.pzidgrabber.com
Waiting for verification...
Challenge failed for domain api.pzidgrabber.com
http-01 challenge for api.pzidgrabber.com

Certbot failed to authenticate some domains (authenticator: nginx). The Certificate Authority reported these problems:
  Domain: api.pzidgrabber.com
  Type:   unauthorized
  Detail: During secondary validation: 2600:3c02::f03c:93ff:fe78:9ff6: Invalid response from http://api.pzidgrabber.com/.well-known/acme-challenge/XX6OLOmaXzZV_EzjGBBy9k4ep2SfUF68aZwSm6E01ag: 404

Hint: The Certificate Authority failed to verify the temporary nginx configuration changes made by Certbot. Ensure the listed domains point to this nginx server and that it is accessible from the internet.

Cleaning up challenges
Failed to renew certificate pzidgrabber.com with error: Some challenges have failed.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
All simulated renewals failed. The following certificates could not be renewed:
  /etc/letsencrypt/live/pzidgrabber.com/fullchain.pem (failure)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1 renew failure(s), 0 parse failure(s)

My web server is (include version): Nginx 1.18.0

The operating system my web server runs on is (include version): Debian 18

My hosting provider, if applicable, is: Linode

I can login to a root shell on my machine (yes or no, or I don't know): Yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): No

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): certbot 2.7.1

My Conf.d file:

server {
    server_name    pzidgrabber.com www.pzidgrabber.com;
    root           /var/www/pzidgrabber.com/frontend;
    index          index.html;

    gzip             on;
    gzip_comp_level  3;
    gzip_types       text/plain text/css application/javascript image/*;

    listen [::]:443 ssl ipv6only=on; # managed by Certbot
    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/pzidgrabber.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/pzidgrabber.com/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

    location ~ /.well-known/acme-challenge/ {
        allow all;
}

}
server {
    if ($host = www.pzidgrabber.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


    if ($host = pzidgrabber.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


    listen         80;
    listen         [::]:80;
    server_name    pzidgrabber.com www.pzidgrabber.com;
    return 404; # managed by Certbot

    location ~ /.well-known/acme-challenge/ {
        allow all;
}



}

server {
        server_name api.pzidgrabber.com;
        location / {
                proxy_pass "http://127.0.0.1:8000";
                }
        error_page 405 =200 $uri;

    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/pzidgrabber.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/pzidgrabber.com/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

      location ~ /.well-known/acme-challenge/ {
        allow all;
        }


}


server {
    if ($host = api.pzidgrabber.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


    server_name api.pzidgrabber.com;
    listen 80;
    return 404; # managed by Certbot

    location ~ /.well-known/acme-challenge/ {
        allow all;
}
}

Thank you!!

1 Like
3

Perhaps order matters ...
I'd try flipping those around [and put the return after the location].

Although, perhaps there is more than meets the eye:

While using the nginx authenticator is difficult to get a 404 error.
hmm...

3 Likes
4

Do you have an IPV 6 listen statement for your API domain the same as you have for your root and www domain because if you forgot that your Nginx would respond using the wrong server block.

2 Likes
5

Seems like a "not".

2 Likes
6

Oh, I didn't scroll down far enough ! :slight_smile:

That is definitely the problem.

3 Likes
7

There also seems to be something weird with the DNS delegation for api.pzidgrabber.com:

https://dnsviz.net/d/api.pzidgrabber.com/dnssec/

pzidgrabber.com to api.pzidgrabber.com: No delegation NS records were detected in the parent zone (pzidgrabber.com). This results in an NXDOMAIN response to a DS query (for DNSSEC), even if the parent servers are authoritative for the child.

I'm not entirely sure what that's trying to convey (especially since it doesn't look like pzidgrabber.com is DNSSEC-signed), but it may be something else worth looking into.

4 Likes
8

So I'd do this instead?
server {
if ($host = api.pzidgrabber.com) {
return 301 https://$host$request_uri;
} # managed by Certbot

server_name api.pzidgrabber.com;
listen [::]:80;
listen 80;
return 404; # managed by Certbot

location ~ /.well-known/acme-challenge/ {
    allow all;

}
}

2 Likes
9

Yes, do the listens just like you did for your other server block. The Let's Encrypt server makes an IPv6 request to your domain (when your DNS has an AAAA record which you do).

Because you were not listening on IPv6 for your api domain the nginx server sent that LE request to the other server block that was listening on IPv6. That is, the other one became your default server for IPv6.

Also, you do not need that Location block for the acme challenge URI. You are using the nginx plug-in which handles the ACME challenge for you by making temp changes to your nginx config.

3 Likes
10

More than likely, the API site was not working via IPv6.
You may not have noticed that if none of its' clients are using IPv6.
But LE noticed the failure because it prefers IPv6 over IPv4:

Name:      api.pzidgrabber.com
Addresses: 2600:3c02::f03c:93ff:fe78:9ff6
           194.195.209.93
4 Likes
11

Okay great I made those changes and it looks like everything is working with the certbot renew --dry-run command. Thank you all for your help @MikeMcQ @rg305 @petercooperjr !

3 Likes
12

So now you will be able to handle IPv6 API clients too - Woo hoo !!!

4 Likes
13

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.