401 during certbot renew - invalid response

Help
1

I'm getting 401 when I'm trying to renew my certificates, this is the first renewal of the certificates.

nginx.log

2600:3000:2710:300::89 - - [30/Jul/2024:19:06:38 +0000] "GET /.well-known/acme-challenge/ZNiQOZyk7NhqBSH57goNM28Jx2NRmEykEBA0yWAEik0 HTTP/1.1" 401 179 "http://www.myserver.net/.well-known/acme-challenge/ZNiQQZyk7NhqBSH56goNM28Jx2NRmEykEBA0yWAEik0" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)" "-"
2600:3000:2710:300::81 - - [30/Jul/2024:19:06:38 +0000] "GET /.well-known/acme-challenge/VVoY5ehTsazE1P_Pa1goxVKT8CtWfgIXocfof9n0Aas HTTP/1.1" 401 179 "http://myserver.net/.well-known/acme-challenge/VVoY5ehTsezE1P_Pz1goxVKT8CtWfgIXocfof9n0Aas" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)" "-"

The command I'm running:

% sudo certbot renew --dry-run -v
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /usr/local/etc/letsencrypt/renewal/www.mydomain.net.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Certificate is due for renewal, auto-renewing...
Plugins selected: Authenticator standalone, Installer None
Simulating renewal of an existing certificate for www.mydomain.net and mydomain.net
Performing the following challenges:
http-01 challenge for mydomain.net
http-01 challenge for www.mydomain.net
Waiting for verification...
Challenge failed for domain www.mydomain.net
Challenge failed for domain mydomain.net
http-01 challenge for www.mydomain.net
http-01 challenge for mydomain.net

Certbot failed to authenticate some domains (authenticator: standalone). The Certificate Authority reported these problems:
  Domain: www.mydomain.net
  Type:   unauthorized
  Detail: 2620:28:c000:23::67: Invalid response from https://www.mydomain.net/.well-known/acme-challenge/3_rGd042f1ILP2XBEa_lSpQ8kuLucCXrCkAlmJS4d0A: 404

  Domain: mydomain.net
  Type:   unauthorized
  Detail: 2620:28:c000:23::67: Invalid response from https://www.mydomain.net/.well-known/acme-challenge/FFrDExw_vcXwEnssPuMGrD7yF5kDKif0W_LcSjCgjHs: 404

Hint: The Certificate Authority failed to download the challenge files from the temporary standalone webserver started by Certbot on port 80. Ensure that the listed domains point to this machine and that it can accept inbound connections from the internet.

Cleaning up challenges
Failed to renew certificate www.mydomain.net with error: Some challenges have failed.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
All simulated renewals failed. The following certificates could not be renewed:
  /usr/local/etc/letsencrypt/live/www.mydomain.net/fullchain.pem (failure)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1 renew failure(s), 0 parse failure(s)
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.

My web server is (include version):
nginx-1.24.0_16

The operating system my web server runs on is (include version):
FreeBSD 13.3-RELEASE

nginx conf:

% cat snippets/letsencrypt.conf
location ^~ /.well-known/acme-challenge/ {
    default_type "text/plain";
    root /usr/local/www/letsencrypt;
}
% stat -x /usr/local/www/letsencrypt
  File: "/usr/local/www/letsencrypt"
  Size: 2            FileType: Directory
  Mode: (0755/drwxr-xr-x)         Uid: (    0/    root)  Gid: (    0/   wheel)
Device: 18446744073624949075,1848312016   Inode: 581    Links: 2
Access: Tue Jul 30 18:50:30 2024
Modify: Tue Jul 30 18:50:30 2024
Change: Tue Jul 30 18:50:30 2024
 Birth: Tue Jul 30 18:50:30 2024

In nginx.conf

   server {
        listen 1.2.3.4:80 default;
        listen [1111:22:c000:13::17]:80 default;
        include snippets/letsencrypt.conf;
        access_log  /var/log/nginx/access.log main;

        location / {
            root   /usr/local/www/data;
            index  index.html;
        }
    }

    server {
        listen 1.2.3.4:80;
        listen [1111:22:a000:13::17]:80;
        include snippets/letsencrypt.conf;
        server_name www.mydomain.net mydomain.net;
        rewrite ^(.*)$ https://${server_name}$1 permanent;
    }

    server {
        listen 1.2.3.4:443 ssl;
        listen [1111:22:a000:13::17]:443 ssl;
        include snippets/letsencrypt.conf;
        server_name www.mydomain.net mydomain.net;

Certbot version: py39-certbot-2.9.0_1

Any ideas what should I try?

2

You need to locate whatever on your system is issuing that 401 error reply.

I'm a little puzzed by the error message you show though. It says the failure was "404" but the 401 in the log is more concerning.

Unless you have the rare situation that needs to specify the IP on the listen statements you should remove the IP addresses from them. That often causes trouble as those server blocks won't process any inbound requests from other IP. It is easy to forget you hard-coded the IP address there.

You should also look at anything that might reply to an ACME HTTP Challenge with a 401 like:

HTTP/1.1 401 Unauthorized
Server: nginx/1.24.0
WWW-Authenticate: Basic realm="Malware Feed"
2 Likes
3

Oh, sorry, I got so hungup looking at your nginx I missed the above statement.

The --standalone option requires exclusive use of port 80 (HTTP). With nginx running and using port 80 you would have to stop it before trying to renew. The automated renew should have failed because standalone could not get access to port 80 without a pre-hook / post-hook to stop/start nginx (which you did not mention so I assume you did not do).

A better option might be to use the --webroot or --nginx options. These allow your nginx to remain running and handle the HTTP Challenge itself. But, the 401 I describe would need to be resolved.

The HTTP server block should also get other changes if you decide on using --webroot. You need to wrap your rewrite in a server location block. Let us know if you need help with that.

2 Likes
4

Thank you @MikeMcQ, after I stopped nginx it worked!
I'm going to change it to use:

authenticator = nginx
installer = nginx

Then I'll try again and update this thread.

1 Like
5

I don't recommend modifying the renewal conf files directly

And, given you have a working system I suggest using certonly rather than --nginx for installer. For authenticator sure, but installer means it may make changes to your nginx config. Maybe you'll like them maybe not. Just warning.

You could do like

sudo certbot certonly --nginx --dry-run -d domainname1 -d domainname2

to test using it as authenticator. Then if it works run again without --dry-run

The --dry-run will not affect existing production certs

3 Likes
6

This worked, thank you!

4 Likes
7

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.