Acme-challenge unauthorized 404 Nginx

professorr-x · June 30, 2023, 2:59pm

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. https://crt.sh/?q=example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: admin.tadfeen.nl

I ran this command: sudo certbot certonly --webroot -w /usr/share/nginx/html/.well-known/acme-challenge -d admin.tadfeen.nl --dry-run

this is my config in
/etc/nginx/sites-available

server {
    server_name admin.tadfeen.nl;


    location ~ /.well-known/acme-challenge/ {
        allow all;
        root /usr/share/nginx/html/.well-known/acme-challenge;
    }

    location / {
        proxy_pass http://localhost:3001;
    }

    return 404; # managed by Certbot
}

It produced this output:


Saving debug log to /var/log/letsencrypt/letsencrypt.log
Simulating a certificate request for admin.tadfeen.nl

Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
  Domain: admin.tadfeen.nl
  Type:   unauthorized
  Detail: 2a01:4f8:c012:10a0::1: Invalid response from http://admin.tadfeen.nl/.well-known/acme-challenge/TJfCIrtYhy071FkbeWOCVumogj1DrUcLUiM24CFqO-Q: 404

Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded from the internet.

2023-06-30 14:53:33,040:DEBUG:acme.client:Sending POST request to https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/7110541454:
{
  "protected": "eyJhbGciOiAiUlMyNTYiLCAia2lkIjogImh0dHBzOi8vYWNtZS1zdGFnaW5nLXYwMi5hcGkubGV0c2VuY3J5cHQub3JnL2FjbWUvYWNjdC8xMDkwOTYyMjQiLCAibm9uY2UiOiAiODhCODhuN3dYR1pWSzdKV2FkQzV2Tms5aGNxQ1dZNW9JbnJBYURYa0RDS1Rpb2ciLCAidXJsIjogImh0dHBzOi8vYWNtZS1zdGFnaW5nLXYwMi5hcGkubGV0c2VuY3J5cHQub3JnL2FjbWUvYXV0aHotdjMvNzExMDU0MTQ1NCJ9",
  "signature": "q0e8ygkj0A9E4DUqOfmvu_MH5bUDCDEQXn8HohTp3TynLFwS9ttPk5t8gv0sw5aaRRhhxkcREQjJGC4DJbsS9OXyrQXT-ipQPZRkQuhHQx4ZXUBb1t7oswsVJIaOD-Vv5M7ameDJNGJ8GbPCCeetF-IwzjZvPn9dz_OlI1BbF-p4Rwl7zqgjL8vIVG5lJsFYsrzX6Ocb1OwHxpSzcM5ibyfvHoQTDvziSf_pWhuZrYsiEX_GEwwR74tauJ3QuTU4lyMAdYDwe2hJAVv7mnB6DlQuUaXA_A6YHo-JmbqVcEz8pqNQ-TziVbSWv4b4TSbq531zUgugapbOinrbQBrpXA",
  "payload": ""
}
2023-06-30 14:53:33,195:DEBUG:urllib3.connectionpool:https://acme-staging-v02.api.letsencrypt.org:443 "POST /acme/authz-v3/7110541454 HTTP/1.1" 200 1089
2023-06-30 14:53:33,196:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Fri, 30 Jun 2023 14:53:33 GMT
Content-Type: application/json
Content-Length: 1089
Connection: keep-alive
Boulder-Requester: 109096224
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-staging-v02.api.letsencrypt.org/directory>;rel="index"
Replay-Nonce: 7F3CPvP1BxhY_Oaf-cy9jrNgAUsAtPXG_BYFyHxgatCZggI
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

{
  "identifier": {
    "type": "dns",
    "value": "admin.tadfeen.nl"
  },
  "status": "invalid",
  "expires": "2023-07-07T14:53:31Z",
  "challenges": [
    {
      "type": "http-01",
      "status": "invalid",
      "error": {
        "type": "urn:ietf:params:acme:error:unauthorized",
        "detail": "2a01:4f8:c012:10a0::1: Invalid response from http://admin.tadfeen.nl/.well-known/acme-challenge/TJfCIrtYhy071FkbeWOCVumogj1DrUcLUiM24CFqO-Q: 404",
        "status": 403
      },
      "url": "https://acme-staging-v02.api.letsencrypt.org/acme/chall-v3/7110541454/OKJntA",
      "token": "TJfCIrtYhy071FkbeWOCVumogj1DrUcLUiM24CFqO-Q",
      "validationRecord": [
        {
          "url": "http://admin.tadfeen.nl/.well-known/acme-challenge/TJfCIrtYhy071FkbeWOCVumogj1DrUcLUiM24CFqO-Q",
          "hostname": "admin.tadfeen.nl",
          "port": "80",
          "addressesResolved": [
            "49.13.51.27",
            "2a01:4f8:c012:10a0::1"
          ],
          "addressUsed": "2a01:4f8:c012:10a0::1"
        }
      ],
      "validated": "2023-06-30T14:53:31Z"
    }
  ]
}
2023-06-30 14:53:33,196:DEBUG:acme.client:Storing nonce: 7F3CPvP1BxhY_Oaf-cy9jrNgAUsAtPXG_BYFyHxgatCZggI
2023-06-30 14:53:33,197:INFO:certbot._internal.auth_handler:Challenge failed for domain admin.tadfeen.nl
2023-06-30 14:53:33,197:INFO:certbot._internal.auth_handler:http-01 challenge for admin.tadfeen.nl
2023-06-30 14:53:33,197:DEBUG:certbot._internal.display.obj:Notifying user: 
Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
  Domain: admin.tadfeen.nl
  Type:   unauthorized
  Detail: 2a01:4f8:c012:10a0::1: Invalid response from http://admin.tadfeen.nl/.well-known/acme-challenge/TJfCIrtYhy071FkbeWOCVumogj1DrUcLUiM24CFqO-Q: 404

Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded from the internet.

2023-06-30 14:53:33,198:DEBUG:certbot._internal.error_handler:Encountered exception:
Traceback (most recent call last):
  File "/snap/certbot/3026/lib/python3.8/site-packages/certbot/_internal/auth_handler.py", line 108, in handle_authorizations
    self._poll_authorizations(authzrs, max_retries, max_time_mins, best_effort)
  File "/snap/certbot/3026/lib/python3.8/site-packages/certbot/_internal/auth_handler.py", line 212, in _poll_authorizations
    raise errors.AuthorizationError('Some challenges have failed.')
certbot.errors.AuthorizationError: Some challenges have failed.```

My web server is (include version): Nginx

The operating system my web server runs on is (include version): Ubuntu 20.0

My hosting provider, if applicable, is: Hertzner

I can login to a root shell on my machine (yes or no, or I don't know): yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): no

The version of my client is (e.g. output of `certbot --version` or `certbot-auto --version` if you're using Certbot): certbot 2.6.0

Bruce5051 · June 30, 2023, 3:12pm

Hello @professorr-x, welcome to the Let's Encrypt community.

Using the online tool Let's Debug yields these results https://letsdebug.net/admin.tadfeen.nl/1534985

MultipleIPAddressDiscrepancy
Warning
admin.tadfeen.nl has multiple IP addresses in its DNS records. While they appear to be accessible on the network, we have detected that they produce differing results when sent an ACME HTTP validation request. This may indicate that some of the IP addresses may unintentionally point to different servers, which would cause validation to fail.
[Address=2a01:4f8:c012:10a0::1,Address Type=IPv6,Server=nginx/1.18.0 (Ubuntu),HTTP Status=404] vs [Address=49.13.51.27,Address Type=IPv4,Server=nginx/1.18.0 (Ubuntu),HTTP Status=200]

Get your server responding identical for both IPv4 and IPv6 addresses.

MikeMcQ · June 30, 2023, 3:16pm

You have at least a couple problems.

One, your DNS has an A and AAAA record for IPv4 and IPv6. But, these are getting different responses. Let's Encrypt will prefer the IPv6. You need to fix these or remove the AAAA if it is not your server.

Your nginx server block does not have any listen clauses. You should add a listen for both IPv4 and IPv6 (or just IPv4 if you don't use IPv6)

The server block you show will always return a 404 Not Found code. For now you should remove that statement. Once you get a cert setup you will want to redirect from HTTP to HTTPS and have your proxy_pass statement in your HTTPS server block.

You can see your public IP addresses with this

curl -4 https://ifconfig.io
curl -6 https://ifconfig.io

You won't get a response to the -6 curl if you don't have IPv6 working so then remove the AAAA record.

rg305 · June 30, 2023, 4:50pm

Also...
Since the location ends with "/", the root should also end with a "/".
OR
Since the root doesn't ends with "/", the location should also not end with a "/".

MikeMcQ · June 30, 2023, 4:58pm

I don't think that should matter. What is important is that the --webroot-path (-w) shown in the first post matches the root value. The location string is just for matching the request URI and so dictates when that root value is used.

rg305 · June 30, 2023, 5:00pm

I have seen such mismatches to cause errors, first hand [that are difficult to track down].
Maybe placing a test file in the expected challenge location would be a good test at this time.

MikeMcQ · June 30, 2023, 5:12pm

Will get a 404 because of the return 404; at the bottom of that server block

rg305 · June 30, 2023, 5:35pm

Doesn't the "location ~" end the search/match there?

MikeMcQ · June 30, 2023, 5:55pm

For other locations in this example but not things outside of locations

rg305 · June 30, 2023, 6:10pm

ACME doesn't care about things outside the challenge path.
[and neither do I - looks down and sees "ACME" tattoo on chest! - LOL]

rg305 · June 30, 2023, 6:11pm

So... I repeat myself:

MikeMcQ · June 30, 2023, 6:22pm

But, nginx will see the return 404; and do that always

This is the same issue as when people put a redirect outside of all the location blocks

rg305 · June 30, 2023, 6:28pm

Agree to disagree.

MikeMcQ · June 30, 2023, 6:58pm

OK. But, I just tested this exact scenario and it works as I described.
nginx 1.18 Ubuntu

One of these times you'll believe me about this

professorr-x · June 30, 2023, 7:07pm

@MikeMcQ @rg305

i had a successful dry run, what i have done so far is:

Fixed the DNS records, i found the issue
Updated my config file to the below:

server {
    listen 80;
    listen [::]:80;  # Add this line for IPv6

    server_name admin.tadfeen.nl;

    location / {
        proxy_pass http://localhost:3001;
    }

    location /.well-known/acme-challenge/ {
        root /usr/share/nginx/html/.well-known/acme-challenge/;
    }

}

Then i ran :

sudo certbot certonly --webroot -w /usr/share/nginx/html/.well-known/acme-challenge/ -d admin.tadfeen.nl --dry-run

MikeMcQ · June 30, 2023, 7:31pm

Excellent progress. Before proceeding, is there a reason you chose certonly --webroot rather than the --nginx plug-in. Because your method requires you to create your own HTTPS (port 443) server block. The --nginx plug-in makes one for you based on your port 80 server block.

I personally prefer certonly webroot as I like to make my own server blocks. But, not everyone likes to do that.

If you wanted to change methods now is the time before proceeding. Let us know.

professorr-x · June 30, 2023, 8:12pm

I started off with the --nginx plugin then i after the acme challenge issue i ended up on here following threads, i didn't know the difference but now that you tell me that i will be switching back to the --nginx plug in

Thanks!

MikeMcQ · June 30, 2023, 8:17pm

Ok, probably good choice.

You can remove the below from your nginx server block. The --nginx plug-in manages the ACME paths and responses for you.

I'm not sure if --nginx plug-in will remove the location for your proxy_pass from your port 80 server block once it sets up your port 443 block. You may need to remove that manually when it's done. Let us know if any questions about the results. Cheers

system · July 30, 2023, 8:18pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
404 ERROR while accessing /.well-known/acme-challenge/ Help	4	16314	July 5, 2019
Cannot access acme challenge file Help	2	224	June 27, 2024
Acme Challenge Location NGINX Help	17	2958	May 18, 2024
Challenge failed for domain Help	20	337	June 19, 2024
Nginx Acme Challenge fails Help	4	305	August 25, 2023

Acme-challenge unauthorized 404 Nginx

Related topics