# Certbot nginx seems to be looking for the challenge in the wrong place

**URL:** <https://community.letsencrypt.org/t/certbot-nginx-seems-to-be-looking-for-the-challenge-in-the-wrong-place/158527>\
**Category:** Help\
**Created:** [August 25, 2021, 9:18am UTC](https://community.letsencrypt.org/t/certbot-nginx-seems-to-be-looking-for-the-challenge-in-the-wrong-place/158527 "2021-08-25T09:18:00Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![UltraBob](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/ultrabob/32/52237_2.png) [@UltraBob](https://community.letsencrypt.org/u/UltraBob)\
**Post date:** [August 25, 2021, 9:18am UTC](https://community.letsencrypt.org/t/certbot-nginx-seems-to-be-looking-for-the-challenge-in-the-wrong-place/158527/1 "2021-08-25T09:18:00Z")

</div>

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. [crt.sh | example.com](https://crt.sh/?q=example.com)), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is:  
solr.iges.jp

I ran this command:  
`sudo certbot --nginx -d solr1.iges.jp`

It produced this output:  
Saving debug log to /var/log/letsencrypt/letsencrypt.log  
Plugins selected: Authenticator nginx, Installer nginx  
Obtaining a new certificate  
Performing the following challenges:  
http-01 challenge for solr1.iges.jp  
Waiting for verification...  
Cleaning up challenges  
Failed authorization procedure. solr1.iges.jp (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from [https://www.iges.or.jp/.well-known/acme-challenge/ex\_UfY-isZ8-0PAvAD6ulw4YDi-IBeuwmr-mHsQ49HY](https://www.iges.or.jp/.well-known/acme-challenge/ex_UfY-isZ8-0PAvAD6ulw4YDi-IBeuwmr-mHsQ49HY)? [2606:4700:10::ac43:a1a]: "\n404 Not Found\n\<body bgcolor="white"\>\n

# 404 Not Found
\n
* * *
ngin"

IMPORTANT NOTES:

- The following errors were reported by the server:

My web server is (include version):  
nginx version: nginx/1.14.0 (Ubuntu)

The operating system my web server runs on is (include version):  
Ubuntu 18.04.5 LTS

My hosting provider, if applicable, is:  
Linode

I can login to a root shell on my machine (yes or no, or I don't know):  
yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):  
no

The version of my client is (e.g. output of `certbot --version` or `certbot-auto --version` if you're using Certbot):  
certbot 0.31.0

Note that the Invalid response seems to be coming from another of our domains, and I can't figure out where it might be getting redirected. How can I debug this?

---

<div class="post-metadata">

**Author:** ![rg305](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/rg305/32/91314_2.png) [@rg305](https://community.letsencrypt.org/u/rg305)\
**Post date:** [August 25, 2021, 10:07am UTC](https://community.letsencrypt.org/t/certbot-nginx-seems-to-be-looking-for-the-challenge-in-the-wrong-place/158527/3 "2021-08-25T10:07:56Z")

</div>

> [@UltraBob](#):
>
> `-d solr1.iges.jp`  
> Invalid response from [https://www.iges.or.jp/](https://www.iges.or.jp/)....

Well the message shows that the original request has been redirected to another FQDN.  
You will need to review the vhost config and likely make an exception to not redirect the challenge requests.

And confirmed with:

```nohighlight
curl -Iki solr1.iges.jp/.well-known/acme-challenge/Test-File-1234

HTTP/1.1 301 Moved Permanently
Server: nginx/1.14.0 (Ubuntu)
Date: Wed, 25 Aug 2021 10:06:20 GMT
Content-Type: text/html
Content-Length: 194
Connection: keep-alive
Location: https://www.iges.or.jp/.well-known/acme-challenge/Test-File-1234?

```

> [@UltraBob](#):
>
> How can I debug this?

Start with the output of:  
`sudo nginx -T`

---

<div class="post-metadata">

**Author:** ![UltraBob](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/ultrabob/32/52237_2.png) [@UltraBob](https://community.letsencrypt.org/u/UltraBob)\
**Post date:** [August 25, 2021, 10:10am UTC](https://community.letsencrypt.org/t/certbot-nginx-seems-to-be-looking-for-the-challenge-in-the-wrong-place/158527/4 "2021-08-25T10:10:35Z")

</div>

Thanks @rg305

There must be some weird dns cache somewhere or something. When I run the same command, I get:

```nohighlight
 ~  curl -Iki solr1.iges.jp/.well-known/acme-challenge/Test-File-1234
HTTP/1.1 404 Not Found
Server: nginx/1.14.0 (Ubuntu)
Date: Wed, 25 Aug 2021 10:07:47 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 178
Connection: keep-alive

```

Also I noticed that somehow I had a prehistoric version of certbot, so I reinstalled it, but get much the same output:

```nohighlight
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Requesting a certificate for solr1.iges.jp

Certbot failed to authenticate some domains (authenticator: nginx). The Certificate Authority reported these problems:
  Domain: solr1.iges.jp
  Type: unauthorized
  Detail: Invalid response from https://www.iges.or.jp/.well-known/acme-challenge/-murAGIIRzUBu2w4fIz_xHdZ6M3H_2bjbH4O8f8K1IE? [2606:4700:10::6816:b1b]: "<html>\n<head><title>404 Not Found</title></head>\n<body bgcolor=\"white\">\n<center><h1>404 Not Found</h1></center>\n<hr><center>ngin"

Hint: The Certificate Authority failed to verify the temporary nginx configuration changes made by Certbot. Ensure the listed domains point to this nginx server and that it is accessible from the internet.

Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.

```

---

<div class="post-metadata">

**Author:** ![rg305](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/rg305/32/91314_2.png) [@rg305](https://community.letsencrypt.org/u/rg305)\
**Post date:** [August 25, 2021, 10:13am UTC](https://community.letsencrypt.org/t/certbot-nginx-seems-to-be-looking-for-the-challenge-in-the-wrong-place/158527/5 "2021-08-25T10:13:00Z")

</div>

The version of `certbot` is not an issue here.  
[Although it was somewhat prehistoric - LOL]

DNS cache isn't to blame.  
I've never visited the site and it gave me the same results.

Let's have a look at the output of:  
`sudo nginx -T`  
and  
`sudo ps -ef | grep -i nginx`

---

<div class="post-metadata">

**Author:** ![rg305](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/rg305/32/91314_2.png) [@rg305](https://community.letsencrypt.org/u/rg305)\
**Post date:** [August 25, 2021, 10:15am UTC](https://community.letsencrypt.org/t/certbot-nginx-seems-to-be-looking-for-the-challenge-in-the-wrong-place/158527/6 "2021-08-25T10:15:29Z")

</div>

> [@UltraBob](#):
>
> ```nohighlight
> curl -Iki solr1.iges.jp/.well-known/acme-challenge/Test-File-1234
> HTTP/1.1 404 Not Found
> 
> ```

Was that from the same system or from the Internet?  
[what the server "sees" doesn't always match what the world "sees"]

---

<div class="post-metadata">

**Author:** ![UltraBob](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/ultrabob/32/52237_2.png) [@UltraBob](https://community.letsencrypt.org/u/UltraBob)\
**Post date:** [August 25, 2021, 10:19am UTC](https://community.letsencrypt.org/t/certbot-nginx-seems-to-be-looking-for-the-challenge-in-the-wrong-place/158527/7 "2021-08-25T10:19:09Z")

</div>

Sorry, I meant to include my virtual host config from nginx, but I forgot. I'll nclude that here. The previous curl output was run on my laptop terminal, not from the server.

solr1.iges.jp.conf:

```nohighlight
server {
    server_name solr1.iges.jp;
    root /var/www/my-iges-solr.iges.jp/testbed;
    index index.html;
    autoindex off;
    charset utf-8;
    error_page 404 /404.html;
    error_log /var/log/nginx/error.solr1.iges.jp.log;
    access_log /var/log/nginx/access.solr1.iges.jp.log;
# include global/restrictions.conf;
# auth_basic "Authorized access only.";
# auth_basic_user_file /var/www/my-iges-solr.iges.jp/.htpasswd;
    # Very rarely should these ever be accessed outside of your lan
    location ~* \.(txt|log)$ {
        allow 192.168.0.0/16;
        deny all;
    }

    location ~ \..*/.*\.php$ {
        return 403;
    }

    # Block access to scripts in site files directory
    location ~ ^/sites/[^/]+/files/.*\.php$ {
        deny all;
    }

    # Allow "Well-Known URIs" as per RFC 5785
    location ~* ^/.well-known/acme-challenge/ {
        allow all;
        auth_basic off;
    }

    # Block access to "hidden" files and directories whose names begin with a
    # period. This includes directories used by version control systems such
    # as Subversion or Git to store control files.
# location ~ (^|/)\. {
# return 403;
# }

    location /solr {
             proxy_set_header Host $host;
             proxy_set_header X-Real-IP $remote_addr;
             proxy_pass "http://127.0.0.1:8983";
    }

    location /data {
         return 403;
    }

    location /conf {
        return 403;
    }

    location /logs {
        return 403;
    }

}

```

```nohighlight
 sudo ps -ef | grep -i nginx (master|✚1…84)
root 23283 1 0 17:50 ? 00:00:00 nginx: master process /usr/sbin/nginx -g daemon on; master_process on;
www-data 31686 23283 0 19:05 ? 00:00:00 nginx: worker process
www-data 31687 23283 0 19:05 ? 00:00:00 nginx: worker process
bob 32260 4975 0 19:18 pts/2 00:00:00 grep --color=auto --exclude-dir=.bzr --exclude-dir=CVS --exclude-dir=.git --exclude-dir=.hg --exclude-dir=.svn --exclude-dir=.idea --exclude-dir=.tox -i nginx

```

---

<div class="post-metadata">

**Author:** ![rg305](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/rg305/32/91314_2.png) [@rg305](https://community.letsencrypt.org/u/rg305)\
**Post date:** [August 25, 2021, 10:20am UTC](https://community.letsencrypt.org/t/certbot-nginx-seems-to-be-looking-for-the-challenge-in-the-wrong-place/158527/8 "2021-08-25T10:20:14Z")

</div>

> [@UltraBob](#):
>
> curl output was run on my laptop terminal

Would that connect locally to the server (same private network) or over the Internet?

---

<div class="post-metadata">

**Author:** ![UltraBob](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/ultrabob/32/52237_2.png) [@UltraBob](https://community.letsencrypt.org/u/UltraBob)\
**Post date:** [August 25, 2021, 10:22am UTC](https://community.letsencrypt.org/t/certbot-nginx-seems-to-be-looking-for-the-challenge-in-the-wrong-place/158527/9 "2021-08-25T10:22:28Z")

</div>

> [@UltraBob](#):
>
> `curl -Iki solr1.iges.jp/.well-known/acme-challenge/Test-File-1234`

It would connect over the internet the server is in a data center, not on the same network as me. I also tried the command from a different linode server and got the same result

---

<div class="post-metadata">

**Author:** ![rg305](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/rg305/32/91314_2.png) [@rg305](https://community.letsencrypt.org/u/rg305)\
**Post date:** [August 25, 2021, 10:23am UTC](https://community.letsencrypt.org/t/certbot-nginx-seems-to-be-looking-for-the-challenge-in-the-wrong-place/158527/10 "2021-08-25T10:23:40Z")

</div>

Well the `ps` output looks good.

The provided piece of the `nginx` config doesn't show the problem - it must be elsewhere.  
If you have trust issues/security concerns (I know I sure do), you can PM the output to me - so the entire Internet doesn't see it.  
But I'm certain the problem can be found therein.

---

<div class="post-metadata">

**Author:** ![rg305](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/rg305/32/91314_2.png) [@rg305](https://community.letsencrypt.org/u/rg305)\
**Post date:** [August 25, 2021, 10:24am UTC](https://community.letsencrypt.org/t/certbot-nginx-seems-to-be-looking-for-the-challenge-in-the-wrong-place/158527/11 "2021-08-25T10:24:44Z")

</div>

> [@UltraBob](#):
>
> It would connect over the internet the server is in a data center

Well that throws a monkey-wrench in this fix plan...

Please show the output from your laptop:  
`ping solr1.iges.jp`

---

<div class="post-metadata">

**Author:** ![UltraBob](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/ultrabob/32/52237_2.png) [@UltraBob](https://community.letsencrypt.org/u/UltraBob)\
**Post date:** [August 25, 2021, 10:24am UTC](https://community.letsencrypt.org/t/certbot-nginx-seems-to-be-looking-for-the-challenge-in-the-wrong-place/158527/12 "2021-08-25T10:24:47Z")

</div>

You can also find a test file in place at [http://solr1.iges.jp/.well-known/acme-challenge/ok.html](http://solr1.iges.jp/.well-known/acme-challenge/ok.html)

---

<div class="post-metadata">

**Author:** ![rg305](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/rg305/32/91314_2.png) [@rg305](https://community.letsencrypt.org/u/rg305)\
**Post date:** [August 25, 2021, 10:25am UTC](https://community.letsencrypt.org/t/certbot-nginx-seems-to-be-looking-for-the-challenge-in-the-wrong-place/158527/13 "2021-08-25T10:25:28Z")

</div>

`html` test files are not best suited for testing that directory.

---

<div class="post-metadata">

**Author:** ![rg305](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/rg305/32/91314_2.png) [@rg305](https://community.letsencrypt.org/u/rg305)\
**Post date:** [August 25, 2021, 10:26am UTC](https://community.letsencrypt.org/t/certbot-nginx-seems-to-be-looking-for-the-challenge-in-the-wrong-place/158527/14 "2021-08-25T10:26:37Z")

</div>

```nohighlight
curl -Iki http://solr1.iges.jp/.well-known/acme-challenge/ok.html
HTTP/1.1 301 Moved Permanently
Location: https://www.iges.or.jp/.well-known/acme-challenge/ok.html?

```

---

<div class="post-metadata">

**Author:** ![rg305](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/rg305/32/91314_2.png) [@rg305](https://community.letsencrypt.org/u/rg305)\
**Post date:** [August 25, 2021, 10:27am UTC](https://community.letsencrypt.org/t/certbot-nginx-seems-to-be-looking-for-the-challenge-in-the-wrong-place/158527/15 "2021-08-25T10:27:43Z")

</div>

OK this might be to blame:

```nohighlight
Name: solr1.iges.jp
Addresses: 2400:8902::f03c:92ff:fe7a:8e69
           139.162.123.237

```

Yes it is!

```nohighlight
curl -Iki4 http://solr1.iges.jp/.well-known/acme-challenge/ok.html
HTTP/1.1 200 OK

curl -Iki6 http://solr1.iges.jp/.well-known/acme-challenge/ok.html
HTTP/1.1 301 Moved Permanently
Location: https://www.iges.or.jp/.well-known/acme-challenge/ok.html?

```

---

<div class="post-metadata">

**Author:** ![UltraBob](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/ultrabob/32/52237_2.png) [@UltraBob](https://community.letsencrypt.org/u/UltraBob)\
**Post date:** [August 25, 2021, 10:28am UTC](https://community.letsencrypt.org/t/certbot-nginx-seems-to-be-looking-for-the-challenge-in-the-wrong-place/158527/16 "2021-08-25T10:28:32Z")

</div>

OK, just a second, I'll remove the extension

ping output from a different server:

```nohighlight
ping solr1.iges.jp
PING solr1.iges.jp (139.162.123.237) 56(84) bytes of data.
64 bytes from beryllium.iges.jp (139.162.123.237): icmp_seq=1 ttl=63 time=0.393 ms
64 bytes from beryllium.iges.jp (139.162.123.237): icmp_seq=2 ttl=63 time=0.452 ms
64 bytes from beryllium.iges.jp (139.162.123.237): icmp_seq=3 ttl=63 time=0.457 ms
64 bytes from beryllium.iges.jp (139.162.123.237): icmp_seq=4 ttl=63 time=0.421 ms
^C
--- solr1.iges.jp ping statistics ---
47 packets transmitted, 47 received, 0% packet loss, time 49697ms
rtt min/avg/max/mdev = 0.311/0.476/0.619/0.063 ms

```

ping output from laptop:

```nohighlight
~  ping solr1.iges.jp
PING solr1.iges.jp (139.162.123.237): 56 data bytes
64 bytes from 139.162.123.237: icmp_seq=0 ttl=55 time=12.763 ms
64 bytes from 139.162.123.237: icmp_seq=1 ttl=55 time=12.419 ms
64 bytes from 139.162.123.237: icmp_seq=2 ttl=55 time=17.871 ms
64 bytes from 139.162.123.237: icmp_seq=3 ttl=55 time=19.560 ms
64 bytes from 139.162.123.237: icmp_seq=4 ttl=55 time=18.658 ms
^C
--- solr1.iges.jp ping statistics ---
5 packets transmitted, 5 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 12.419/16.254/19.560/3.040 ms

```

---

<div class="post-metadata">

**Author:** ![UltraBob](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/ultrabob/32/52237_2.png) [@UltraBob](https://community.letsencrypt.org/u/UltraBob)\
**Post date:** [August 25, 2021, 10:29am UTC](https://community.letsencrypt.org/t/certbot-nginx-seems-to-be-looking-for-the-challenge-in-the-wrong-place/158527/17 "2021-08-25T10:29:09Z")

</div>

hmmm, that should havebeen the right ipv6 address, bt let me double check, thank you!

---

<div class="post-metadata">

**Author:** ![rg305](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/rg305/32/91314_2.png) [@rg305](https://community.letsencrypt.org/u/rg305)\
**Post date:** [August 25, 2021, 10:29am UTC](https://community.letsencrypt.org/t/certbot-nginx-seems-to-be-looking-for-the-challenge-in-the-wrong-place/158527/18 "2021-08-25T10:29:55Z")

</div>

> [@UltraBob](#):
>
> This is resolved, the issue was that I didn't have a listen directive in my nginx virtual host config file, and so ipv6 requests were going somewhere unexpected. After adding in
> 
> ```nohighlight
> listen 80;
> listen [::]:80;
> 
> ```

Even if it is the right IP address, make sure that `nginx` is listening on both IPv4 and IPv6.  
There are no listen statements shown in the file: `solr1.iges.jp.conf`

---

<div class="post-metadata">

**Author:** ![UltraBob](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/ultrabob/32/52237_2.png) [@UltraBob](https://community.letsencrypt.org/u/UltraBob)\
**Post date:** [August 25, 2021, 10:35am UTC](https://community.letsencrypt.org/t/certbot-nginx-seems-to-be-looking-for-the-challenge-in-the-wrong-place/158527/19 "2021-08-25T10:35:58Z")

</div>

@rg305 thank you, I'm definitely learning some curl-foo

> [@rg305](#):
>
> `curl -Iki6 http://solr1.iges.jp/.well-known/acme-challenge/ok.html`

I added a listen directive in there, could you see if that command still gives you the same error?

---

<div class="post-metadata">

**Author:** ![rg305](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/rg305/32/91314_2.png) [@rg305](https://community.letsencrypt.org/u/rg305)\
**Post date:** [August 25, 2021, 10:37am UTC](https://community.letsencrypt.org/t/certbot-nginx-seems-to-be-looking-for-the-challenge-in-the-wrong-place/158527/20 "2021-08-25T10:37:33Z")

</div>

```nohighlight
curl -Iki6 http://solr1.iges.jp/.well-known/acme-challenge/ok.html
HTTP/1.1 200 OK

```

Back in business!

---

<div class="post-metadata">

**Author:** ![UltraBob](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/ultrabob/32/52237_2.png) [@UltraBob](https://community.letsencrypt.org/u/UltraBob)\
**Post date:** [August 25, 2021, 10:38am UTC](https://community.letsencrypt.org/t/certbot-nginx-seems-to-be-looking-for-the-challenge-in-the-wrong-place/158527/21 "2021-08-25T10:38:10Z")

</div>

Great, let me try that certbot command again!

[Next page](https://community.letsencrypt.org/t/certbot-nginx-seems-to-be-looking-for-the-challenge-in-the-wrong-place/158527.md?page=2)