Invalid response ".well-known"

Hi!

I followed this guide how to generate certificate for my domain.

But I get get the error ‘invalid response from… .well-known/acme-challenge/VH-gFediU…’

I seems it’s looking for a file/page for a response.

Is it setting up this path to page by itself, or am I supposed to do this?

Doesn’t mention anything about this is the tutorial I linked to.

Hope to get any clearification about this.

Hi @sundrik

please answer all of these questions:


Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. https://crt.sh/?q=example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is:

I ran this command:

It produced this output:

My web server is (include version):

The operating system my web server runs on is (include version):

My hosting provider, if applicable, is:

I can login to a root shell on my machine (yes or no, or I don’t know):

I’m using a control panel to manage my site (no, or provide the name and version of the control panel):

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you’re using Certbot):

My domain is: N/A

I ran this command: sudo certbot --nginx -d mysite.com -d www.mysite.com

It produced this output:
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator nginx, Installer nginx
Starting new HTTPS connection (1): acme-v02.api.letsencrypt.org
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for mysite.com
http-01 challenge for www.mysite.com
Using default address 80 for authentication.
Using default address 80 for authentication.
Waiting for verification…
Cleaning up challenges
Failed authorization procedure. mysite.com (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://mysite.com/.well-known/acme-challenge/UH-geFrEdUN10Br3dAN4uoT0biA2sxXXrf80: "

Not Found

The requested URL /.well-known/acme-challenge/UH-geFrEdUN10Br3dAN4uoT0biA2sxXXrf80 was not found on ", www.mysite.com (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://www.mysite.com/.well-known/acme-challenge/geFrEdUN10Br3dAN4uoT0biA2sxXXrf80: "

Not Found

The requested URL /.well-known/acme-challenge/geFrEdUN10Br3dAN4uoT0biA2sxXXrf80 was not found on "

IMPORTANT NOTES:

My web server is (include version):
ubuntu 16.04
nginx 1.10.3
certbot 0.28.0

The operating system my web server runs on is (include version):
as above

My hosting provider, if applicable, is: N/A

I can login to a root shell on my machine (yes or no, or I don’t know): Yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel): No

The version of my client is N/A

Renewals are now being done over port 80 (HTTP).
If you were previously using port 443 (HTTPS) for renewals, you might just be able to redirect all HTTP requests to HTTPS and LE will follow the redirection and find the file it was looking for.

If that fails, then you need to compare the vhost configs (:80 and :443) and depending on the differences found, take action to allow the authentication to complete via port 80 (HTTP).

I realize this redirection is somewhat vague, but without any specific information this is the best I can do.

Thanks for answering. This is not a renewal. It’s initial.

I have now added the location in nginx config. And created the folder manually with a test file in.

I can reach my testfile when typing the url in a browser:
http://mysite.com/.well-known/acme-challenge/testfile

Now when I try: sudo certbot --nginx -d mysite.com

I get:
http://mysite.com/.well-known/acme-challenge/FZnOTbx3FGRhtrLeLNSG9hCehCeTQn66AxPqkaOlyO8:
“\r\n404 Not Found\r\n<body
bgcolor=“white”>\r\n

404 Not
Found

\r\n

So I get the response. But the Crtbot is not creating whatever file it supposed to do.

I can't help test it without the actual domain name.
So, are you sure you are testing from the Internet (not from the same system)?

Can you show the detail of these steps?:

I added this line in nginx config

# SSL Configuration
        location ~ /.well-known {
                allow all;
                root /var/www/html;
        }

And I created the folder:

/var/www/html/.well-known/acme-challenge

And in this folder a file with the text 'test'.

When I use the same url that fails, but with the file in the end instead of the random hexdec. I can fetch the file.

Did you also add it to the non-ssl vhost config block/file?

No. The guide didn’t mention any of that. I don’t even know what it is.

The block where the location was added, does it have a listen statement?
Show that line (or all lines) with listen.

Yes, it listen on my port for this server.

listen 8004;

server_name mysite.com www.mysite.com;

I redirect incoming 80 and 443 to this server right now. I checked that it works.

Where does 80 go?
Where does 443 go?
[who talks to 8004?]

I don’t know. To the server via gunicorn?

    upstream mysite_server {

        server unix:/home/user/projects/mysite/gunicorn.sock fail_timeout=10s;
}

server {
        listen 8004;
        access_log /home/user/projects/logs/mysite_access.log;
        error_log /home/user/projects/logs/mysite_error.log info;

        server_name mysite.com www.mysite.com;

        location / {

        proxy_pass http://unix:/home/user/projects/mysite/gunicorn.sock;

        proxy_pass_header Server;
        proxy_set_header X-Forwarded-Host $server_name;
        proxy_set_header X-Real-IPP $remote_addr;
        proxy_set_header X-Forwarded-for $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;

        }

        location /static {

                alias /home/user/projects/mysite/static;

        }

        location /media {

                alias /home/user/projects/mysite/mysite/media;

        }

        # SSL Configuration
        location ~ /.well-known {
                allow all;
                root /var/www/html;
        }
}

That’s all of the config.

That config seems to be only handling HTTP connections
There should be a second for the HTTPS connections.
Where do you handle the port forwarding?
Check where does port 80 go to and where does port 443 go to.

Both 80 and 443 is redirected to 8004 on this server right now from my switch.
Do I have to use two different ports for this because config can’t handle both in same listen?

According to the guide I mention in the beginning. It only says the nginx config to look like this.

server {
    listen 80 default_server;
    listen [::]:80 default_server;
    root /var/www/html;
    server_name example.com www.example.com;
}

And when you run the certbot command, it will change the config accordingly.

Why is this not working? Because gunicorn?

Exactly!
Each listener can only do one protocol HTTP or HTTPS - if you need both, then you need two listeners.
80 > X = (8004)
443 > Y = (????)

Then this should be your http section (change to listen 8004):

I see that this is added by Certbot in the Guide

listen 443 ssl; # managed by Certbot

    # RSA certificate
    ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem; # managed by Certbot

    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot

    # Redirect non-https traffic to https
    if ($scheme != "https") {
        return 301 https://$host$request_uri;
    } # managed by Certbot

Should I just add all this and use my new port instead of 443.
Even add the paths to .pem and .conf files, even though they don’t exsist yet?

If the cert files don’t yet exist, then the web server will fail to start.
Order matters.

Get port 80 to reach your web server.
Get a cert.
Add cert to whatever service/port you need (other than to same port 80 goes).

OK the way they show it here doesn’t seem (to me) to be best practice:
[both listeners are in the same server block - without any real separation]

server {
    listen 80 default_server;
    listen [::]:80 default_server;
    root /var/www/html;
    server_name  example.com www.example.com;

    listen 443 ssl; # managed by Certbot

    # RSA certificate
    ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem; # managed by Certbot

    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot

    # Redirect non-https traffic to https
    if ($scheme != "https") {
        return 301 https://$host$request_uri;
    } # managed by Certbot
}