Cannot generate new certificate: unauthorized Invalid response from

Hi all,

I have tried many ways to solve this problem but I keep running into the same error and over again. I am able to access a test.txt file inside /.well-known/acme/test.txt through the browser and I have my ports 80 and 443 open for acme servers confirmed by telnet and traceroute. I am using nginx as the web server, I have tried many ways but I keep running into the same error.

Any help please, appreciated.

Below is my nginx configuration file (I am trying to verify with http only to start off with after successful generation of certificate I will switch to https):

server {
listen 80;

server_name portal.test.feenix.co.nz;

root /var/www/test/portal.test.feenix.co.nz/current;

access_log /var/log/nginx/portal.test.feenix.co.nz.log;
error_log /var/log/nginx/error.portal.test.feenix.co.nz.log;

index index.html index.htm;

location ~ /\.well-known\/acme-challenge {
    allow all;
}

location / {
    try_files $uri $uri/index.html $uri.html =404;
}
}

My domain is:

portal.test.feenix.co.nz

I ran this command:
sudo certbot certonly -a webroot --webroot-path=/var/www/test/portal.test.feenix.co.nz/current -d portal.test.feenix.co.nz -n --agree-tos --email

It produced this output:

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Starting new HTTPS connection (1): acme-staging.api.letsencrypt.org
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for portal.test.feenix.co.nz
Using the webroot path /var/www/test/portal.test.feenix.co.nz/current for all unmatched domains.
Waiting for verification...
Cleaning up challenges
Unable to clean up challenge directory /var/www/test/portal.test.feenix.co.nz/current/.well-known/acme-challenge
Failed authorization procedure. portal.test.feenix.co.nz (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://portal.test.feenix.co.nz/.well-known/acme-challenge/A2Ji6fAL0L2jv-NqitYXxcE6odA1gYg3FOSe5JByD9o:"<html>
    <head><title>404 Not Found</title></head>
    <body bgcolor="white">
    <center><h1>404 Not Found</h1></center>
    <hr><center>"     

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: portal.test.feenix.co.nz
   Type:   unauthorized
   Detail: Invalid response from
   http://portal.test.feenix.co.nz/.well-known/acme-challenge/A2Ji6fAL0L2jv-NqitYXxcE6odA1gYg3FOSe5JByD9o:
       "<html>
       <head><title>404 Not Found</title></head>
       <body bgcolor="white">
       <center><h1>404 Not Found</h1></center>
       <hr><center>"    indent preformatted text by 4 spaces

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A record(s) for that domain
   contain(s) the right IP address.

My web server is:
nginx version: nginx/1.6.2

The operating system my web server runs on is:
Linux Debian 8 Jessie

My hosting provider, if applicable, is:
Own data centres

I can login to a root shell on my machine (yes):
Yes

Hello @ahmadkhudeish,

Requests to http://portal.test.feenix.co.nz/ are being redirected to other domain https://test.feenix.co.nz/

$ curl -IkL http://portal.test.feenix.co.nz/.well-known/acme-challenge/A2Ji6fAL0L2jv-NqitYXxcE6odA1gYg3FOSe5JByD9o
HTTP/1.1 301 Moved Permanently
Server: nginx/1.6.2
Date: Wed, 04 Oct 2017 22:46:23 GMT
Content-Type: text/html
Content-Length: 184
Connection: keep-alive
Location: https://test.feenix.co.nz/.well-known/acme-challenge/A2Ji6fAL0L2jv-NqitYXxcE6odA1gYg3FOSe5JByD9o

HTTP/1.1 404 Not Found
Server: nginx/1.6.2
Date: Wed, 04 Oct 2017 22:46:24 GMT
Content-Type: text/html
Connection: keep-alive

So two things:

1.- I can’t see any return/rewrite directive in the nginx conf you posted so, or you didn’t post the right conf file or you have not enabled this one… or you have other conf file that takes precedence over the one posted.

2.- As you have a redirection, the webroot path used in certbot command should be the one defined for test.feenix.co.nz instead of the path /var/www/test/portal.test.feenix.co.nz/current used for portal.test.feenix.co.nz.

Cheers,
sahsanu

2 Likes

Option 2 might fail as the two names resolve to two different IPs.
And thus might not be served by the same server.
Without additional info, Option 1 seems most likely.

2 Likes

@sahsanu @rg305 I am not sure why this is redirecting, but what I have setup here is some form of redundancy with quagga server anycasting the same ip address from two boxes. This is done at the router level, so I have two vms named web1t and web2t both advertising the same IP address. Could this be the reason why it’s failing? The routing is setup to direct requests to the nearest server to the client, it’s not round-robin. In this instance web1t is the closest, so all the portal.test.feenix.co.nz requests are hitting that web server.

I checked nginx sites-available and sites-enabled and there is only one vhost in there which is the one I posted above. What might be the other reason why this is redirecting to test.feenix.co.nz? I am a little confused here. Thanks for your help

Yes, that explains a lot.
You have a 50/50 chance of hitting the server that made the request - and it doesn’t scale well from there as LE is now implementing multiple source IP authentication. That means 50/50 per each authenticating IP; of which you must pass a majority…
Any statistics majors online? I’d say
3/2 * 50% * 50% * 50% or a 3/16 chance for 3 source IPs and
5/3 * 50% * 50% * 50% * 50% * 50% or a 5/96 chance for 5 source IPs
And that’s not to mention the possibility of adding any more servers to your anycasting.

So, unless you could programmatically bring one node out of the anycasting list.
then run the cert program
then bring the node back in.
You might then want to look into DNS auth.

1 Like

You can configure Nginx to redirect to the other server if the file doesn’t exist locally. Then you could pass HTTP-01 validation regardless of which server received the request. E.g.

location /.well-known/acme-challenge/ {
    try_files $uri @redirect;
}
location @redirect {
    return http://server-b.example.net$request_uri;
}
2 Likes

@rg305 @mnordhoff Wohoo I took down web2t from the anycast list and ran it again and it managed to create the certificate sucessfully :grinning: I think what was happening is I was requesting the certificate from web1t and Letsencrypt server was certifying/talking with the server closer to it as guided by the anycast router. In this instance web2t was deemed geographically closer to the Letsencrypt web server and that’s why it was redirecting to test.feenix.co.nz which is located on web2t.

So now that the certs are generated sucessfully, going forward, what is the best way to manage automated renewals?
I want the certs to also be on web2t.
Is there an easy way around this?
or do I have to create a script to copy out the file from from web1t to web2t? ( Less preferable)

Thanks a lot for your help.

tricky pete!

IP redirection would not be followed but providing unique FQDNs could do the trick.

I’d like to see that in action but it makes sense.

And although I did provide “a solution”, this would be a much better solution.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.