Certbot unauthorized

siegfried0 · June 20, 2020, 12:55am

My domain is: grepolis-update.php-test.de

I ran this command (as root): certbot certonly --nginx --rsa-key-size 4096
(and selected grepolis-update.php-test.de)

It produced this output:

Failed authorization procedure. grepolis-update.php-test.de (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://grepolis-update.php-test.de/.well-known/acme-challenge/j61o90QIBEuN08AqZ5s0sWmx2O_cLRTG6P5LnAMgbCA [167.86.110.125]: "<html>\r\n<head><title>404 Not Found</title></head>\r\n<body>\r\n<center><h1>404 Not Found</h1></center>\r\n<hr><center>nginx/1.18.0</ce"

My web server is: nginx 1.18.0 (with ModSecurity)

The operating system my web server runs on is (include version): Debian 10.4

My hosting provider, if applicable, is: not applicable

I can login to a root shell on my machine: yes

I'm using a control panel to manage my site: no

The version of my client is: certbot 0.31.0

.
Complete description of my case:

My problem is: I already have other pages in the nginx config where the certificates were issued without problems. (e.g. api.chirmi.info - if someone wants to check that...)
Neither the configuration (of the http/80 server) nor the software versions were changed.

server {

   modsecurity on;
   modsecurity_rules_file /etc/nginx/modsec/main.conf;

   listen 80;
   root /var/www/cert;

   server_name grepolis-update.php-test.de;

   location / {
       return 301 https://$server_name$request_uri;
   }
   location ^~ /.well-known {
       try_files $uri $uri/ =404;
   }

}

The configuration also seems to work, because I get the following access log after the failed certbot command. (404 - not 301)

18.196.96.172 - - [20/Jun/2020:02:11:28 +0200] "GET /.well-known/acme-challenge/j61o90QIBEuN08AqZ5s0sWmx2O_cLRTG6P5LnAMgbCA HTTP/1.1" 404 125 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)" "-"
3.14.255.131 - - [20/Jun/2020:02:11:29 +0200] "GET /.well-known/acme-challenge/j61o90QIBEuN08AqZ5s0sWmx2O_cLRTG6P5LnAMgbCA HTTP/1.1" 404 125 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)" "-"
34.222.229.130 - - [20/Jun/2020:02:11:29 +0200] "GET /.well-known/acme-challenge/j61o90QIBEuN08AqZ5s0sWmx2O_cLRTG6P5LnAMgbCA HTTP/1.1" 404 125 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)" "-"
64.78.149.164 - - [20/Jun/2020:02:11:29 +0200] "GET /.well-known/acme-challenge/j61o90QIBEuN08AqZ5s0sWmx2O_cLRTG6P5LnAMgbCA HTTP/1.1" 404 125 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)" "-"

If I create the folders manually (mkdir -p /var/www/cert/.well-known/acme-challenge/) and create this file as a test (echo "test" > /var/www/cert/.well-known/acme-challenge/j61o90QIBEuN08AqZ5s0sWmx2O_cLRTG6P5LnAMgbCA), I can retrieve it with my browser as usual.

<MY-IP-ADDRESS> - - [20/Jun/2020:02:24:48 +0200] "GET /.well-known/acme-challenge/j61o90QIBEuN08AqZ5s0sWmx2O_cLRTG6P5LnAMgbCA HTTP/1.1" 200 3 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:76.0) Gecko/20100101 Firefox/76.0" "-"

Does anybody have an idea what else could be the reason for this?
(I'm still hoping that I'm just tired and missing something obvious.)

rg305 · June 20, 2020, 1:22am

I would try changing that line to:
location /.well-known/acme-challenge/ {

If that doesn't do the trick...
What is in this file?:

siegfried0 · June 20, 2020, 1:49am

/etc/nginx/modsec/main.conf contains a list of conf files for the ModSecurity module containing rule sets (see OWASP).
(If you are interested, look at GitHub - SpiderLabs/ModSecurity: ModSecurity is an open source, cross platform web application firewall (WAF) engine for Apache, IIS and Nginx that is developed by Trustwave's SpiderLabs. It has a robust event-based programming language which provides protection from a range of attacks against web applications and allows for HTTP traffic monitoring, logging and real-time analysis. With over 10,000 deployments world-wide, ModSecurity is the most widely deployed WAF in existence. and/or GitHub - SpiderLabs/owasp-modsecurity-crs: OWASP ModSecurity Core Rule Set (CRS) Project (Official Repository))
But that does not really matter. I commented it out together with the modsecurity on;.
http should not be used anyway and will not be forwarded - I should probably just disable the module for these connections anyway.

I also changed the location as mentioned, but the problem still remains.
(404 for letsencrypt)

server config now (current):

server { # modsecurity on; # modsecurity_rules_file /etc/nginx/modsec/main.conf;


   listen 80;
   root /var/www/cert;

   server_name grepolis-update.php-test.de;

   location / {
       return 301 https://$server_name$request_uri;
   }
   location /.well-known/acme-challenge/ {
       try_files $uri $uri/ =404;
   }

}

rg305 · June 20, 2020, 1:56am

Try it this way:

certbot certonly --webroot -w /var/www/cert -d grepolis-update.php-test.de --rsa-key-size 4096

siegfried0 · June 20, 2020, 2:04am

Thanks a lot, it worked out now!
Is this a bug in this certbot version?
(It worked with other sites so far - maybe the config got too big for it?)

I will (hopefully) remember to run certbot without the server-specific comfort in case of doubt.

rg305 · June 20, 2020, 2:08am

You are very welcomed.

I can't say for sure; I haven't seen your entire config.
[something in there must be tricking/confusing certbot]
What says?:
nginx -t

siegfried0 · June 20, 2020, 2:12am

nginx will never be reconfigured without this step.. so of course everything is fine.

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

I will not publish my complete config file - but as I already said, all my http "servers" are (have been) built the same way - a simple copy and paste

rg305 · June 20, 2020, 2:16am

I did not ask for your config.
I only made it clear that I could not say for sure if there is a bug without seeing the entire code.
I can only guess that cerbot is being confused somehow.
Maybe there are some lines that may be well commented out “#” but are somehow being prcessed by certbot.
Again, I am just guessing here.
And I don’t work for LE and don’t really care to see your code nor debug that “problem”.
My only goal was to get you a cert.
So “I win”, you have a cert.

siegfried0 · June 20, 2020, 2:19am

Yeah, no offense - I didn’t mean that as an accusation
Will remain a mystery, at least in my case.
Thanks again for the help!

rg305 · June 20, 2020, 2:21am

If you got all that you need then you “win” too!
If you still want to “lab” that, I would try “exporting” the entire config (nginx -T) and remove all the commented lines and go from there.

In any case, cheers from Miami

benb540 · July 11, 2020, 5:15pm

https://www.motorforlife.com/

benb540 · July 11, 2020, 5:16pm

https://www.motorforlife.com/

same issue with my website

rg305 · July 11, 2020, 11:24pm

@benb540
You should remove/delete these last two posts from this topic and start a new thread/topic to address your problem(s).

system · August 10, 2020, 11:24pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Certbot Failed authorization procedure Help	2	5681	October 9, 2019
Certbot unauthorized Help	4	3572	October 25, 2021
Certbot-auto failure, nginx server Help	9	1857	February 28, 2019
Unauthorized response no matter what ... =\| Help	12	3832	March 5, 2018
Failed authorization procedure Debian 8 Nginx Help	14	2649	January 14, 2018

Certbot unauthorized

Related topics