Unauthorized invalid response from (cannot renew)

jlh · January 25, 2021, 5:32am

I ran this command: certbot --force-renewal -d www.kyouikulogistics.com

It produced this output: Type: Unauthorized, Detail: Invalid response from

http://www.kyouikulogistics.com/.well-known/acme-challenge/long string[my ip]: "<DOCYTPE HTML PUBLIC \ "-//IETF//DTD HTML
2.0//EN">\n\n

Not Found

\n<p"

I probably got a few forward/back slashes incorrect in the above...

My web server is (include version): Apache 2

The operating system my web server runs on is (include version): Ubuntu 16.04

My hosting provider, if applicable, is: Home (no provider)

I can login to a root shell on my machine (yes or no, or I don't know): Yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): Combination of Webmin and command line

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): 0.31.0

I had some difficulty renewing back in November but managed to accomplish by running a virtual server on port 80 along side my 443 version. The only thing I've messed with since then was to setup a port forward within Apache...guess that is what's causing trouble but am having a terrible time trying to figure out how to rectify the situation so I can renew my cert.

Any advice/assistance will be much appreciated.

Jason

griffin · January 25, 2021, 7:07am

Welcome Back to the Let's Encrypt Community, Jason

I just tried to run some tests on www.kyouikulogistics.com, but it stopped responding on both ports 80 and 443. Apache stopped? Adaptive firewall?

Please show the output of:

sudo apachectl -S

Certificate History

griffin · January 25, 2021, 7:13am

It looks like it's your application interfering with the responses to some of my tests. We would still benefit by looking into your apache configuration per the command I gave in my last post.

jlh · January 25, 2021, 7:18am

Grifffin,

Thanks...I've just realized that, for some reason, another server I have behind the one I am trying to renew the cert for, is responding to www.kyouikulogistics.com...which is odd...because that is behavior I was trying to accomplish with my proxy hosting on the first machine. Hmm, I've managed to mess something up somewhere....one moment and I'll paste the output of the apache configuration.

Here is is:

sudo apachectl -S
AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1. Set the 'ServerName' directive globally to suppress this message
VirtualHost configuration:
192.168.11.37:443 www.kyouikulogistics.com (/etc/apache2/sites-enabled/000-default.conf:11)
192.168.11.37:80 www.kyouikulogistics.com (/etc/apache2/sites-enabled/webmin.1600238302.conf:1)
ServerRoot: "/etc/apache2"
Main DocumentRoot: "/var/www/html"
Main ErrorLog: "/var/log/apache2/error.log"
Mutex default: dir="/var/lock/apache2" mechanism=fcntl
Mutex mpm-accept: using_defaults
Mutex watchdog-callback: using_defaults
Mutex rewrite-map: using_defaults
Mutex ssl-stapling-refresh: using_defaults
Mutex ssl-stapling: using_defaults
Mutex proxy: using_defaults
Mutex ssl-cache: using_defaults
PidFile: "/var/run/apache2/apache2.pid"
Define: DUMP_VHOSTS
Define: DUMP_RUN_CFG
Define: ENABLE_USR_LIB_CGI_BIN
User: name="www-data" id=33
Group: name="www-data" id=33

jlh · January 25, 2021, 7:25am

I setup schlc.kyouikulogistics.com back on the 30th of November last year but didn't have success in getting ProxyPass to work so I deleted it with the goal of working on it later. For some reason that subdomain seems to still be alive. Maybe that is not related to this issue but...

griffin · January 25, 2021, 7:28am

I ought to know better than to dig in without much time.

I'll be back later when I can, but for now, check into the two files containing VirtualHosts listed in the output you posted.

jlh · January 25, 2021, 7:29am

Griffin,

Thanks and much appreciated. I'll fiddle around and see what I can find/accomplish.

Jason

jlh · January 25, 2021, 7:47am

Griffin,

Thanks again...and here is a brief update.

I've deleted all reverse proxy settings and deleted the virtual server running on port 80. I then recreated a virtual server on port 80 with pathways to the letsencrypt fullchain.pem and privkey.pem. Restarted Apache and tried again but get the same error.

For reference, the contents of the two virtual servers are below

1: 000-default.conf (note - I deleted all lines that have been commented out except the first line...it looks like I may have commented that out and added the 443 but I am embarrassed to say...I cannot remember)

#<VirtualHost 192.168.11.37:80>

<VirtualHost 192.168.11.37:443>
ServerAdmin webmaster@localhost
DocumentRoot /var/www/html
ErrorLog {APACHE_LOG_DIR}/error.log CustomLog {APACHE_LOG_DIR}/access.log combined
SSLEngine on

            ServerAdmin webmaster@localhost

ErrorLog ${APACHE_LOG_DIR}/error.log

CustomLog ${APACHE_LOG_DIR}/access.log combined

    ServerName www.kyouikulogistics.com
Include /etc/letsencrypt/options-ssl-apache.conf

SSLCertificateFile /etc/letsencrypt/live/www.kyouikulogistics.com/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/www.kyouikulogistics.com/privkey.pem

2: webmin.1611560222.conf (created today after deleting the previous virtual server...via Webmin)

<VirtualHost 192.168.11.37:80>
DocumentRoot "/var/www/html"
<Directory "/var/www/html">
allow from all
Options None
Require all granted

            ServerAdmin webmaster@localhost
            DocumentRoot /var/www/html
            ErrorLog ${APACHE_LOG_DIR}/error.log
            CustomLog ${APACHE_LOG_DIR}/access.log combined
            SSLEngine on

            ServerAdmin webmaster@localhost

ErrorLog ${APACHE_LOG_DIR}/error.log

CustomLog ${APACHE_LOG_DIR}/access.log combined

    ServerName www.kyouikulogistics.com
Include /etc/letsencrypt/options-ssl-apache.conf

SSLCertificateFile /etc/letsencrypt/live/www.kyouikulogistics.com/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/www.kyouikulogistics.com/privkey.pem

jlh · January 25, 2021, 9:19am

Another quick update here.

Part of my problem was that I had two servers running at the same IP and, for some reason, the one running with port 80 only was being prioritized over the one running with two virtual servers (one on 80 and one on 443). I shut off Apache on the other server and now get the following error:

Type: Connection
Detail: Fetching
http://www.kyouikulogistics.com/.well-known....
Connection refused

I guess this represents baby steps in the correct direction but I'm not sure why the connection is being refused.

griffin · January 25, 2021, 9:48am

What's the current output of this:

sudo apachectl -S

jlh · January 25, 2021, 11:44am

Thanks,

Here it is. I messed around a bit more and didn't have much success.

sudo apachectl -S
AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1. Set the 'ServerName' directive globally to suppress this message
VirtualHost configuration:
192.168.11.14:443 www.kyouikulogistics.com (/etc/apache2/sites-enabled/000-default.conf:11)
192.168.11.14:80 www.kyouikulogistics.com (/etc/apache2/sites-enabled/webmin.1611560222.conf:1)
ServerRoot: "/etc/apache2"
Main DocumentRoot: "/var/www/html"
Main ErrorLog: "/var/log/apache2/error.log"
Mutex ssl-stapling: using_defaults
Mutex proxy: using_defaults
Mutex ssl-cache: using_defaults
Mutex default: dir="/var/lock/apache2" mechanism=fcntl
Mutex mpm-accept: using_defaults
Mutex watchdog-callback: using_defaults
Mutex rewrite-map: using_defaults
Mutex ssl-stapling-refresh: using_defaults
PidFile: "/var/run/apache2/apache2.pid"
Define: DUMP_VHOSTS
Define: DUMP_RUN_CFG
Define: ENABLE_USR_LIB_CGI_BIN
User: name="www-data" id=33
Group: name="www-data" id=33

rg305 · January 25, 2021, 2:46pm

Please don't use IPs in configs.
Things like:
<VirtualHost 192.168.11.37:443>
Should be just:
<VirtualHost *:443>

Q1: Are there more then one internal server that will be accessed via HTTP or HTTPS from the Internet?

Q2: To which IP does the NAT/FW send HTTP connections to? And to which for HTTPS?

jlh · January 26, 2021, 11:06am

rg305,

Thanks for the advice. I can change the virtual host settings but I don't think that is the problem in this case.

Q1: There was another server running but I didn't want to complicate things further so I have shut it down and removed any/all settings in the first server that refer to that second machine. The certbot still fails.

Q2: External IP for kyouikulogistics.com is 211.133.220.51. The internal machine is currently 192.168.11.14

On the machine listed in Q2, there are two virtual servers (one for 443 and one for 80).

I plan to work on this again tomorrow morning (evening here in Japan now) and hope to figure out the problem. Any advice/suggestions will be appreciated.

rg305 · January 26, 2021, 3:19pm

Why don't these IPs match?:

jlh · January 26, 2021, 11:38pm

Sorry, yeah, this is an evolving situation. I restarted the machine and got a new IP (not sure why) but the virtual server settings have changed as well. I can, will, change to *<VirtualHost :443> to avoid further issues.

So, currently it is <VirtualHost 192.168.11.24:443>
and
<VirtualHost 192.168.11.24:80>

Regards

Jason

jlh · January 26, 2021, 11:44pm

My current error, with certbot is:

"Error getting validation data" so it seems the connection is working but the fullchain and privkey files are not being accessed.

Jason

griffin · January 26, 2021, 11:45pm

Make those

would likely make things much easier.

Otherwise, you're binding your virtual hosts to only private IP addresses!

jlh · January 26, 2021, 11:48pm

griffin,

Thanks, done.

Now I just have to figure out why the validation data cannot be accessed...

Thanks for your support thus far.

Jason

griffin · January 26, 2021, 11:49pm

Full circle...

sudo apachectl -k graceful

What says this now?

sudo apachectl -S

jlh · January 26, 2021, 11:53pm

Thanks,

This is what I have now.

sudo apachectl -S
AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1. Set the 'ServerName' directive globally to suppress this message
VirtualHost configuration:
*:443 www.kyouikulogistics.com (/etc/apache2/sites-enabled/000-default.conf:11)
*:80 www.kyouikulogistics.com (/etc/apache2/sites-enabled/webmin.1611560222.conf:1)
ServerRoot: "/etc/apache2"
Main DocumentRoot: "/var/www/html"
Main ErrorLog: "/var/log/apache2/error.log"
Mutex proxy: using_defaults
Mutex ssl-cache: using_defaults
Mutex default: dir="/var/lock/apache2" mechanism=fcntl
Mutex mpm-accept: using_defaults
Mutex watchdog-callback: using_defaults
Mutex rewrite-map: using_defaults
Mutex ssl-stapling-refresh: using_defaults
Mutex ssl-stapling: using_defaults
PidFile: "/var/run/apache2/apache2.pid"
Define: DUMP_VHOSTS
Define: DUMP_RUN_CFG
Define: ENABLE_USR_LIB_CGI_BIN
User: name="www-data" id=33
Group: name="www-data" id=33

Topic		Replies	Views
Error renewing certificate Help	3	1096	July 14, 2021
Failed renew cert Help	9	1020	December 25, 2019
Certbot failed to authenticate, Invalid response Help	15	7461	December 17, 2022
Auto renew failed - invalid response Help	8	690	March 16, 2021
Invalid response: unauthorized (404) with certbot certificate generation (all details provided) Help	9	19958	August 2, 2019

Unauthorized invalid response from (cannot renew)

Not Found

Related topics