Certbot renewal http-01 error

Help
1

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. https://crt.sh/?q=example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: jstjesuits.org

I ran this command: certbot --dry-run renew

It produced this output:
urn:ietf:params:acme:error:unauthorized
The client lacks sufficient authorization :: Invalid response from

http://jstjesuits.org/.well-known/acme-challenge/UsjOo6dIyF00FEqN2unTD-fzwvQrFMy79qjer2eT4uE
[70.90.166.4]: "\n\n404 Not Found\n\n

Not
Found

\n<p". Skipping.

My web server is (include version):
apache Apache/2.4.29 (Ubuntu)

The operating system my web server runs on is (include version):
Ubuntu 18.04.4 LTS
My hosting provider, if applicable, is:
N/A
I can login to a root shell on my machine (yes or no, or I don't know):
yes
I'm using a control panel to manage my site (no, or provide the name and version of the control panel):

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):
certbot 0.27.0

Certbot succesfully creates /var/www/jstb/.well-known/acme-challenge/X8f87rm59NX70M35NEIQ3v6x-6XiqjARcckc9DrBY7c

But then I see in apache logs
GET /.well-known/acme-challenge/X8f87rm59NX70M35NEIQ3v6x-6XiqjARcckc9DrBY7c HTTP/1.1" 404

from Let's Encrypt validation server; +https://www.letsencrypt.org

I did this several times without the dry-run so now it says I've exceeded my limit. So am using the staging area. Ports 80,443 are open to all. nslookup to jstjesuits.org gives the right IP. Reverse lookup gives a different domain name, but I don't think thats the issue, as the letsencrypt server GET is seen in the apache logs

2 Likes
2

I'm guessing you're using the apache authenticator plugin. Probably your Apache configuration is unconventional so that the apache plugin doesn't work properly. There are two options:

  • Fix your Apache configuration to a more rational configuration
  • Use the webroot authenticator plugin.
3 Likes
3

Hi @pathma

there is an Apache (sorry), so Certbot doesn't understand your configuration.

What says

apachectl -S
3 Likes
4

Thanks for the quick reply!!
Weird things, it says its not installed. But I think I saw nginx somewhere in the logs.
I'll install nginx. OK, after install got the same error
Invalid response from http://jstjesuits.org/.well-known/acme-challenge/yvABIN_NXFqDWnRnUf_IECPD2OTKHAj46yn9D-GbqEw
My sites-enabled shows:
ServerName jstjesuits.org
<VirtualHost 70.90.166.4:80>
Redirect permanent / https://jstjesuits.org/
ServerAdmin webmaster@localhost
ServerAlias www.jstjesuits.org
DocumentRoot /var/www/jstb

1 Like
5

70.90.166.4:80 jstjesuits.org (/etc/apache2/sites-enabled/000-default.conf:2)
*:443 jstjesuits.org (/etc/apache2/sites-enabled/default-ssl.conf:2)
ServerRoot: "/etc/apache2"
Main DocumentRoot: "/var/www/html"
Main ErrorLog: "/var/log/apache2/error.log"
Mutex watchdog-callback: using_defaults
Mutex ssl-stapling-refresh: using_defaults
Mutex ssl-stapling: using_defaults
Mutex ssl-cache: using_defaults
Mutex default: dir="/var/run/apache2/" mechanism=default
Mutex mpm-accept: using_defaults
PidFile: "/var/run/apache2/apache2.pid"
Define: DUMP_VHOSTS
Define: DUMP_RUN_CFG
User: name="www-data" id=33
Group: name="www-data" id=33

6

The complete output of

apachectl -S

is required. Forget the nginx - was an own typo.

3 Likes
7

I just noticed DocRoot as reported by apachectl is different says /var/www/html
But I defined it as /var/www/jstb. Where is the other one coming from ? I did a grep -r on /etc/apache2/ and cant find anything like that

8

Personally, I'm not a fan of those IP address bound VirtualHosts.. It could have precedence above the temporary VirtualHost added by certbot. What happens if you change the IP address to an asterisk (*) in your port 80 VirtualHost?

2 Likes
9

I have a seperate section for a private IP using :80 thats why its not a * The private one has access to some programs I don't want to expose to internet

10

apachectl -S
VirtualHost configuration:
192.168.3.2:80 adminer (/etc/apache2/sites-enabled/000-default.conf:55)
70.90.166.4:80 jstjesuits.org (/etc/apache2/sites-enabled/000-default.conf:2)
*:443 jstjesuits.org (/etc/apache2/sites-enabled/default-ssl.conf:2)
ServerRoot: "/etc/apache2"
Main DocumentRoot: "/var/www/html"
Main ErrorLog: "/var/log/apache2/error.log"
Mutex ssl-stapling: using_defaults
Mutex ssl-cache: using_defaults
Mutex default: dir="/var/run/apache2/" mechanism=default
Mutex mpm-accept: using_defaults
Mutex watchdog-callback: using_defaults
Mutex ssl-stapling-refresh: using_defaults
PidFile: "/var/run/apache2/apache2.pid"
Define: DUMP_VHOSTS
Define: DUMP_RUN_CFG
User: name="www-data" id=33
Group: name="www-data" id=33

11

For your "main" site jstjesuits.org, it would be fine to use *, no?

You could always use the webroot plugin though.

1 Like
12

As @Osiris wrote: Using an ip address there may be the problem.

If the internal routing doesn't use that ip address, the vHost definition is ignored and the standard vHost is used -> error.

Use *:80.

1 Like
13

Or the webroot plugin :slight_smile:

1 Like
14

Thanks I'll try changing it. Pls provide example usage of webroot. I did try that previously but w/o luck. /usr/bin/certbot -a webroot -i apache -w /var/www/jstb renew

15

It's the same problem. If the vHost isn't used, the webroot isn't used.

16

I made the changes got the same error. THis all worked when I first got the certificate 3 mths ago.
apachectl -S
VirtualHost configuration:
192.168.3.2:80 adminer (/etc/apache2/sites-enabled/000-default.conf:56)
*:80 jstjesuits.org (/etc/apache2/sites-enabled/000-default.conf:2)
*:443 jstjesuits.org (/etc/apache2/sites-enabled/default-ssl.conf:2)
ServerRoot: "/etc/apache2"
Main DocumentRoot: "/var/www/html"
Main ErrorLog: "/var/log/apache2/error.log"
Mutex ssl-stapling: using_defaults
Mutex ssl-cache: using_defaults
Mutex default: dir="/var/run/apache2/" mechanism=default
Mutex mpm-accept: using_defaults
Mutex watchdog-callback: using_defaults
Mutex ssl-stapling-refresh: using_defaults
PidFile: "/var/run/apache2/apache2.pid"
Define: DUMP_VHOSTS
Define: DUMP_RUN_CFG
User: name="www-data" id=33
Group: name="www-data" id=33

/usr/bin/certbot -a webroot -i apache -w /var/www/jstb renew

17

Running certbot as root would create folders as root, but apache runs as www-data . Could that be an issue? CLearly the letencrypt server is producing the right URL, but getting 404. So I thought it might be a permission issue

18

That would result in a 403 error, not 404 file not found.

That should have worked if /var/www/jstb is the same path as what is configured as DocumentRoot in your Apache configuration. Are you sure it's correct? What if you placed a file with some random content in /var/www/jstb/.well-known/acme-challenge/testfile, are you able to get it through http://jstjesuits.org/.well-known/acme-challenge/testfile?

1 Like
19

Thats true about the permission. I just realised that mine is a pasword protected website, other then the login page itself. Sorry I am not a apache expert. Can I point to another location using the --webroot? Is there a pointer to using webroot, I dont understand how that works. Do I need the -i apache for that? Thanks for all your help, appreciate it!

1 Like
20

I am trying to test with a file in the hidden folder, but its just hanging. Probably have to disable the redirect to https, as the cert is expired now

1 Like