Invalid response 500 error (unauthorized) when /public_html/ is CGI context

Help
1

This error only happens when I make my HTML root a CGI context. I can install and renew certs fine if / isn't a CGI. But not sure why. Is permissions issue?

Simulating renewal of an existing certificate for www.poolghoul.com and poolghoul.com

Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
Domain: poolghoul.com
Type: unauthorized
Detail: 67.227.227.161: Invalid response from https://poolghoul.com/.well-known/acme-challenge/sRTECmkJHnh8iscr1aW23zCZQC7PV74vjcGOvmudFL4: 500

Domain: www.poolghoul.com
Type: unauthorized
Detail: 67.227.227.161: Invalid response from https://poolghoul.com/.well-known/acme-challenge/jj7lRqxl_7VuCqcfqkUi4uqYvoaacn7trHVrPo2TVWE: 500

Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded from the internet.

Failed to renew certificate www.poolghoul.com with error: Some challenges have failed.

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: poolghoul.com

I ran this command: certbot renew --dry-run

It produced this output: see above

My web server is (include version): litespeed

The operating system my web server runs on is (include version): ubuntu

My hosting provider, if applicable, is: lw

I can login to a root shell on my machine (yes or no, or I don't know): yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): certbot 2.11.0

2

Your LiteSpeed server is using a cert you got today. Did you sort out your problem?

The "500" in the error is an HTTP Internal Server error (your server). And, you can see in the error the URL started with HTTPS:// This means you redirected the HTTP request coming from the Let's Encrypt server to HTTPS successfully. But, when LE sent that HTTPS request to you it got a 500 error reply.

If you haven't sorted it yet review your LiteSpeed error logs and general config. It is best if you don't even redirect the acme-challenge from HTTP to HTTPS.

2 Likes
3

I was only able to install the cert without making / a CGI context

I need to keep turning on and off the CGI context to install certs and renew them.

I am trying now to make a CGI context for /.well-known/ to see if this helps
it has something to do with a CGI context but not sure why it matters

4

I think I know why... I must make a Static Context to map the directory which allows all files... trying this now... then static context has higher priority than the / CGI context

1 Like
5

I make a Static Context
https://poolghoul.com/.well-known/acme-challenge/abc123
This file, you can access it correctly. it shows in browser. works 100%

But now letsencrypt throws a 404 not found error on the acme files.. very confused

Domain: www.poolghoul.com
Type: unauthorized
Detail: 67.227.227.161: Invalid response from http://www.poolghoul.com/.well-known/acme-challenge/W_NhHlFwLiA19_OkZj1Kiipd4U2XDSsQdhxKoPXJO2Y: 404

6

Certbot places the challenge token file in the webroot-path that you previously used to get the cert. This path is saved in the renewal config file in /etc/letsencrypt/renewal

Does that path match the location your LiteSpeed will use for static files using this format of URL?

http://(domain)/.well-known/acme-challenge/(token)

The "404" is an HTTP Not Found so this tells me these values do not match. Your example used HTTPS. Is it the same if use HTTP?

Also check that requests to your www subdomain work same as when using the apex name.

2 Likes
7

hey, thanks for the assist. I see this in config file which seems correct:

webroot_path = /usr/local/lsws/sites/poolghoul/public_html

[[webroot_map]]
poolghoul.com = /usr/local/lsws/sites/poolghoul/public_html
www.poolghoul.com = /usr/local/lsws/sites/poolghoul/public_html

certbot -v renew --dry-run

Using the webroot path /usr/local/lsws/sites/poolghoul/public_html for all unmatched domains.
Waiting for verification...
Challenge failed for domain poolghoul.com
Challenge failed for domain www.poolghoul.com
http-01 challenge for poolghoul.com
http-01 challenge for www.poolghoul.com

Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
Domain: poolghoul.com
Type: unauthorized
Detail: 67.227.227.161: Invalid response from http://poolghoul.com/.well-known/acme-challenge/bpbCSJCXiPAeYJl7Q1NUhASYm50zfMkvkigYBbJiYwo: 404

Domain: www.poolghoul.com
Type: unauthorized
Detail: 67.227.227.161: Invalid response from http://www.poolghoul.com/.well-known/acme-challenge/SU21a8hYRCpc7lo3mrnEjFUKgl_q-7pqr-FKHgptFXY: 404

Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded from the internet.

Cleaning up challenges
Failed to renew certificate www.poolghoul.com with error: Some challenges have failed.

.well-known directory and acme-challenge directory have permission of 777 and are "pre created". the user/owner is same process webserver runs as lsadm:lsadm

2 Likes
8

Hmmm. I can see your test 'abc123' file using HTTP.

But, the "404" is definitely a reply from your server that says "Not Found"

Are you running Certbot on the same machine as that LiteSpeed?

Do you have multiple LiteSpeed systems and maybe you direct traffic to a different one?

Can you check your error and/or access logs for LiteSpeed and see if it gives a reason it replies 404?

2 Likes
9

appreciate the time trying to help with it, mike

cerbot is on the same litespeed. only a single system

I see this in error log:

2024-11-02 18:16:38.583110 [INFO] [10620] [T0] [57.103.73.199:25917#poolghoul] File not found [/usr/local/lsws/sites/poolghoul/.well-known/acme-challenge/sRTECmkJHnh8iscr1aW23zCZQC7PV74vjcGOvmudFL4:]

I feel maybe this is a permissions issue? Is it possible the file "sRTECmkJHnh8iscr1aW23zCZQC7PV74vjcGOvmudFL4" is not even getting created/dump in that folder? Hence the 404 happens b/c the file was never made in the first place?

Do you think I should try again but delete this ".well-known" direcory recursively?

x:/usr/local/lsws/sites/poolghoul# ls -alrt

drwxrwxrwx 3 lsadm lsadm 4096 Nov 2 15:29 .well-known

1 Like
10

Above was the path in your Certbot renewal profile. That is the path Certbot uses

The above is the path LiteSpeed is using. Not quite the same :slight_smile:

2 Likes
11

Apologize but don't understand. I have this static context which maps the directory

URI
/.well-known/acme-challenge/

Location:
$VH_ROOT/.well-known/acme-challenge/

I would think that makes everything work ok.

I did try changing the config to be like this (removing "public_html"):

webroot_path = /usr/local/lsws/sites/poolghoul,
[[webroot_map]]
poolghoul.com = /usr/local/lsws/sites/poolghoul
www.poolghoul.com = /usr/local/lsws/sites/poolghoul

however still the same error. All my other sites work fine and have public_html. The only difference on this site and my other domains is it has a cgi context in the webroot

12

Is this the folder you have that "abc123" test file?

/usr/local/lsws/sites/poolghoul/.well-known/acme-challenge

Try this. When it says "Waiting for verification" do NOT press Enter. Go to the above folder and see if that token file is where you expect it to be

sudo certbot renew --dry-run --debug-challenges -v
1 Like
13

It doesn't pause for some reason so can't tell but suspect it isn't creating the file.

Permissions are literally 777 recursively

/usr/local/lsws/sites/poolghoul# ls -alrt

drwxrwxrwx 3 lsadm lsadm 4096 Nov 2 19:00 .well-known
drwxrwxrwx 2 lsadm lsadm 4096 Nov 2 19:54 public_html

14

@chr0meice2 try changing directory into the .well-known with cd .well-known
then retry your ls command.

1 Like
15

I am rapidly running ls in this dir, but no file is created.

/usr/local/lsws/sites/poolghoul/.well-known/acme-challenge# ls -alrt
total 12
drwxrwxrwx 3 lsadm lsadm 4096 Nov 2 19:00 ..
-rw-r--r-- 1 root root 9 Nov 2 19:29 test.txt
drwxrwxrwx 2 lsadm lsadm 4096 Nov 2 19:29 .

16

I see the test file

$ curl -i http://poolghoul.com/.well-known/acme-challenge/test.txt
HTTP/1.1 200 OK
Connection: Keep-Alive
Keep-Alive: timeout=5, max=100
content-type: text/plain
last-modified: Sat, 02 Nov 2024 19:29:36 GMT
etag: "9-67267da0-b1fcf19206b5572e;;;"
accept-ranges: bytes
content-length: 9
date: Sat, 02 Nov 2024 20:08:55 GMT
server: LiteSpeed

asdfasdf

Edit:
Also certbot doesn't typically remove the challenge TOKEN file.

1 Like
17

yeah exactly. i don't understand why that works but certbot fails

if i use no Static CGI context at all, then i get 500 internal server error from certbot

18

Um, yes it does otherwise that folder would get very full :slight_smile:

2 Likes
19

Thanks Mike! :slight_smile:
My bad. :frowning:

1 Like
20

What does this show

sudo ls -l /etc/letsencrypt/renewal
2 Likes