Certbot failed to authenticate some domains (authenticator: apache). The Certificate Authority reported these problems:
Domain: physicslibrary.org
Type: unauthorized
Detail: 3.145.27.141: Invalid response from http://physicslibrary.org/.well-known/acme-challenge/IzvLh-5oCuSOZbpvkNRTdR_iyA62ek5TTnH-cwguVrM: "<html>\n <head>\n <head>\n<meta http-equiv=\"Content-Type\" content=\"text/html; charset=utf-8\" />\n<link rel=\"stylesheet\" hr"
Hint: The Certificate Authority failed to verify the temporary Apache configuration changes made by Certbot. Ensure that the listed domains point to this Apache server and that it is accessible from the internet.
My web server is (include version):
Server version: Apache/2.4.62 (Amazon Linux)
mod_perl/rewrite using
The operating system my web server runs on is (include version):
amzn2023
My hosting provider, if applicable, is:
AWS EC2
I can login to a root shell on my machine (yes or no, or I don't know):
yes
I'm using a control panel to manage my site (no, or provide the name and version of the control panel):
no
The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):
certbot 3.2.0
Reading through other posts, I have 2 leading ideas for the issue.
Something to do with my rewrite, when it goes to the /.well-known/acme-challenge .. page it shows the front page of physicslibrary.org. Played around with the Rewrite and example here in forums but so far no luck.
Permissions issue, the temporary config points to /var/lib/letsencrypt/http_challenges/ which I run certbot as root so it generates the root file/directory but apache runs as apache user, I played around a bit but I can't seem to run certbot as say apache
When it steps through the debug challenges the files all seem to be generated etc. so kind of stumped.
Thank you for the suggestions, after commenting out the rewriterule statements
Output of first command:
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Simulating a certificate request for physicslibrary.org
Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
Domain: physicslibrary.org
Type: unauthorized
Detail: 3.145.27.141: Invalid response from Physics Library "\n \n \n<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />\n<link rel="stylesheet" hr"
Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded from the internet.
Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.
Output of 2nd command:
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Simulating a certificate request for physicslibrary.org
Certbot failed to authenticate some domains (authenticator: apache). The Certificate Authority reported these problems:
Domain: physicslibrary.org
Type: unauthorized
Detail: 3.145.27.141: Invalid response from Physics Library "\n \n \n<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />\n<link rel="stylesheet" hr"
Hint: The Certificate Authority failed to verify the temporary Apache configuration changes made by Certbot. Ensure that the listed domains point to this Apache server and that it is accessible from the internet.
Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.
Hmmm. The perl handler is probably grabbing the requests. You won't need that in your port 80 (HTTP) VHost once you get a cert and setup a VirtualHost for HTTPS (port 443).
So, let's try these two changes.
First, change the port from 80 to 81 in this existing VirtualHost (the exact port doesn't matter at this stage, just not 80)
<VirtualHost *:80> # make this *:81
ServerName physicslibrary.org
Second, create a new VirtualHost with just this. We'll uncomment the 3 lines for the redirect later if this plan is suitable.
Once complete and Apache reloaded, retry the two --dry-run commands from before
I'll note I'm signing off for the night so if you don't want to leave your server in this testing state post back tomorrow. Or, maybe someone else will have suggestions.
Yeah, it was the perl I guess. I think this might be easiest. Couple steps
Remove the 3 comment lines I described for the redirect in the port 80 VHost (the IF/redirect lines)
Then run this
sudo certbot --apache -d physicslibrary.org
It will get a production cert and update the port 80 VHost with suitable redirect.
It also creates a bare-bones VHost for port 443 with the needed SSL config. You will then have to manually copy/paste your perl stuff from the saved port 81 Vhost into the port 443 VHost. Do not add any of those into the VHost for port 80. That is now just a simple redirector and handler for the ACME Challenge
When all that is done should Restart (not just reload) and test HTTPS
Woot that worked great! Thank you so much for the help, really appreciated.
One final thing in case others get here... one hold up with this last step was amazon default http conf had a sneaky default virtual host under /etc/httpd/conf.d/ssl.conf that created 'Forbidden Access' when testing the https://physicslibrary.org
httpd -t -D DUMP_VHOSTS
helped as it showed it was pointing to a different ssl.conf instead of just the le-ssl.conf
Just commented out the default virtual host and left the other config statements in there for now.. so much has changed and I am 20 years behind lol. Cheers.