Unathorized invalid response that includes HTML from the page

[KcajCrash]

My domain is: www.voltchure.com

I ran this command: sudo certbot -d voltchure.com -d www.voltchure.com --webroot -w /var/www/html/volt/ certonly --dry-run

It produced this output:

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for www.voltchure.com
Using the webroot path /var/www/html/volt for all unmatched domains.
Waiting for verification…
Cleaning up challenges
Failed authorization procedure. www.voltchure.com (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from https://voltchure.com [216.186.249.253]: “\n\n \n Index of /\n \n \n

Index of”

IMPORTANT NOTES:

• The following errors were reported by the server:

  Domain: www.voltchure.com
  Type: unauthorized
  Detail: Invalid response from https://voltchure.com
  [216.186.249.253]: "\n\n \n Index of /\n

  \n \n

  Index of"

  To fix these errors, please make sure that your domain name was
  entered correctly and the DNS A/AAAA record(s) for that domain
  contain(s) the right IP address.

My web server is (include version):

The operating system my web server runs on is (include version):
Server version: Apache/2.4.38 (Debian)
Server built: 2019-10-15T19:53:42

My hosting provider, if applicable, is:

I can login to a root shell on my machine (yes or no, or I don’t know): YES

I’m using a control panel to manage my site (no, or provide the name and version of the control panel): NO

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you’re using Certbot): certbot 0.31.0

[JuergenAuer]

Hi @KcajCrash

@KcajCrash

Failed authorization procedure. www.voltchure.com (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from https://voltchure.com [216.186.249.253]: “\n\n \n Index of /\n \n \n

see the answer of your system - or check https://check-your-website.server-daten.de/?q=voltchure.com

http + www + /.well-known/acme-challenge/random-filename is redirected to https + /, that can't work, there isn't your validation file.

Remove that wrong redirect.

[Osiris]

@JuergenAuer

Remove that wrong redirect.

Or fix it so it doesn't redirect from the HTTP URL with /path/ to HTTPS /, but to HTTPS /path/ too.

[KcajCrash]

Where is this redirect happening?

My two .conf files for the apache2 vhosts definitely do not have a redirect.

I use namecheap for the domain and thought maybe that was causing the issue, but I can’t see any option there that is redirecting to an https://.

Sorry, I am having a difficulty understanding how to correct this issue.

I’ve exhausted most of the ideas I can think of with apache2 or the domain name provider - every time I get a domain I go through this lengthy process and I’m never sure what fixes it because different solutions always seem to not work the next time

[JuergenAuer]

@KcajCrash

Where is this redirect happening?

It's your system. So you have created that redirect.

PS: Read your output:

Host	Type	IP-Address	is auth.	∑ Queries	∑ Timeout
voltchure.com	A	216.186.249.253 Largo/Florida/United States (US) - WideOpenWest Finance LLC Hostname: dynamic-216-186-249-253.knology.net	yes	1	0
	AAAA		yes
www.voltchure.com	A	192.64.119.86 Newark/New Jersey/United States (US) - Namecheap No Hostname found	yes	1	0
	AAAA		yes

Non-www and www have different ip addresses.

Change the www to your non-www ip.

[KcajCrash]

Oh my gosh! Thanks so much!

It turns out this whole time, the default “working” namecheap configuration I was using was incorrect.

For those who might encounter this, I needed TWO A Records - one for my www and the regular. Instead, as is somehow default in their template, there was a URL Redirect Record which contained the www - this can work normally for website operation, but causes validation failure for SSL.

Strangely enough, I’ve got several other domains with the URL Redirect Record that I was able to make work after several days of trying, various methods. (Standalone was even failing for me, but that had worked before, I’d even got to the point of trying to manually enter the DNS method of validation for certbot - which is also different on namecheap and difficult).

Oh well. Now I know the exact problem, what caused it and it shall plague me no more!

<3 <3 <3 All the thanks in the world, once again, fantastic community and service!