Virtualmin: Lets Encrypt Web Based Validation failed

Help
1

I’m getting the below error on this domain only. I have other domains on this box (shared IP) that all work without issue.

I had assumed it was because i orignially changed it from a longer domain name to a shorter one (renamed it within virtualmin). So today i exported the wordpress DB, downloaded the contents files. Then deleted the account from virtualmin and re-created from a-fresh. Re-installed wordpress from the installer script within virtualmin, uploded content files from previous backup and imported DB.

From reading other threads this seems to be maybe some sort of redirect?. I’ve removed the default .htaccess for wordpress to stop the re-direct and there are no “Website redirects” configured within Virtualmin.

I’m at a bit of a loss where to check for where this redirect is, if it is even that.

Any advice or guidance would be appriecahted so i can track this problem down (since its currently throwing cert errors as its using self-signed certs)

My domain is:
sentinelfs.co

I ran this command:
Using webUI within Virtualmin > Server configuration > SSL Certificatie > Lets Encrypt

It produced this output:

Requesting a certificate for sentinelfs.co, www.sentinelfs.co from Let’s Encrypt …
… request failed : Web-based validation failed : Failed to request certificate :
Traceback (most recent call last):
File “/usr/share/webmin/webmin/acme_tiny.py”, line 198, in
main(sys.argv[1:])
File “/usr/share/webmin/webmin/acme_tiny.py”, line 194, in main
signed_crt = get_crt(args.account_key, args.csr, args.acme_dir, log=LOGGER, CA=args.ca, disable_check=args.disable_check, directory_url=args.directory_url, contact=args.contact)
File “/usr/share/webmin/webmin/acme_tiny.py”, line 143, in get_crt
raise ValueError(“Wrote file to {0}, but couldn’t download {1}: {2}”.format(wellknown_path, wellknown_url, e))
ValueError: Wrote file to /home/sentinelfs/public_html/.well-known/acme-challenge/T5d2cmSY9zzCjdk36iPcLUA85btXXSrgyFWihbMSUiA, but couldn’t download http://sentinelfs.co/.well-known/acme-challenge/T5d2cmSY9zzCjdk36iPcLUA85btXXSrgyFWihbMSUiA: Error:
Url: http://sentinelfs.co/.well-known/acme-challenge/T5d2cmSY9zzCjdk36iPcLUA85btXXSrgyFWihbMSUiA
Data: None
Response Code: None
Response: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:661)>

My web server is (include version):
Apache 2.4.25

The operating system my web server runs on is (include version):
Debian 9

My hosting provider, if applicable, is:
Self-hosted, Dedicated hardware.

I can login to a root shell on my machine (yes or no, or I don’t know):
Yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel):
Virtualmin - V6.08

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you’re using Certbot):
Not using Certbot CLI

2

See how your redirection is performed:

% curl -kIL http://sentinelfs.co/.well-known/acme-challenge/T5d2cmSY9zzCjdk36iPcLUA85btXXSrgyFWihbMSUiA
HTTP/1.1 302 Found
Date: Mon, 23 Mar 2020 15:33:21 GMT
Server: Apache/2.4.25
Location: https://sentinelfs.co/index.php
Content-Type: text/html; charset=iso-8859-1

HTTP/1.1 301 Moved Permanently
Date: Mon, 23 Mar 2020 15:33:21 GMT
Server: Apache/2.4.25
X-Redirect-By: WordPress
Location: https://sentinelfs.co/
Content-Type: text/html; charset=UTF-8

HTTP/1.1 200 OK
Date: Mon, 23 Mar 2020 15:33:21 GMT
Server: Apache/2.4.25
Link: <https://sentinelfs.co/wp-json/>; rel="https://api.w.org/"
Content-Type: text/html; charset=UTF-8

You need to redirect to http -> https without removing the requested url.

3

Or handle the authentication requests via HTTP (without redirection).

1 Like
4

Great, thank you for the reply. So its something wordpress has decided to do for website redirect. Interesting. Would explain why the problem followed me when i re-installed and updated.

I’ve re-instaged the .htaccess wordpress creates. I’m fairly techy but unsure how best to modify this:

BEGIN WordPress

RewriteEngine On RewriteBase / RewriteRule ^index\.php$ - [L] RewriteCond %{REQUEST_FILENAME} !-f RewriteCond %{REQUEST_FILENAME} !-d RewriteRule . /index.php [L]

END WordPress

Suggestions?

5

before that, you should add something like:

## Force https
RewriteEngine On
RewriteBase /
RewriteCond %{HTTP:X-Forwarded-Proto} !https
RewriteCond %{HTTPS} off
RewriteRule ^(.*)$ https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301]
7

Ok, thanks for the that. I edited the .htaccess and ensured that what you posted was the only line items in there. However i’m still getting the error.

Did i interpret what you said incorrectly?

sentinelfs@arkpweb:~/public_html$ cat .htaccess

Force https

RewriteEngine On
RewriteBase /
RewriteCond %{HTTP:X-Forwarded-Proto} !https
RewriteCond %{HTTPS} off
RewriteRule ^(.*) https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301] sentinelfs@arkpweb:~/public_html

Traceback (most recent call last):
File “/usr/share/webmin/webmin/acme_tiny.py”, line 198, in
main(sys.argv[1:])
File “/usr/share/webmin/webmin/acme_tiny.py”, line 194, in main
signed_crt = get_crt(args.account_key, args.csr, args.acme_dir, log=LOGGER, CA=args.ca, disable_check=args.disable_check, directory_url=args.directory_url, contact=args.contact)
File “/usr/share/webmin/webmin/acme_tiny.py”, line 143, in get_crt
raise ValueError(“Wrote file to {0}, but couldn’t download {1}: {2}”.format(wellknown_path, wellknown_url, e))
ValueError: Wrote file to /home/sentinelfs/public_html/.well-known/acme-challenge/T5d2cmSY9zzCjdk36iPcLUA85btXXSrgyFWihbMSUiA, but couldn’t download http://sentinelfs.co/.well-known/acme-challenge/T5d2cmSY9zzCjdk36iPcLUA85btXXSrgyFWihbMSUiA: Error:
Url: http://sentinelfs.co/.well-known/acme-challenge/T5d2cmSY9zzCjdk36iPcLUA85btXXSrgyFWihbMSUiA
Data: None
Response Code: None
Response: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:661)>

8

Your redirect is working:

% curl -kIL http://sentinelfs.co/.well-known/acme-challenge/T5d2cmSY9zzCjdk36iPcLUA85btXXSrgyFWihbMSUiA:
HTTP/1.1 301 Moved Permanently
Date: Mon, 23 Mar 2020 17:29:17 GMT
Server: Apache/2.4.25
Location: https://sentinelfs.co/.well-known/acme-challenge/T5d2cmSY9zzCjdk36iPcLUA85btXXSrgyFWihbMSUiA:
Content-Type: text/html; charset=iso-8859-1

HTTP/1.1 404 Not Found
Date: Mon, 23 Mar 2020 17:29:17 GMT
Server: Apache/2.4.25
Content-Type: text/html; charset=iso-8859-1

but your client wants you to serve that directory with a valid certificate. Or over http. We can do that.

Add

RewriteCond %{REQUEST_URI} !^.well-known/acme-challenge

among the other RewriteConds

(and test it, because I haven’t)

9

Thank you. Done that, but same problem.

I manually tried to browser to the URL but got page not found.

Would i expect to see this directory within my folder structure?

root@arkpweb:/home/sentinelfs/public_html# ls -lh .well-known
ls: cannot access ‘.well-known’: No such file or directory

Hm, no, i’d assume not, none of my other working domains (with https+letsencrypt) have that. An example being https://306oc.co.uk/forum/

root@arkpweb:/home/three06oc/public_html# ls -lh .well-known
ls: cannot access ‘.well-known’: No such file or directory.

There is clearly something i’m not understanding here. Does that well-known url/folder path only get generated at the point in time for it performing the authentication or something?

10

Probably, yes.

But that last line has just started to take effect. I didn’t when I started responding. — and now it went back

11

Correct:

% curl -IL http://sentinelfs.co/.well-known/acme-challenge/T5d2cmSY9zzCjdk36iPcLUA85btXXSrgyFWihbMSUiA:
HTTP/1.1 404 Not Found
Date: Mon, 23 Mar 2020 17:56:06 GMT
Server: Apache/2.4.25
Content-Type: text/html; charset=iso-8859-1

Incorrect (not really, only for acme-tiny):

% curl -IL http://sentinelfs.co/.well-known/acme-challenge/T5d2cmSY9zzCjdk36iPcLUA85btXXSrgyFWihbMSUiA:
HTTP/1.1 301 Moved Permanently
Date: Mon, 23 Mar 2020 17:57:29 GMT
Server: Apache/2.4.25
Location: https://sentinelfs.co/.well-known/acme-challenge/T5d2cmSY9zzCjdk36iPcLUA85btXXSrgyFWihbMSUiA:
Content-Type: text/html; charset=iso-8859-1

curl: (60) SSL certificate problem: self signed certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

% curl -kIL http://sentinelfs.co/.well-known/acme-challenge/T5d2cmSY9zzCjdk36iPcLUA85btXXSrgyFWihbMSUiA:
HTTP/1.1 301 Moved Permanently
Date: Mon, 23 Mar 2020 18:03:08 GMT
Server: Apache/2.4.25
Location: https://sentinelfs.co/.well-known/acme-challenge/T5d2cmSY9zzCjdk36iPcLUA85btXXSrgyFWihbMSUiA:
Content-Type: text/html; charset=iso-8859-1

HTTP/1.1 404 Not Found
Date: Mon, 23 Mar 2020 18:03:09 GMT
Server: Apache/2.4.25
Content-Type: text/html; charset=iso-8859-1
12

The directories are added and removed as needed.

13

@9peppe, you many need to tell him exactly where to place that line.

@fooby, please show the .htaccess file as it is now.

1 Like
14

Will do,

he said ensure its with the other ReWriteCond.

However, looking at it now, i’m thinking it needs to be higher up the chain?

Force https

RewriteEngine On
RewriteBase /
RewriteCond %{HTTP:X-Forwarded-Proto} !https
RewriteCond %{HTTPS} off
RewriteCond %{REQUEST_URI} !^.well-known/acme-challenge
RewriteRule ^(.*)$ https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301]

16

That looks like it should do what you require…
So there may be some other place where redirection is taking place.
Easy enough to test, rename the .htaccess file or remove that redirection from it.
And see if redirection is still happening.
If so, then Virtualmin (or some other code) may be preceding you.

17

At some time, for a minute or so, it worked. What did you try?

18

i stole another .htaccess file from a working wordpress site (that has letsencrypt working) but then it still failed to generate SSL with the same error, then i swapped it back.

I’ve removed the .htaccess as of now.

adam@ARKP-RYZEN:~$ curl -kIL http://sentinelfs.co/.well-known/acme-challenge/T5d2cmSY9zzCjdk36iPcLUA85btXXSrgyFWihbMSUiA:
HTTP/1.1 404 Not Found
Date: Mon, 23 Mar 2020 18:27:53 GMT
Server: Apache/2.4.25
Content-Type: text/html; charset=iso-8859-1

This is what i get. It still fails.

Really appreciate the help on this one. Fully accept this is some sort of weird system setup. Although i dont tend to mess with it very much so i’m confused as to how it is happening.

19

but you get a 404 over http, not a redirect. The error you get from the panel must be different.

20

apologies thought i included that, clearly didnt. Looks the same to my un-trained eye.

Traceback (most recent call last):
File “/usr/share/webmin/webmin/acme_tiny.py”, line 198, in
main(sys.argv[1:])
File “/usr/share/webmin/webmin/acme_tiny.py”, line 194, in main
signed_crt = get_crt(args.account_key, args.csr, args.acme_dir, log=LOGGER, CA=args.ca, disable_check=args.disable_check, directory_url=args.directory_url, contact=args.contact)
File “/usr/share/webmin/webmin/acme_tiny.py”, line 143, in get_crt
raise ValueError(“Wrote file to {0}, but couldn’t download {1}: {2}”.format(wellknown_path, wellknown_url, e))
ValueError: Wrote file to /home/sentinelfs/public_html/.well-known/acme-challenge/T5d2cmSY9zzCjdk36iPcLUA85btXXSrgyFWihbMSUiA, but couldn’t download http://sentinelfs.co/.well-known/acme-challenge/T5d2cmSY9zzCjdk36iPcLUA85btXXSrgyFWihbMSUiA: Error:
Url: http://sentinelfs.co/.well-known/acme-challenge/T5d2cmSY9zzCjdk36iPcLUA85btXXSrgyFWihbMSUiA
Data: None
Response Code: None
Response: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:661)>

21

Ok, I understand what’s going on. It’s “remembering” the 301 permanent redirect.

How do we clear that cache?

22

hm, interesting. Because at every stage i’ve restarted the apache service. I’ll look into where this cache may be.