URL Rewrites Preventing Domain Validation

[MontyHu]

Hi, I’ve been using Certbot to create and renew SSL certificates on my server successfully for all websites, except for one in which the URL is rewritten to make it “pretty.” So instead of mydomain.com?page=viewer&edit=1299 it will display in the browser as mydomain.com/viewer/1299.

The problem is that when I try to set up an LE certificate for this domain using this command:

certbot certonly --webroot -d mydomain.com -w /home/mysite/www

Certbot gives me the following error:

Failed authorization procedure. mydomain.com (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://mydomain.com/.well-known/acme-challenge/HD7hKQ8sUNBjPbmLna6X_zCt_Kzzs66mTEdAX1UBqjo

I have a CentOS 7 server running Apache 2.4. These are the rewrite rules for this website located in the .htaccess file:

RewriteEngine On
RewriteCond %{HTTPS} on
RewriteCond %{HTTP_HOST} !mydomain.com [NC]

RewriteRule ^(.*)$ http://mydomain.com/$1 [R=301,L]
RewriteRule /*\.(css|txt|js|gif|png|pdf|ico|jpe?g)$ - [NC,L]
RewriteRule "^(.*)$"  "index.php?_url=$1" [QSA,L]

I found something online which suggested allowing the .well-known folder to be seen as-is and stop rewriting if that folder is being accessed, so I added a new RewriteRule line to my .htaccess file:

RewriteEngine On
RewriteCond %{HTTPS} on
RewriteCond %{HTTP_HOST} !mydomain.com [NC]

RewriteRule ^\.well-known\/acme-challenge\/ - [L]
RewriteRule ^(.*)$ http://mydomain.com/$1 [R=301,L]
RewriteRule /*\.(css|txt|js|gif|png|pdf|ico|jpe?g)$ - [NC,L]
RewriteRule "^(.*)$"  "index.php?_url=$1" [QSA,L]

However, adding RewriteRule ^\.well-known\/acme-challenge\/ - [L] doesn’t work, it produces a “too many redirects” error in the browser and the page fails to load.

I’m not really terribly familiar with Rewrite rules and was wondering if anyone else has already figured out how to allow Certbot access to .well-known on sites that use re-writing to make the URLs pretty, and how I can modify my own to work so renewals will work without having to manually create and renew certificates for this and other sites that use the same type of URL re-writing.

Thanks!

[jmorahan]

Looks like you added the line in the wrong place. Try putting it above the first RewriteCond.

(Explanation: The RewriteCond lines apply conditions to the first RewriteRule that follows them. Your original rules read like this (I’ve changed the spacing to make it clearer):

RewriteEngine On

RewriteCond %{HTTPS} on                             # if HTTPS
RewriteCond %{HTTP_HOST} !mydomain.com [NC]         # and not mydomain.com
RewriteRule ^(.*)$ http://mydomain.com/$1 [R=301,L] # then redirect and finish

RewriteRule /*\.(css|txt|js|gif|png|pdf|ico|jpe?g)$ - [NC,L]

RewriteRule "^(.*)$"  "index.php?_url=$1" [QSA,L]

If the request is over HTTPS and the hostname is not mydomain.com, then redirect to http://mydomain.com/....

You inserted the line just after the conditions, so now it reads like this:

RewriteEngine On

RewriteCond %{HTTPS} on                           # if HTTPS
RewriteCond %{HTTP_HOST} !mydomain.com [NC]       # and not mydomain.com
RewriteRule ^\.well-known\/acme-challenge\/ - [L] # then exclude acme challenge urls from rewriting

RewriteRule ^(.*)$ http://mydomain.com/$1 [R=301,L]

RewriteRule /*\.(css|txt|js|gif|png|pdf|ico|jpe?g)$ - [NC,L]

RewriteRule "^(.*)$"  "index.php?_url=$1" [QSA,L]

Now the conditions apply to the new line, instead of the redirect. If the request is over HTTPS and the hostname is not mydomain.com, then check for an ACME challenge and stop rewriting. Otherwise it continues to the next rule, which is: redirect to http://mydomain.com/....

Since there’s no longer any condition applied to the redirect, it happens on every request and you get an infinite redirect loop).

[MontyHu]

Thank you so much!

I moved the .well-known line above the first RewriteCond line and now the website works, and Certbot was able to validate the domain. Yay!

Also, thanks a lot for the detailed explanation of how the Rewrite lines work.