How to renew a redirecting subdomain?

kghbln · August 1, 2021, 4:00pm

I have a subdomain https://jenny.besch.ws which is redirecting to Anmeldung erforderlich – Besch with a permanent redirect. When trying to renew the cert for the subdomain I get challenge errors

Attempting to renew cert (jenny.besch.ws) from /etc/letsencrypt/renewal/jenny.besch.ws.conf produced an unexpected error: Failed authorization procedure. jenny.besch.ws (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from https://www.besch.ws/Jenny:Besch.well-known/acme-challenge/QJ6OweP1O9xhFGn2tUHl8RNRGnWVNjfNfy77QWE-8xs [2001:67c:1400:2180::1]: "<!DOCTYPE html>\n<html class=\"client-nojs\" lang=\"de\" dir=\"ltr\" version=\"HTML+RDFa 1.0\">\n<head>\n<meta charset=\"UTF-8\"/>\n<title>Anm". Skipping.

In the end not really a surprise. But how to renew?

I tried a couple of rewrite rules which should fire up correctly just for the challenge and use https://www.besch.ws/.well-known/acme-challenge/.... , but in the end I always get errors. Moreover the subdomain redirection fires up incorrectly for non challenge requests too.

Example:

RewriteEngine on
RewriteRule ^.well-known/acme-challenge/ - [L]
RewriteCond %{HTTPS} off
RewriteRule ^(.*)$ https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301]

which I found here in some thread but this ends in disaster, too.

In the end I need a rewrite rule that takes the challenge for jenny.besch.ws and takes it to www.besch.ws and all es good.

It will be nice to be able to certify a redirecting subdomain. I understand that it is not possible to redirect a https site without a cert even if it is just a simple redirect. Any help appreciated.

rg305 · August 1, 2021, 7:38pm

I'll do my best with the little that has been said/shown.

Both names (jenny.besch.ws & www.besch.ws) resolve to the same IPs:

Name:      jenny.besch.ws
Addresses: 2001:67c:1400:2180::1
           217.197.83.171

Name:      www.besch.ws
Addresses: 2001:67c:1400:2180::1
           217.197.83.171

So, I don't really understand the need to redirect the challenge requests (to the same site - wasted step).
And also must point out the obvious:
LE prefers IPv6 over IPv4 when present (as is the case here).
But the IPv6 address doesn't return the expected (similar to IPv4); as shown here:
[abbreviated outputs for immediate clarity]

curl -Iki6 http://www.besch.ws/.well-known/acme-challenge/test-file-1234
HTTP/1.1 403 Forbidden

curl -Iki4 http://www.besch.ws/.well-known/acme-challenge/test-file-1234
HTTP/1.1 301 Moved Permanently
Location: https://www.besch.ws/.well-known/acme-challenge/test-file-1234

curl -Iki6 http://jenny.besch.ws/.well-known/acme-challenge/test-file-1234
HTTP/1.1 403 Forbidden

curl -Iki4 http://jenny.besch.ws/.well-known/acme-challenge/test-file-1234
HTTP/1.1 301 Moved Permanently
Location: http://www.besch.ws/.well-known/

curl -Iki6 http://www.besch.ws/
HTTP/1.1 403 Forbidden

curl -Iki4 http://www.besch.ws/
HTTP/1.1 301 Moved Permanently
Location: https://www.besch.ws/

curl -Iki6 http://jenny.besch.ws/
HTTP/1.1 403 Forbidden

curl -Iki4 http://jenny.besch.ws/
HTTP/1.1 301 Moved Permanently
Location: http://www.besch.ws/.well-known/

So, it seems that ALL IPv6 connections are presently failing with "403 Forbidden".
You can remove the IPv6 ("AAAA") record from your DNS zone.
Or, if possible, fix the system so that it can serve the same content via both IPv6 and IPv4.

rg305 · August 1, 2021, 7:39pm

In review, one of the redirects isn't correct:

curl -Iki4 http://jenny.besch.ws/.well-known/acme-challenge/test-file-1234
HTTP/1.1 301 Moved Permanently
Date: Sun, 01 Aug 2021 19:38:53 GMT
Server: Apache
Location: http://www.besch.ws/.well-known/
Content-Type: text/html; charset=iso-8859-1

curl -Iki4 http://jenny.besch.ws/
HTTP/1.1 301 Moved Permanently
Date: Sun, 01 Aug 2021 19:40:19 GMT
Server: Apache
Location: http://www.besch.ws/.well-known/
Content-Type: text/html; charset=iso-8859-1

The redirected address has mangled the paths.

kghbln · August 2, 2021, 8:06pm

Thanks a lot for looking at this.

I believed that it may perhaps help. Something like this was suggested in other posts. I do not insist. Anyhow without it it does not work and this is what brought me here.

Good to know. I was not aware. However this is not the issue here. I have a couple of websites on the same server with the very same AAAA records for them and the challenges work perfect. Moreover ...

Well, I cannot explain. Once I let jenny.besch.ws use the webroot with some html doc the verification works, even without a presumably borked IPv6 setup. Also it does not appear to be totally borked since the response from IPv6 goes to correct spot for login to the website: The client lacks sufficient authorization :: Invalid response from Anmeldung erforderlich – Besch [2001:67c:1400:2180::1]:

kghbln · August 2, 2021, 8:15pm

Let's restart.

I have the following redirect for <jenny.besch.ws> both for 80 and 443:

RedirectPermanent / https://www.besch.ws/Jenny:Besch`

When certbot tries to perform the challenge it derails with:

Attempting to renew cert (jenny.besch.ws) from /etc/letsencrypt/renewal/jenny.besch.ws.conf produced an unexpected error: Failed authorization procedure. jenny.besch.ws (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from Anmeldung erforderlich – Besch [2001:67c:1400:2180::1]: "\n<html class="client-nojs" lang="de" dir="ltr" version="HTML+RDFa 1.0">\n\n<meta charset="UTF-8"/>\nAnm". Skipping.

Probably because <www.besch.ws> is handled by some rewrite rules which should not be applied when certbot challenges. These are to be ignored by cerbot:

RewriteEngine on

RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} !-f
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} !-d
RewriteRule ^(.*)$ %{DOCUMENT_ROOT}/w/index.php [L]
RewriteRule ^/*$ %{DOCUMENT_ROOT}/w/index.php [L]

RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} !-f
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} !-d
RewriteRule ^/?w/images/thumb/[0-9a-f]/[0-9a-f][0-9a-f]/([^/]+)/([0-9]+)px-.*$ %{DOCUMENT_ROOT}/w/thumb.php?f=$1&width=$2 [L,QSA,B]

RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} !-f
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} !-d
RewriteRule ^/?w/images/thumb/archive/[0-9a-f]/[0-9a-f][0-9a-f]/([^/]+)/([0-9]+)px-.*$ %{DOCUMENT_ROOT}/w/thumb.php?f=$1&width=$2&archived=1 [L,QSA,B]

To cut it short. How to I get certbot to ignore the rewrite rules for <www.besch.ws>? I figured that putting some rewrite rules to <jenny.besch.ws> just for certbot could do the trick but I am not sure. Perhaps some rewrite rule just for certbot needs to be added to <www.besch.ws> instead of fiddling with <jenny.besch.ws>.

Instead of calling https://www.besch.ws/Jenny:Besch.well-known/acme-challenge/... only https://www.besch.ws/.well-known/acme-challenge/... should be called.

rg305 · August 2, 2021, 9:08pm

If you show the complete vhost config, we might make heads or tails of it.

The redirection has changed, but it doesn't look like it will work this way either:

curl -Iki4 http://jenny.besch.ws/.well-known/acme-challenge/test-file-1234
HTTP/1.1 301 Moved Permanently
Date: Mon, 02 Aug 2021 21:06:23 GMT
Server: Apache
Location: https://www.besch.ws/Jenny:Besch.well-known/acme-challenge/test-file-1234
Content-Type: text/html; charset=iso-8859-1

curl -Iki4 http://jenny.besch.ws/
HTTP/1.1 301 Moved Permanently
Date: Mon, 02 Aug 2021 21:06:38 GMT
Server: Apache
Location: https://www.besch.ws/Jenny:Besch
Content-Type: text/html; charset=iso-8859-1

curl -Iki6 http://jenny.besch.ws/.well-known/acme-challenge/test-file-1234
HTTP/1.1 301 Moved Permanently
Date: Mon, 02 Aug 2021 21:07:43 GMT
Server: Apache
Location: https://www.besch.ws/Jenny:Besch.well-known/acme-challenge/test-file-1234
Content-Type: text/html; charset=iso-8859-1

curl -Iki6 http://jenny.besch.ws/
HTTP/1.1 301 Moved Permanently
Date: Mon, 02 Aug 2021 21:07:50 GMT
Server: Apache
Location: https://www.besch.ws/Jenny:Besch
Content-Type: text/html; charset=iso-8859-1

IPv6 and IPv4 seem to do the same thing now - so that's an improvement.

kghbln · August 3, 2021, 3:49pm

I guess before the redirection stuff I tried to add for cerbot caused pain.

These are the tow VirtualHosts. Not much more info I guess.
<www.besch.ws>

	SSLStaplingCache shmcb:${APACHE_RUN_DIR}/ssl_ocache(128000)

	<VirtualHost *:443>
		ServerAdmin contact@example.org
		ServerName www.besch.ws

		DocumentRoot /var/www/htdocs/mw/17609/

		<Directory />
			Options FollowSymLinks
			AllowOverride None
		</Directory>

		<Directory /var/www>
			Options -Indexes
		</Directory>

		<Directory /var/www/htdocs/mw/17609>
			Options FollowSymLinks MultiViews
			AllowOverride None
			Require all granted
		</Directory>

		<Directory /var/www/htdocs/mw/17609/w/images>
			AllowOverride None
			AddType text/plain .html .htm .phtml .shtml .php .php3 .php4 .lua .pl .py .rb
			php_flag engine off
		</Directory>

		<IfModule mod_headers.c>
			Header always set Referrer-Policy "strict-origin-when-cross-origin"
			Header always set X-XSS-Protection "1; mode=block"
			Header always set Strict-Transport-Security "max-age=31536000"
			Header unset ETag
		</IfModule>

		FileETag None

		<IfModule mod_alias.c>
			RedirectMatch 404 "^.*/(docs|mw-config|tests)/(.+?)(\.[^.]*$|$)"
		</IfModule>

		<IfModule mod_rewrite.c>
			RewriteEngine on

			RewriteRule ^/?index.php(/.*)?$ %{DOCUMENT_ROOT}/w/index.php [L]
			RewriteRule ^/?$ %{DOCUMENT_ROOT}/w/index.php [L]

			RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} !-f
			RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} !-d
			RewriteRule ^/?w/images/thumb/[0-9a-f]/[0-9a-f][0-9a-f]/([^/]+)/([0-9]+)px-.*$ %{DOCUMENT_ROOT}/w/thumb.php?f=$1&width=$2 [L,QSA,B]

			RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} !-f
			RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} !-d
			RewriteRule ^/?w/images/thumb/archive/[0-9a-f]/[0-9a-f][0-9a-f]/([^/]+)/([0-9]+)px-.*$ %{DOCUMENT_ROOT}/w/thumb.php?f=$1&width=$2&archived=1 [L,QSA,B]
		</IfModule>

		SSLEngine on
		SSLUseStapling on
		SSLStaplingResponderTimeout 5
		SSLStaplingReturnResponderErrors off
		SSLCertificateFile /etc/letsencrypt/live/jenny.besch.ws/fullchain.pem
		SSLCertificateKeyFile /etc/letsencrypt/live/jenny.besch.ws/privkey.pem
		SSLCipherSuite EECDH+AESGCM:EDH+AESGCM

		ErrorDocument 400 https://www.besch.ws/
		ErrorDocument 403 https://www.besch.ws/
		ErrorDocument 404 https://www.besch.ws/

		ErrorLog ${APACHE_LOG_DIR}/error/17609-error.log

		LogLevel warn

		CustomLog ${APACHE_LOG_DIR}/access/17609-access.log combined
	</VirtualHost>
	<VirtualHost *:443>
		ServerAdmin contact@example.org
		ServerName besch.ws

		RedirectPermanent / https://www.besch.ws/

		SSLEngine on
		SSLUseStapling on
		SSLStaplingResponderTimeout 5
		SSLStaplingReturnResponderErrors off
		SSLCertificateFile /etc/letsencrypt/live/jenny.besch.ws/fullchain.pem
		SSLCertificateKeyFile /etc/letsencrypt/live/jenny.besch.ws/privkey.pem
		SSLCipherSuite EECDH+AESGCM:EDH+AESGCM

		ErrorLog ${APACHE_LOG_DIR}/error/17609-error.log

		LogLevel warn

		CustomLog ${APACHE_LOG_DIR}/access/17609-access.log combined
	</VirtualHost>
</IfModule>

<jenny.besch.ws>:

<IfModule mod_ssl.c>
	SSLStaplingCache shmcb:${APACHE_RUN_DIR}/ssl_ocache(128000)

	<VirtualHost *:443>
		ServerAdmin kontakt@wikihoster.net
		ServerName jenny.besch.ws

		RedirectPermanent / https://www.besch.ws/Jenny:Besch

		SSLEngine on
		SSLUseStapling on
		SSLStaplingResponderTimeout 5
		SSLStaplingReturnResponderErrors off
		SSLCertificateFile /etc/letsencrypt/live/jenny.besch.ws/fullchain.pem
		SSLCertificateKeyFile /etc/letsencrypt/live/jenny.besch.ws/privkey.pem
		SSLCipherSuite EECDH+AESGCM:EDH+AESGCM

		ErrorLog ${APACHE_LOG_DIR}/error/17609-error.log

		LogLevel warn

		CustomLog ${APACHE_LOG_DIR}/access/17609-access.log combined
	</VirtualHost>
</IfModule>

I still suspect that we need to add some redirection to <www.besch.ws> which is just firing up for certbot. Unfortunately I am not the Apache redirect creation guy, I'm afraid.

rg305 · August 3, 2021, 5:45pm

You have to include a Rewrite Condition that handles the challenge requests from LE (per request from certbot).
Adding in something like this might work:

RewriteCond %{REQUEST_URI} ^\.well\-known
RewriteRule - [L]

[immediately after line: RewriteEngine on]

rg305 · August 3, 2021, 5:47pm

And this redirection still seems broken/unnecessary:

[abbreviated for clarity]

curl -Iki https://jenny.besch.ws/.well-known/acme-challenge/Test-File-1234
HTTP/1.1 301 Moved Permanently
Location: https://www.besch.ws/Jenny:Besch.well-known/acme-challenge/Test-File-1234

kghbln · August 3, 2021, 7:10pm

Yes this worked. certbot renew --dry-run completes without error:

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/jenny.besch.ws.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cert not due for renewal, but simulating renewal for dry run
Plugins selected: Authenticator webroot, Installer None
Renewing an existing certificate

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
new certificate deployed without reload, fullchain is
/etc/letsencrypt/live/jenny.besch.ws/fullchain.pem
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

However now when I access https://jenny.besch.ws I get redirected to https://www.besch.ws/.well-known/ instead of https://www.besch.ws/Jenny:Besch. Thus I still have an issue. The redirect is not unnecessary since this is exactly what the subdomain should do, point whatever calls it to https://www.besch.ws/Jenny:Besch.

This is basically the reason why I am here. Point certbot to the correct challenge location and all the rest to the redirection location. This is exactly my dilemma.

kghbln · August 3, 2021, 7:23pm

Hmm, hold on. Perhaps this was a browser cache issue. Now <jenny.besch.ws> actually also does redirect to https://www.besch.ws/Jenny:Besch - which is good. If this is true this will have resolved my issue.

kghbln · August 3, 2021, 8:10pm

Affirmative. It works now. Wow, cool. Just two tiny extra lines. Thanks a ton @rg305 !!! Your help is very much appreciated.

rg305 · August 3, 2021, 8:14pm

Glad to hear that
Cheers from Miami

#FreeCuba

system · September 2, 2021, 8:15pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.