404 / Invalid Response from .well-known/acme challenge, test file is fine

My domain is: foofighterslive.com

I ran this command: sudo certbot renew --dry-run

It produced this output:

Cert is due for renewal, auto-renewing...
Plugins selected: Authenticator apache, Installer apache
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for 7corners.foofighterslive.com
http-01 challenge for foofighterslive.com
http-01 challenge for player.foofighterslive.com
http-01 challenge for www.foofighterslive.com
Waiting for verification...
Cleaning up challenges
Attempting to renew cert (foofighterslive.com) from /etc/letsencrypt/renewal/foofighterslive.com.conf produced an unexpected error: Failed authorization procedure. www.foofighterslive.com (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from https://www.foofighterslive.com/.well-known/acme-challenge/FNg3ikPJU52sc3NjjEtOO3RqsDbf9AbzdDPzJTKvBic.well-known/acme-challenge/FNg3ikPJU52sc3NjjEtOO3RqsDbf9AbzdDPzJTKvBic [45.77.101.41]: "\n\n<html lang="en">\n\n\n\n\n\n Page Not Found - FooFightersLive.com\n\n\n\n <meta charset="UTF-8">\n\n", foofighterslive.com (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from https://www.foofighterslive.com/.well-known/acme-challenge/G_P7jKVPykXeOr8r64w7_VuliR5l1dnWIh3JvGgabik.well-known/acme-challenge/G_P7jKVPykXeOr8r64w7_VuliR5l1dnWIh3JvGgabik [45.77.101.41]: "\n\n<html lang="en">\n\n\n\n\n\n Page Not Found - FooFightersLive.com\n\n\n\n <meta charset="UTF-8">\n\n". Skipping.
The following certs could not be renewed:
/etc/letsencrypt/live/foofighterslive.com/fullchain.pem (failure)


** DRY RUN: simulating 'certbot renew' close to cert expiry
** (The test certificates below have not been saved.)

The following certs were successfully renewed:
/etc/letsencrypt/live/player.foofighterslive.com/fullchain.pem (success)

The following certs could not be renewed:
/etc/letsencrypt/live/foofighterslive.com/fullchain.pem (failure)
** DRY RUN: simulating 'certbot renew' close to cert expiry
** (The test certificates above have not been saved.)


1 renew failure(s), 0 parse failure(s)

IMPORTANT NOTES:

My web server is (include version):Apache/2.4.38 (Ubuntu)

The operating system my web server runs on is (include version): Ubuntu 16.04

My hosting provider, if applicable, is: Vultr

I can login to a root shell on my machine: yes

I'm using a control panel to manage my site): no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): 0.28.0


I have read some similar questions regarding this but none of the solutions seem to have worked or applied to myself. I was able to create a test file at https://www.foofighterslive.com/.well-known/acme-challenge/test.txt which I can reach in a browser, but still a 404 on the acme-challenge.

I do not have any AAAA records on the domain and as you can see the problem only seems to be with the main foofighterslive.com domain, not any of the subdomains.

I thought it may be a redirect issue but I temporarily removed my entire .htaccess file with no joy. As far as I recall I have no made any major changes to the server config in recent months.

Thanks

Well, it is a redirect issue:

osiris@client ~ $ curl -vL http://www.foofighterslive.com/.well-known/acme-challenge/FNg3ikPJU52sc3NjjEtOO3RqsDbf9AbzdDPzJTKvBic
*   Trying 45.77.101.41...
* TCP_NODELAY set
* Connected to www.foofighterslive.com (45.77.101.41) port 80 (#0)
> GET /.well-known/acme-challenge/FNg3ikPJU52sc3NjjEtOO3RqsDbf9AbzdDPzJTKvBic HTTP/1.1
> Host: www.foofighterslive.com
> User-Agent: curl/7.61.1
> Accept: */*
> 
< HTTP/1.1 302 Found
< Date: Sun, 17 Feb 2019 16:34:30 GMT
< Server: Apache
< X-Frame-Options: SAMEORIGIN
< Location: https://www.foofighterslive.com/.well-known/acme-challenge/FNg3ikPJU52sc3NjjEtOO3RqsDbf9AbzdDPzJTKvBic.well-known/acme-challenge/FNg3ikPJU52sc3NjjEtOO3RqsDbf9AbzdDPzJTKvBic
< Cache-Control: max-age=0
< Expires: Sun, 17 Feb 2019 16:34:30 GMT
< Content-Length: 356
< Content-Type: text/html; charset=iso-8859-1
(...)

See the contents of the "Location" header in the response. It just isn't correct as you can see.

Your test file attempt misleads you, because you directly went to the HTTPS site: try the HTTP site for your test file and you'll see the same erroneous redirect behaviour.

It is the HTTP to HTTPS redirect which is giving you troubles.

Hi @weeniebeenie

additional: There is a mix of 301 and 302 redirects, switch to 301.

302 means "temporarily", not permanent. So change these redirects to 301-redirects.

Your redirect looks that you have somewhere $1$1 instead of $1.

But this

means: You should have found your correct webroot. So use it:

certbot run -a webroot -w yourWebroot -d www.foofighterslive.com -d foofighterslive.com --dry-run

Later

certbot run -a webroot -w yourWebroot -d www.foofighterslive.com -d foofighterslive.com -i apache

No need to complicate things if this is "just" a redirect problem. If @weeniebeenie has a perfectly running Apache authenticator plugin when his redirect problem is fixed, I would advice to leave the authenticator as it is now.

Only if the fixed redirect issue doesn't solve his problem, I would suggest switching to the webroot authenticator.

1 Like

@Osiris , @JuergenAuer thank you for the replies. I believe I have fixed any redirection issues but still the problem persists. This is my latest results. The URL for the acme-challenge now looks correct?

Cert is due for renewal, auto-renewing...

Attempting to renew cert (foofighterslive.com) from /etc/letsencrypt/renewal/foo fighterslive.com.conf produced an unexpected error: Failed authorization procedu re. foofighterslive.com (http-01): urn:ietf:params:acme:error:unauthorized :: Th e client lacks sufficient authorization :: Invalid response from https://foofigh terslive.com/.well-known/acme-challenge/-OWJsAQNQjn4MsFYQwjVj_ojnYNcX8tCoYkI5hlg Xnc [45.77.101.41]: "\n\n<html lang="en">\n\n\n\n\n\n Page Not Found - FooFightersLive.com\n\n\n\n <meta charset="UTF-8 ">\n\n", www.foofighterslive.com (http-01): urn:ietf:params:acme:error:unauthor ized :: The client lacks sufficient authorization :: Invalid response from https ://www.foofighterslive.com/.well-known/acme-challenge/Q4xsxoIa5B-lNm_GyHQdZ9sclZ CYSwDWgYFx7_5Sioo [45.77.101.41]: "\n\n<html lang="en">\n\n\n\n \n\n Page Not Found - FooFightersLive.com\n\n\n\n <meta c harset="UTF-8">\n\n". Skipping.
The following certs could not be renewed:
/etc/letsencrypt/live/foofighterslive.com/fullchain.pem (failure)


** DRY RUN: simulating 'certbot renew' close to cert expiry
** (The test certificates below have not been saved.)

The following certs were successfully renewed:
/etc/letsencrypt/live/player.foofighterslive.com/fullchain.pem (success)

The following certs could not be renewed:
/etc/letsencrypt/live/foofighterslive.com/fullchain.pem (failure)
** DRY RUN: simulating 'certbot renew' close to cert expiry
** (The test certificates above have not been saved.)


1 renew failure(s), 0 parse failure(s)

IMPORTANT NOTES:

--

Any thoughts? In my sites-enabled/foofighterslive.com-le-ssl.conf file there are a couple of entries with the comment

Some rewrite rules in this file were disabled on your HTTPS site, because they have the potential to create redirection loops.

But ALL lines are commented out, so not operational.

I have found the solution myself, it was a rogue www ServerAlias line in the non-https conf file. Not sure how/when that was added.

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.