Cannot reissue certificates unless HTTPS redirect is first switched off

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: grahamjones.co.uk

I ran this command: No command - the certificate is set to auto-renew through the Plesk extension

It produced this output: Could not renew Let`s Encrypt certificates

My web server is (include version): Plesk 18.0.35 Update #2

The operating system my web server runs on is (include version): Ubuntu 20.4

My hosting provider, if applicable, is: IONOS

I can login to a root shell on my machine (yes or no, or I don't know): Yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): PLesk 18.0.35 Update #2

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): n/a

The error occurs if the website is set to "Redirect from http to https" in the Let's Encrypt extension in Plesk. The only solution for renewals is to switch that option off, go to the Hosting Settings and also make sure that "Permanent SEO-safe 301 redirect from HTTP to HTTPS" is switched off. Then I can reissue manually.

What's the point of having an automated renewal option if you then have to do it manually?

This always used to work OK. It no longer works Why?

1 Like

Not sure if this just the case when your webbrowser can't find the requested file, but your HTTP to HTTPS redirect seems to be incompatible with the ACME challenge. If I curl a test file (which doesn't exist) with the command curl -LIv grahamjones.co.uk/.well-known/acme-challenge/test, I get the following redirects:

First redirect to: "http://www.grahamjones.co.uk/.well-known/acme-challenge/test"
Second redirect to: "https://www.grahamjones.co.uk/2009/about/about-graham-jones/testimonials.html"

Obviously the Let's Encrypt ACME server won't find the challenge token at /2009/about/about-graham-jones/testimonials.html.

Could you put the file "test" (with random contents) under the path /.well-known/acme-challenge/ so we can rule out this erroneous redirect only takes place when the token file doesn't exist?

2 Likes

I have uploaded a test file to the acme-challenge folder.

I have no idea what the second redirect is about...! There is no such redirect on the website.

1 Like

I can confirm that what @Osiris reports above is how it currently behaves even with the file existing. In the second redirect it says to my client:

X-Redirect-By: WordPress

Which could suggest that this is a setting in your WordPress configuration somehow?

3 Likes

I can't access it: grahIamjones.co.uk/.well-known/acme-challenge/test gives me the same redirect as before, to your testimonials page.

Well, it's gotta be somewhere. This is preventing the Let's Encrypt validation server from accessing the token.

Perhaps a WordPress .htaccess is responsible? And needs an exemption for the path /.well-known/acme-challenge/

2 Likes

I think this is going to be a bit of detective work.

I have one plugin (Rankmath) that has redirections. There is no matching redirect.

I have checked the database for the site using phpMyAdmin - no matching redirect there.

I have been through the .htaccess file and there is no redirect there either...!

HOWEVER...!

Having said all this - the real problem still exists on all my other sites as well. And this problem did not exist until the past couple of weeks. Automatic renewals fail. Manual renewals are only possible if I turn off the HTTP to HTTPS redirect. That's the real issue I need solving - though thanks for pointing out the problem with one of my sites which I now have to repair...!

1 Like

Also one other thing - no new plugins have been added to the grahamjones.co.uk site and the manual renewals always worked OK for the past few years. So quite where the redirect is coming from.

I have deleted all cache, switched off all plugins, deleted the .htaccess file - but still the redirect occurs...!

1 Like

As we have been trying to get across, that is not problem at the Let's Encrypt end. Boulder (the software Let's Encrypt uses) is quite happy to follow an HTTPS redirect to find the file it is looking for. But your servers aren't just sending it to an HTTPS redirect with the file it wants at the far end, they're sending it to look at something else entirely, in the case we've looked at it was this testimonials.html and of course that isn't the file it needs to see.

Here's an insight: Why the testimonials page? Well, if I visit a URL that ends with holiday instead of test, I get a page about " Holiday web sites arrive late to the party". So I suspect there is some SEO "magic" trying to guess what your visitors might want and redirect them to it. Do you remember installing stuff like that?

2 Likes

Actually, the HTTP to HTTPS is the problem.

It happens on ALL sites even those not run on WordPress.

Also, although I get your point about potential SEO magic, that issue you have found with the holiday example is happening with ALL plugins switched off and ALL caches deleted...!

1 Like

Welcome Back to the Let's Encrypt Community, Graham :slightly_smiling_face:

This redirect:

is almost always caused by the Site Address (URL) and WordPress Address (URL) settings in the core of WordPress. This can be seen clearly by entering http://grahamjones.co.uk/.well-known/acme-challenge/test into redirect-checker.org. The page linked below explains many ways to change these settings.

1 Like

Thanks, but I still don't understand. The redirect-checker says OK for both the http and https. The Site address and WordPress address are both saying https://www.grahamjones.co.uk

If I revert the site theme, if I disable all plugins, if I delete all caches the redirect problem still exists.

I have checked the database

BUT - this is not the issue on this WordPress site....!

I am unable to manually or automatically renew any Let'sEncrypt certificates unless I switch OFF the HTTP to HTTPS redirect.

That's the issue I want solved - not the WordPress conundrum...!

1 Like

At present, I'm not even seeing a direct http to https redirect.

>>> http://grahamjones.co.uk/.well-known/acme-challenge/test

> --------------------------------------------
> 301 Moved Permanently
> --------------------------------------------

Status: 301 Moved Permanently
Code: 301
Server: nginx
Date:Mon, 07 Jun 2021 08:14:55 GMT
Content-Type: text/html
Content-Length: 162
Connection: close
Location: http://www.grahamjones.co.uk/.well-known/acme-challenge/test

>>> http://www.grahamjones.co.uk/.well-known/acme-challenge/test

> --------------------------------------------
> 301 Moved Permanently
> --------------------------------------------

Status: 301 Moved Permanently
Code: 301
Server: nginx
Date:Mon, 07 Jun 2021 08:14:56 GMT
Content-Type: text/html; charset=UTF-8
Connection:close
X-Powered-By: PleskLin
X-UA-Compatible: IE=edge
Expires: Wed, 11 Jan 1984 05:00:00 GMT
Cache-Control: no-cache, must-revalidate, max-age=0
X-Redirect-By: WordPress
Location: https://www.grahamjones.co.uk/2009/about/about-graham-jones/testimonials.html

>>> https://www.grahamjones.co.uk/2009/about/about-graham-jones/testimonials.html

> --------------------------------------------
> 200 OK
> --------------------------------------------

Status: 200 OK
Code: 200
Server: nginx
Date: Mon, 07 Jun 2021 08:14:56 GMT
Content-Type: text/html; charset=UTF-8
Connection: close
Vary: Accept-Encoding
X-Powered-By: PleskLin


So... Let's Encrypt wants to be served this:

http://grahamjones.co.uk/.well-known/acme-challenge/test

and would be OK with being served this:

http://www.grahamjones.co.uk/.well-known/acme-challenge/test

but is instead being served this (because of the WordPress redirect):

https://www.grahamjones.co.uk/2009/about/about-graham-jones/testimonials.html

Based on the presence of the X-Powered-By: PleskLin header in the second redirect, I would surmise that some aspect of the WordPress plugin/app itself in Plesk may be responsible for your wayward redirect.

1 Like

You might consider this guide for identifying wayward redirects:

1 Like

I really am very grateful for your attempts to help me resolve the problem. I had already completed all the steps in the WPBeginng tutorial anyway.

However, I get the same issue with this site (as an example) https://howtostudyonline.com/

There is only a Plesk holding page there.

But I cannot renew the certificate manually or automatically unless I switch OFF the redirect from HTTP to HTTPS.

The issue is not a WordPress problem (though I clearly have one of them to fix...!)

1 Like

The trouble with howtostudyonline.com is a different story. Let's Debug is seeing a timeout when connecting to your IPv6 address (AAAA record in your DNS). This is happening before any http to https redirect.

1 Like

Yes - I know - that's because the HTTPS divert is currently ON. When I switch it OFF that timeout does not happen.

1 Like

I'm not sure how switching off http to https redirects fixes an IPv6 functionality problem. :thinking:

1 Like

No, it doesn't make sense to me either, but that is what is happening...! Meanwhile, I am on the detective hunt for the problem with WordPress. Thanks for all your help and thoughts, I appreciate it.

1 Like

The AAAA timeout being reported by Let's Debug is happening when connecting via http, not https, so the only way I can think that an http to https redirect could be responsible is if having that redirect turned on results in somehow crippling the AAAA port 80 VirtualHost in the underlying webserver configuration (which is highly unlikely).

Timeouts are most likely caused by firewalls. If there isn't a service answering on a certain port, usually Linux servers respond with a "Connection refused". Timeouts are a sign that packets are actively dropped and firewalls are the most likely culprits to be dropping packets.

2 Likes