Https prevents certificate auto renewal - why and what to do?

Help
1

The shared cloud hosting service I use provides a useful button to deploy let’s encrypt (with no guarantees). It worked fine for me. I have https! I followed google’s advice and setup a redirect (rewrite rule) so http requests go to https. Great! some months later I got news that my certificate had expired. It turns out the host does run an auto renewal script as a cron job, but they told me it failed because my website:

"has a forced https which has prevented the Lets Encrypt from auto-renewing. Please bear in mind that due to the nature of the free third party SSL Lets Encrypt it cannot be manually or auto-renewed as long as it has https or any other redirect in place. I have temporary disabled your website https and I have renewed your Lets Encrypt and I have then re-enabled your https: https://www.sslshopper.com/ssl-checker.html#hostname=spucnottingham.org.uk "

It was kind they fixed it this time round, but I wanted to know what to do for the future. After some conversations with them I’m none the wiser, and the best we came up with is I can disable the redirect for a day on day 89, every quarter.

This seems to me unsatisfactory. My thoughts on the matter are:

  1. if google advise preventing access to plain http, then allowing it even for just 4 days a year can’t be good.

  2. if google penalise plain http, then exposure of plain http versions even for a day could be harmful. (by the way I do provide canonical pointers to https in the html headers). I have seen than many sites do run both plain http and https versions alongside, but they do this I think because of past search reputation, giving time to build a new reputation on https. In my case I just want one site.

  3. I haven’t found anything over many internet searches about https preventing auto renewal. Can anyone confirm if that’s so, and provide a low-tech explanation of why that’s so?

Standard answers:
My domain is: spucnottingham.org.uk, though I have others I want to deploy the same way.

I ran this command: n/a my host does some magic to auto renew the certificate for me, but hasn’t revealed whether they use certbot or some home grown solution

It produced this output: n/a

My web server is (include version): don’t know, but it may help if I say I’m not using cloudflare

The operating system my web server runs on is (include version): some version of linux

My hosting provider, if applicable, is: www.tsohost.com

I can login to a root shell on my machine (yes or no, or I don’t know): I can ssh to a limited zone, but not the whole tree

I’m using a control panel to manage my site (no, or provide the name and version of the control panel): tsohost’s home grown cloud panel

Thanks folks!

2

Hi,

I’m wondering if you are using cPanel Hosting or something else?

cPanel Hosting does automatically come with a feature called autossl, that feature showed up as “manage ssl / TLS status” on control panel, and indeed that doesn’t allow redirection…

Thank you

3

Please show the http to https redirect you are using.

You should allow http to just a specific folder (i.e. /.well-known/acme-challenge) while requiring https to all other folders [all year long].

4

.htaccess file in public folder:

RewriteEngine On
RewriteCond %{HTTP:X-Forwarded-Proto} !https
RewriteRule ^(.*)$ https://spucnottingham.org.uk/$1 [R=301,L]

5

It’s the host’s own home made cloud control panel

6

Try:
RewriteEngine On
ReweiteRule ^\.well-known\/acme-challenge\/ - [L]
RewriteCond %{HTTP:X-Forwarded-Proto} !https
RewriteRule ^(.*)$ https://spucnottingham.org.uk/$1 [R=301,L]

7

Hi @beanhoppo,

There is no problem at all if your domain redirects from http to https, Let's Encrypt follows redirects so that doesn't prevent to auto renew your cert.

Before trying to change your RewriteRules or something else, you should talk to your hosting provider to know what really happened because the https explanation doesn't have any sense. Also, you use Apache webserver but it is really being proxified through an nginx web server that already have rules to not send requests with the pattern /.well-known/acme-challenge/ to the Apache Server, those requests are being served by the nginx server.

$ curl -IkL http://spucnottingham.org.uk/.well-known/acme-challenge/test
HTTP/1.1 404 Not Found
Server: nginx/0.8.35
Date: Tue, 24 Jul 2018 22:07:02 GMT
Content-Type: text/html
Content-Length: 169
Connection: close

Here a normal connection to your domain (pay attention to the web server, it is Apache):

$ curl -IkL http://spucnottingham.org.uk/
HTTP/1.1 301 Moved Permanently
Date: Tue, 24 Jul 2018 22:11:39 GMT
Server: Apache
Location: https://spucnottingham.org.uk/
Connection: close
Content-Type: text/html; charset=iso-8859-1
Set-Cookie: DYNSRV=lin-10-170-0-60; path=/

HTTP/1.1 200 OK
Date: Tue, 24 Jul 2018 22:11:39 GMT
Server: Apache
Last-Modified: Sun, 22 Apr 2018 21:10:59 GMT
ETag: "668461f0-c87-56a765b991ec0"
Accept-Ranges: bytes
Content-Length: 3207
Connection: close
Content-Type: text/html
Set-Cookie: DYNSRV=lin-10-170-0-147; path=/

So, double check with your hosting provider whether this is the way they are serving the needed challenges to be able to renew your certs.

Cheers,
sahsanu

1 Like
8

Can you clarify if .well-known/acme-challenge is likely to be the real name of the path? I’d always assumed it was a proxy term and server admins would use something of their own…

9

Yes that is required in the path of the authentication check from Let’s Encrypt.

10

Thanks sahsanu
I can confirm that my public folder does not have a sub-path of ".well-known/acme-challenge/test"

However I'm not sure what I should ask my host, they no doubt want to keep their secret sauce secret, so I need your help to ask a targeted question that won't seem too intrusive.

11

@beanhoppo, the first thing you need to ask to your provider is that you need an explanation about the real problem about the reason why your certificate didn't renew automatically. The already provided explanation makes no sense:

"has a forced https which has prevented the Lets Encrypt from auto-renewing. Please bear in mind that due to the nature of the free third party SSL Lets Encrypt it cannot be manually or auto-renewed as long as it has https or any other redirect in place

To issue or renew a certificate for your domain, Let's Encrypt needs to validate your domain and in this case uses the http-01 challenge, this challenge means that you must put an specific file with an specific content in this path /.well-known/acme-challenge/herethespecificfile so once you have put that file in your web server, Let's Encrypt will try to validate it connecting to http://spucnottingham.org.uk/.well-known/acme-challenge/herethespecificfile and doesn't matter whether you are redirecting http to https, Let's Encrypt will follow that redirection so you should be able to renew your cert.

Now, your hosting provider is using nginx as a reverse proxy, in normal requests you didn't even notice there is an nginx server in the middle, all the requests are passed to the Apache server that is serving your domain BUT, not all request are being passed to Apache, the ones that are /.well-known/acme-challenge/* are being served by nginx, at this point we don't know whether nginx is configured to serve the right challenges to validate your domain nor the reason to not pass these requests to Apache. They are the only ones that knows how they have configured the process to request a certificate, put the right challenges in the right places and serve them with the right tool, in this case nginx.

So, you could contact your hosting provider and ask something like:

1.- I've talked to some guys in community.letsencrypt.org and they told me that Let's Encrypt follows redirects so I don't understand the reason why you are saying that the renewal didn't work because I'm redirecting my domain from http to https.

2.- Requests to validate my domain using http://spucnottingham.org.uk/.well-known/acme-challenge/herethespecificfile are being served by your nginx server instead of Apache server and these challenge requests are not being redirected from http to https as all the other ones for my domain so the point number 1 has even less sense, could you please explain this situation?.

Maybe when you contacted them the first time, they already fixed the conf and the current one is the right one to auto renew your certs but that is something only they can answer.

Good luck,
sahsanu

2 Likes
12

Thanks sahsanu for your comprehensive reply, I’ll report back when I have some news…

1 Like
13

I put sahsanu’s two points to the host, their reply is below, but first for context: the host provides two options to customers like me: traditional cpanel and cloud, I chose the latter.

Host:
…"Please note that our cloud platform is different than cpanel and if the certificate is renewing successfully on the cpanel with a redirection it may fail on cloud. This is a known issue and the only way to fix failed renewal is to remove the redirection. Then when you check your domains you will get “OK” and will be able to renew it manually.
Unfortunately there is nothing we can do from our end. Our cloud specialists are aware of this since the certificate was included on our platform and they are working on fixing this but I can not provide you with ETA when this bug would be fully resolved. "

So not much new detail for all of sahsanu’s effort.

I did find the following page which has mention of load balancing, I’m guessing this explains the nginx server before the Apache server that the curl commands identified above: https://help.tsohost.com/knowledge-base/article/4988

I had one more go with the host’s helpline:

Me:
“Thanks for the info. Bearing in mind that running a plain http version of a website is now poor practice being criticized in the press, there is a significant difference between the host saying that free certificate renewal is not guaranteed to work (as your pre-sales info says identically for both cpanel and cloud hosting options), and saying that it can’t work on the cloud service at all because of a know design bug that prevents good practice redirection… …In the circumstances, and given the importance to other customers, please consider putting more info / guidance on this issue on your website and help-files. I think you should press your colleagues / business partners providing the cloud platform for a fix date, as many customers will naturally look elsewhere.”

Host:
“Thank you for the feedback and apologies again for the caused inconvenience so far. Hopefully this should be fixed. Note that in the future if you are considering a paid certificate - it will be renewed on time and such issues won’t face your site.”

I think the final point is they probably have to do a manual refresh even for paying customers. So for the time being, I think those of us not paying for a certificate just have to wait for someone to develop a work around for load balanced services, while there are lists of hosts providing Let’s Encrypt, I don’t think there’s a list saying which ones have renewal mechanisms that work with https redirection. If someone is looking for a host, then the non-cloud option might be a safer gamble.

14

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.