Encountering a redirect loop error through CertSage

That is absolutely true but how do you explain that sometimes a request gets a 302 and other times a 404?

4 Likes

The non-deterministic behavior is a bit perplexing. :thinking:

I've seen rewrite rules do some apparently pretty strange things, but they usually do those things consistently. I'm wondering if there's some resource limit (like memory) in Apache that's triggering a timeout resulting in the 404 at times due to the loop where at other times there are enough resources to reach a different limit (like number of rewrites allowed). As @rg305 always says, Apache will run at all costs (even if doing so results in maddening problems).

Update: based upon @MikeMcQ's recognition below of a key detail (that I myself pointed out earlier then failed to take into account in this post), it's unlikely that the suspect redirects are happening because of anything Apache.

5 Likes

Will @SeanHsu have access to the Apache access and error logs or do they need to ask GoDaddy for them?

Would be nice to know if the 302 is actually shown in access log.

I can issue 10 URLs back-back and get varied 302/404. It "feels" like a stuck worker we see in other contexts. Although the missing "Server: Apache" header and the just wrong 302 destination maybe makes me think of some sort of security or other service fronting Apache. Why would they behave inconsistently? No good theory :slight_smile:

5 Likes

That's a good point. I can't explain why the redirect, which is deliberately sent, doesn't have an Apache Server header line. :thinking: This also invalidates the "restart Apache" solution since the suspect redirects are clearly not coming from Apache.

5 Likes

Maybe there's some sort of WordPress plugin borking things? Based on the relative URL in the suspect redirect sample above, it's almost like something is "rebasing" the requested URL to a relative path (that matches the requested path).

This could be caused by a cPanel setting maybe, but I would expect a corresponding Server line in the redirect if that were the case.

4 Likes

A good system reboot should do wonders

3 Likes

I would like to think so, but I'm not thinking this is a temporary glitch. It looks like a static misconfiguration to me that will persist even with a reboot.

Some kind of update on the other hand, might hold sway.

4 Likes

What says?:
uptime

3 Likes

That's what I thought too but ...

5 Likes

Yes, but they [Go Papi] can.

4 Likes

Hi there,

I am another GoDaddy customer encountering a redirect loop while attempting to renew a Let'sEncrypt certificate. My domain name is andersongomes.tech.

My web server details are:

IItem Detail
Server Name sg2plcpnl0227
cPanel Version 102.0 (build 24)
Apache Version 2.4.56
PHP Version 7.4.33
MySQL Version 5.6.51-cll-lve
Architecture x86_64
Operating System linux
Shared IP Address 43.255.154.37
Path to Sendmail /usr/sbin/sendmail
Path to Perl /usr/bin/perl
Perl Version 5.10.1
Kernel Version 2.6.32-954.3.5.lve1.4.90.el6.x86_64

I have been using for around two years a Red Hat Ansible that uses the acme_certificate module to retrieve and install Let'sEncrypt certificates (source). Today, the renewal is failing with a "Redirect loop detected" error message.


I strongly suspect that the issue lies in GoDaddy's infrastructure; however, I called their Customer Service and failed to convince them that there is an issue on their end. They have recently changed something so that HTTP requests issued against their servers return HTTP 302 Found responses and same addresses as location targets before returning expected HTTP data.

To exemplify, a request to a static file that does not exist on my server is returning a HTTP 302 Found response before a HTTP 404 Not Found.

$ curl --verbose http://andersongomes.tech/.test/this-file-does-not-exist.txt
*   Trying 43.255.154.37:80...
* Connected to andersongomes.tech (43.255.154.37) port 80 (#0)
> GET /.test/this-file-does-not-exist.txt HTTP/1.1
> Host: andersongomes.tech
> User-Agent: curl/8.0.1
> Accept: */*
> 
< HTTP/1.1 302 Found
< Connection: close
< Pragma: no-cache
< cache-control: no-cache
< Location: /.test/this-file-does-not-exist.txt
< 
* Closing connection 0
$ curl --location --verbose http://andersongomes.tech/.test/this-file-does-not-exist.txt
*   Trying 43.255.154.37:80...
* Connected to andersongomes.tech (43.255.154.37) port 80 (#0)
> GET /.test/this-file-does-not-exist.txt HTTP/1.1
> Host: andersongomes.tech
> User-Agent: curl/8.0.1
> Accept: */*
> 
< HTTP/1.1 302 Found
< Connection: close
< Pragma: no-cache
< cache-control: no-cache
< Location: /.test/this-file-does-not-exist.txt
< 
* Closing connection 0
* Issue another request to this URL: 'http://andersongomes.tech/.test/this-file-does-not-exist.txt'
* Hostname andersongomes.tech was found in DNS cache
*   Trying 43.255.154.37:80...
* Connected to andersongomes.tech (43.255.154.37) port 80 (#1)
> GET /.test/this-file-does-not-exist.txt HTTP/1.1
> Host: andersongomes.tech
> User-Agent: curl/8.0.1
> Accept: */*
> 
< HTTP/1.1 404 Not Found
< Date: Sat, 20 May 2023 02:37:05 GMT
< Server: Apache
< Upgrade: h2,h2c
< Connection: Upgrade
< Accept-Ranges: bytes
< Vary: Accept-Encoding
< Content-Length: 1699
< Content-Type: text/html
< 
$ curl --verbose http://yidefaze.org/.well-known/acme-challenge/this-file-does-not-exist.txt
*   Trying 184.168.98.160:80...
* Connected to yidefaze.org (184.168.98.160) port 80 (#0)
> GET /.well-known/acme-challenge/this-file-does-not-exist.txt HTTP/1.1
> Host: yidefaze.org
> User-Agent: curl/8.0.1
> Accept: */*
> 
< HTTP/1.1 302 Found
< Connection: close
< Pragma: no-cache
< cache-control: no-cache
< Location: /.well-known/acme-challenge/this-file-does-not-exist.txt
< 
* Closing connection 0
3 Likes

That's a good find @amg1127 And, one of the rare times when someone posting on another's thread is actually helpful :slight_smile:

For your domain I issued a test request (see below) and got two 302 errors followed by many 404. But, if I waited a couple minutes I again got a 302 and then 404's and so on.

Your test is conclusively not a Let's Encrypt problem. And, it reminded me that we have seen this before with GoDaddy

It technically isn't a "redirect loop" because if you follow the 302's you will eventually get the correct response. But, Let's Encrypt is protecting itself and sees a problem when it is being redirected to the same place it just requested. It is a silly, wasteful, and slower sequence if not technically a loop. Especially when at least some times multiple 302 are issued before the 404. See below

Also, note no "Server: Apache" header in the 302 and generally a very different set of response headers between the 302 and 404. This same pattern is the same as I see with the original poster's domain.

In short, show this to GoDaddy and have them fix this. That's what worked in the other thread I linked to.

curl -I http://andersongomes.tech/ForumTest123.html
HTTP/1.1 302 Found
Connection: close
Pragma: no-cache
cache-control: no-cache
Location: /ForumTest123.html

curl -I http://andersongomes.tech/ForumTest123.html
HTTP/1.1 302 Found
Connection: close
Pragma: no-cache
cache-control: no-cache
Location: /ForumTest123.html

curl -I http://andersongomes.tech/ForumTest123.html
HTTP/1.1 404 Not Found
Date: Sat, 20 May 2023 02:50:40 GMT
Server: Apache
Upgrade: h2,h2c
Connection: Upgrade
Accept-Ranges: bytes
Vary: Accept-Encoding
Content-Length: 1699
Content-Type: text/html

curl -I http://andersongomes.tech/ForumTest123.html
HTTP/1.1 404 Not Found
Date: Sat, 20 May 2023 02:50:42 GMT
Server: Apache
Upgrade: h2,h2c
Connection: Upgrade
Accept-Ranges: bytes
Vary: Accept-Encoding
Content-Length: 1699
Content-Type: text/html
5 Likes

show this to GoDaddy and have them fix this.

I will retry doing so next week. I spent 1 hour on call trying to convince the 2nd-level support that there is an issue on their end and gave up. English is not my first language and, as a result, I've got my brain exhausted. To make things worse, GoDaddy no longer offer a ticket support system and the only way to request support from them is via phone call.

Yeah, I opted to post my report here to help other people find this thread via Google Search.
I am going to post some links that may be helpful to GoDaddy customers:

By following the two links above, I managed to observed some of @MikeMcQ's test requests in server logs and some recent Let'sEncrypt bot's attempts to retrieve domain ownership challenge responses. However, no HTTP 302 Found responses were recorded.

4 Likes

This confirms that some other component is interfering with the request and issuing the 302. Show godaddy this thread or at least the sample curl requests. That should help eliminate some of the language barrier

5 Likes

For note, I am a GoDaddy cPanel shared hosting costumer myself and I have not had any issues with these redirects when using CertSage to acquire my certificates (though that was a couple of weeks ago). It's possible that some default behavior with some product common to the two cases presented in this topic is resulting in these redirects rather than a core functionality change with GoDaddy hosting itself.

4 Likes

Thank you all (especially, @griffin, @MikeMcQ, @amg1127) very much for the continuing efforts to figure out this issue!

Following your suggestions, I happened to get some new findings.

I found the cPanel provides the recent Raw Access Logs in METRICS. So I checked them to see if there was 302. I do found 302 as below (I have hid these IP for potential privacy concerns):

...229 - - [19/May/2023:18:53:48 -0700] "GET /fr-cmt29/?variant=zh HTTP/1.1" 302 - "-" "Mozilla/5.0 (compatible; DotBot/1.2; +https://opensiteexplorer.org/dotbot; help@moz.com)" 271 0/271108
..
.229 - - [19/May/2023:18:54:30 -0700] "GET /lllib_2002_0120/?variant=zh HTTP/1.1" 302 - "-" "Mozilla/5.0 (compatible; DotBot/1.2; +https://opensiteexplorer.org/dotbot; help@moz.com)" 242 0/242551
...229 - - [19/May/2023:18:54:35 -0700] "GET /lllib_2002_1216/?variant=zh HTTP/1.1" 302 - "-" "Mozilla/5.0 (compatible; DotBot/1.2; +https://opensiteexplorer.org/dotbot; help@moz.com)" 240 0/240008
..
.229 - - [19/May/2023:18:54:39 -0700] "GET /lllib_2019_0330/?variant=zh HTTP/1.1" 302 - "-" "Mozilla/5.0 (compatible; DotBot/1.2; +https://opensiteexplorer.org/dotbot; help@moz.com)" 277 0/277220
...43 - - [19/May/2023:18:55:07 -0700] "GET /certsage.php HTTP/1.1" 200 1987 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36" 14 0/14600
..
.43 - - [19/May/2023:18:55:07 -0700] "GET /favicon.ico HTTP/1.1" 302 - "http://yidefaze.org/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36" 258 0/258180

It seems that most of the 302 were related to the plugin I use to convert language between Chinese Simplified and Chinese Traditional. Then I deactivated it. But I still encounter the redirect loop error.

But I happened to find that I can successfully acquire staging certificate via CertSage for yidefaze.org, although I still run into the redirect loop error for www.yidefaze.org. I was unaware of this difference, because I get used to acquiring certificate for both yidefaze.org and www.yidefaze.org at the same time (i.e., one domain name per line).

As I hit the rate limits today, I will tried to see if I can acquire production certificate for yidefaze.org in another day. I hope it will work.


Update: After the rate limits were gone, I have succeed in adding a Let's Encrypt certificate back to yidefaze.org via CertSage, although I still encountered the same redirect loop error three times (in acquiring production certificate for yidefaze.org) before I succeed in the fourth time. In other words, persistence is needed here. :sweat:

Update2: Then, I also succeeded in acquiring a Let's Encrypt production certificate for www.yidefaze.org via CertSage after several successive attempts. However, I have not succeeded in acquiring certificate for both yidefaze.org and www.yidefaze.org at the same time (i.e., one domain name per line). Since the rate limits come back, I will give it another try tomorrow to see if I can succeed by persistence.....lol

1 Like

Eventually you will succeed. But, this problem does not just affect you getting a cert.
It affects everyone visiting your site.

In fact, whatever the root cause it looks like it affects HTTPS requests worse than HTTP

For HTTP requests the pattern is as I described ... some 302 followed by some successful requests (was 404 not found but this morning I get 301 redirect to HTTPS). But, waiting a minute or so again sees 302's followed by success.

For HTTPS it looks worse. I get "connection reset by peer" errors instead of 302. You really should show this thread to GoDaddy. People using your website will not get good results

I consistently reproduce below results. After the 404 if I wait a minute or two I get more "reset by peer" errors.

curl -i https://yidefaze.org/ForumTestHTTPSagain2.html
curl: (35) OpenSSL SSL_connect: Connection reset by peer in connection to yidefaze.org:443

curl -i https://yidefaze.org/ForumTestHTTPSagain2.html
curl: (35) OpenSSL SSL_connect: Connection reset by peer in connection to yidefaze.org:443

curl -i https://yidefaze.org/ForumTestHTTPSagain2.html
HTTP/2 404
x-powered-by: PHP/7.4.33
x-ua-compatible: IE=edge
expires: Wed, 11 Jan 1984 05:00:00 GMT
cache-control: no-cache, must-revalidate, max-age=0
link: <https://yidefaze.org/wp-json/>; rel="https://api.w.org/"
vary: Accept-Encoding
content-type: text/html; charset=UTF-8
date: Sat, 20 May 2023 14:53:51 GMT
server: Apache
4 Likes

Food for thought:

4 Likes

Thank you, MikeMcQ, for more tests and suggestion! You're very generous!

I really succeed in acquiring and installing certificate (via CertSage) for both yidefaze.org and www.yidefaze.org today after several failed attempts. So it seems that my site has recovered to its former state. Nevertheless, I acknowledge that the 302 redirect problem still affects everyone visiting my site. I suspect this problem has been there for a long time before I encountered this error through CertSage a few days ago.

this morning I get 301 redirect to HTTPS

This was because CertSage has successfully enabled the forced HTTP -> HTTPS redirect as mentioned by griffin in a previous post:

When you use the button in CertSage to install your certificate into cPanel, CertSage enables the builtin HTTP->HTTPS redirect in cPanel, which is implemented at the vHost level, which is much more efficient and less error-prone than using rewrites.

And thank you, griffin, for sharing that helpful link for thought. I have read it through, although I am still unable to troubleshoot the 302 code error in my website. Since too much time has been spent on this issue, I would like to give up for now. At least, I have Let's Encrypt certificate reinstalled for my site....

3 Likes

I have given up trying to resolve my issue.
Again, I failed to convince GoDaddy's customer service that there is a problem on their end.

I will need to move my domain, website and e-mail to another provider to get my issue with Let'sEncrypt certificates resolved. I hope that I am able to find a provider with a customer service that understand RFC2616.

1 Like