Googlebot gives Bad Gateway 502 but website works in Firefox and Chrome

Migrating my website from http to https using Certbot gave me some issues with the Google search engine. Somehow it is not showing the site other than 502 error. I used Certbot auto install/configure to configure my NginX. All other browsers open my website perfectly but somehow Google will not index my website anymore because of the error

My Nginx configuration:

server {


  location / {
    proxy_pass http://localhost:3000; # Change the port if needed
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection 'upgrade';
    proxy_set_header Host $host;
    proxy_cache_bypass $http_upgrade;
                                                                                                                                                                                                                                                listen 443 ssl; # managed by Certbot                                                                                                                                                                                                         ssl_certificate /etc/letsencrypt/live/; # managed by Certbot                                                                                                                                                ssl_certificate_key /etc/letsencrypt/live/; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot


server {

    if ($host = {
        return 301 https://$host$request_uri;
    } # managed by Certbot

    if ($host = {
        return 301 https://$host$request_uri;
    } # managed by Certbot

  listen 80;

    return 404; # managed by Certbot

My domain is:

I ran this command: Problem with migrating from http to https using Certbot and NginX on a server with ExpressJS Platform

It produced this output: Running Chrome with Googlebot as useragent gives Bad Gateway 502 error. All other browsers run the website with no problems

My web server is (include version): Linux

The operating system my web server runs on is (include version): Linux Ubuntu 16.04

My hosting provider, if applicable, is: Myself

I can login to a root shell on my machine (yes or no, or I don’t know): yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel): no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you’re using Certbot): certbot 0.31.0

Do you have a log line from the nginx log with the Googlebot request? Does your nginx log_format directive include TLS parameters? If not, see on how to add this. Could help.

Hi @Larsmanson


is your answer.

If you have really that problem, you have a not working (and bad) useragent sniffing in your system.

Find that and change that.

I agree with @JuergenAuer here. Simple test:

  • curl -Li

Returns a single HTTP 301 redirect to the HTTPS site after which the HTTPS request answers with HTTP 200 immediately and returns your site contents.


  • curl -LiA "Mozilla/5.0 (compatible; Googlebot/2.1; +"

returns a HTTP 301 redirect to HTTPS (so far so good), but after that your backend keeps returning HTTP 301 redirects too! And it keeps doing so in a loop.

This is also the case for even curl -LiA "Googlebot" by the way.

So Express is malfunctioning.

Thanks, this will help me finding the issue here

Probably some kind of plugin installed I think.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.