Certbot redirect loop

So I have several servers configured similarly. All of them redirect http directly to https. On the main server the certbot renewal process has been running for years without a problem, upon renewal I see a request to the http server, it gets a 301 response and I then see a request to the https server and things work as normal.

On the server I'm attempting to refresh I see the initial request to the http server and the 301 response. I then do not see another request and I simply see a redirect loop detected error. Attempting this using curl I see appropriate responses and they work as expected:
curl http://kibana.spansh.co.uk/.well-known/acme-challenge/kOonsBWq4xZeV7RLzKn1bwVNM2i9s0vkOFulVfAUTsM -v
responds with the following header
Location: https://kibana.spansh.co.uk/.well-known/acme-challenge/kOonsBWq4xZeV7RLzKn1bwVNM2i9s0vkOFulVfAUTsM

My domain is: kibana.spansh.co.uk

I ran this command: certbot certonly --webroot --webroot-path /var/www/ssl-proof/ --force-renewal --post-hook '/etc/init.d/nginx reload' --dry-run --cert-name kibana.spansh.co.uk --debug-challenges -v

It produced this output:

Challenge failed for domain kibana.spansh.co.uk
http-01 challenge for kibana.spansh.co.uk

Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
  Domain: kibana.spansh.co.uk
  Type:   connection
  Detail: Fetching https://kibana.spansh.co.uk/.well-known/acme-challenge/ZnvEukHLrVvoBJL5V-VHMyXIZZTYkswH4eWcrCXhj9Q: Redirect loop detected

Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded from the internet.

My web server is (include version): nginx 1.22.0

The operating system my web server runs on is (include version): Linux Ubuntu bullseye

My hosting provider, if applicable, is: Hetzner

I can login to a root shell on my machine (yes or no, or I don't know): Yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): No

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): 1.28.0

Currently, I'm not getting a redirect loop, but a 404 file not found when trying the challenge. Probably due to the fact Certbot isn't running any longer. Which makes this kinda difficult to debug.

First thing I'd think is rogue .htaccess files redirecting or perhaps misconfigured Wordpress.


I've currently paused the challenge before sent to the CA at http://kibana.spansh.co.uk/.well-known/acme-challenge/CRnhQDdm1gPWsnkM5zjKngQrI9TG-frvrD4T-tdA7k8

It's not running Wordpress, in fact there's not really much running on the rest of the site at the moment. The well-known direction (location) is specifically handled as a separate docroot, so it's purely nginx, There isn't a htaccess, though the rest of the site (not under .well-known) is behind basic http auth.

Weird, I can just reach that file perfectly:

osiris@erazer ~ $ curl -Lv http://kibana.spansh.co.uk/.well-known/acme-challenge/CRnhQDdm1gPWsnkM5zjKngQrI9TG-frvrD4T-tdA7k8
*   Trying 2a01:4f8:211:1147::2:80...
* Connected to kibana.spansh.co.uk (2a01:4f8:211:1147::2) port 80 (#0)
> GET /.well-known/acme-challenge/CRnhQDdm1gPWsnkM5zjKngQrI9TG-frvrD4T-tdA7k8 HTTP/1.1
> Host: kibana.spansh.co.uk
> User-Agent: curl/7.83.1
> Accept: */*
* Mark bundle as not supporting multiuse
< HTTP/1.1 301 Moved Permanently
< Server: nginx/1.22.0
< Date: Thu, 30 Jun 2022 10:29:54 GMT
< Content-Type: text/html
< Content-Length: 169
< Connection: keep-alive
< Location: https://kibana.spansh.co.uk/.well-known/acme-challenge/CRnhQDdm1gPWsnkM5zjKngQrI9TG-frvrD4T-tdA7k8
* Ignoring the response-body
* Connection #0 to host kibana.spansh.co.uk left intact
* Clear auth, redirects to port from 80 to 443
* Issue another request to this URL: 'https://kibana.spansh.co.uk/.well-known/acme-challenge/CRnhQDdm1gPWsnkM5zjKngQrI9TG-frvrD4T-tdA7k8'
*   Trying 2a01:4f8:211:1147::2:443...
* connect to 2a01:4f8:211:1147::2 port 443 failed: Connection refused
*   Trying
* Connected to kibana.spansh.co.uk ( port 443 (#1)
* ALPN: offers h2
* ALPN: offers http/1.1
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=kibana.spansh.co.uk
*  start date: Apr 18 09:06:04 2022 GMT
*  expire date: Jul 17 09:06:03 2022 GMT
*  subjectAltName: host "kibana.spansh.co.uk" matched cert's "kibana.spansh.co.uk"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
* Using HTTP2, server supports multiplexing
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* h2h3 [:method: GET]
* h2h3 [:path: /.well-known/acme-challenge/CRnhQDdm1gPWsnkM5zjKngQrI9TG-frvrD4T-tdA7k8]
* h2h3 [:scheme: https]
* h2h3 [:authority: kibana.spansh.co.uk]
* h2h3 [user-agent: curl/7.83.1]
* h2h3 [accept: */*]
* Using Stream ID: 1 (easy handle 0x560767204220)
> GET /.well-known/acme-challenge/CRnhQDdm1gPWsnkM5zjKngQrI9TG-frvrD4T-tdA7k8 HTTP/2
> Host: kibana.spansh.co.uk
> user-agent: curl/7.83.1
> accept: */*
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
< HTTP/2 200 
< server: nginx/1.22.0
< date: Thu, 30 Jun 2022 10:29:54 GMT
< content-type: application/octet-stream
< content-length: 87
< last-modified: Thu, 30 Jun 2022 10:28:28 GMT
< etag: "62bd7acc-57"
< strict-transport-security: max-age=15768000
< x-frame-options: SAMEORIGIN
< accept-ranges: bytes
* Connection #1 to host kibana.spansh.co.uk left intact
CRnhQDdm1gPWsnkM5zjKngQrI9TG-frvrD4T-tdA7k8.wZUQxCp8nlYOVuACXDIbnCqAGnv5WOcqMyWQhck3eusosiris@erazer ~ $ 

The thing I notice is your webserver not listening on IPv6 port 443, but I don't think that leads to a redirect loop (which I'm also not seeing).


Ok, after your IPv6 prod I've checked it and I'd forgotten to add a listen [::]:443 ssl http2; to my https side.

After I fixed that it has allowed me to renew fine. I suspect that the error reporting on the certbot/letsencrypt side was flawed. It saw it could request on IPv6 on port 80 and the failure on port 443 caused it to think it was a redirect loop.


Not sure if IPv6 was the issue. I could get the challenge fine without your fix and Boulder, the software doing the Let's Encrypt validation, would definitely give a different error if there was something else than an actual redirect loop.

But I guess it's a good thing it worked now? Although I'm not sure you're definitely out of the woods for future renewals, keep a close eye I'd say :slight_smile:


Let's Encrypt does not implement a fallback to IPv4 when following redirects, so that was probably what went wrong. :+1:


Hm, but how would that give a redirect loop? It would just plain fail, right?


I suspect perhaps a logic error on the Lets Encrypt side, http worked and redirected, https failed due to no connection and it reported redirect loop when it potentially should have been caught another way.

On an unrelated note, my god the amount of attempted scans/hacks/nmaps/probes on that server since I posted here (it always had some, I just saw an increase of a few thousand percent).

Threads on this site are also indexed by search engines VERY quickly, so I'm not surprised.

Also note that all certificates are logged to Certificate Transparancy Logs and hackers can easily monitor those logs for new hostnames and try out stuff. For example, if your Wordpress site isn't secure and "wide open", but you get a public certificate anyway, this can lead to trouble.

Which emphasises the importance of having your server and public applications secure FIRST before you put it out there on the internet.


This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.