Manual certbot renewal error: Redirect loop detected

Hello Everyone...

Manual renewal failed this time (it usually passes without issue). The offending URL is:
https://artprocess.org/.well-known/acme-challenge/WyW9VyLo05hXtLtq3SzgU-__O58kUFz785xh4PAADYE
which is responding okay, so if anyone can give guidance on this I'd be grateful.

domain: https://artprocess.org

I ran this command: sudo certbot certonly --cert-name artprocess.org --manual -d '.artprocess.org' -d artprocess.org -d '.justatest.com' -d justatest.com -d '.artprocess.com' -d artprocess.com -d '.artprocess.net' -d artprocess.net -d '.justatest.org' -d justatest.org -d '.justatest.ie' -d justatest.ie -d '*.alternativeart.org' -d alternativeart.org --email jp@justatest.com

It produced this output: Certbot failed to authenticate some domains (authenticator: manual). The Certificate Authority reported these problems:
Domain: artprocess.org
Type: connection
Detail: During secondary validation: 172.105.67.175: Fetching https://artprocess.org/.well-known/acme-challenge/WyW9VyLo05hXtLtq3SzgU-__O58kUFz785xh4PAADYE : Redirect loop detected

Hint: The Certificate Authority failed to verify the manually created challenge files. Ensure that you created these in the correct location.

My web server is (include version): Tomcat 10.0.8

The operating system my web server runs on is (include version): Ubuntu Linux 5.4.0-216-generic GNU/Linux

My hosting provider is: Linode

I can login to a root shell on my machine (yes or no, or I don't know): Yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): No

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): certbot 5.2.2

Many thanks!
jp

1 Like

Welcome @justatest

First, we do not recommend using manual processes to get certs. More on this later.

You are requesting a cert with a wildcard name. Those require a DNS Challenge and you need to add --preferred-challenges=dns to your command.

The error you are getting is for an HTTP Challenge. Usually there is an error about not using a proper challenge method but try adding the preferred-challenges and retry.

Manual cert requests have a number of drawbacks. One is that it requires repeating fairly frequently and is a burden and often forgotten. This becomes greater as cert lifetimes shorten which they are in the next year or two. See: Decreasing Certificate Lifetimes to 45 Days - Let's Encrypt

I see you have your own DNS servers but see if yours would support automating a DNS Challenge with: Welcome to certbot-dns-rfc2136’s documentation! — certbot-dns-rfc2136 0 documentation If not that there are other options like acme-dns.

3 Likes

Thanks Mike for that quick reply.

I tried a test run (as I'm afraid I'll be blocked by too many failed attempts) with this command (using your suggested --preferred-challenges=dns) :

sudo certbot certonly --dry-run --cert-name artprocess.org --manual -d '.artprocess.org' -d artprocess.org -d '.justatest.com' -d justatest.com -d '.artprocess.com' -d artprocess.com -d '.artprocess.net' -d artprocess.net -d '.justatest.org' -d justatest.org -d '.justatest.ie' -d justatest.ie -d '*.alternativeart.org' -d alternativeart.org --email jp@justatest.com --preferred-challenges=dns

but it failed with the following message:
Certbot failed to authenticate some domains (authenticator: manual). The Certificate Authority reported these problems:
Domain: justatest.org
Type: unauthorized
Detail: Incorrect TXT record "q15fvekMS5Miv8hG4WBH7KiRhoU2tVVYnyAPVVjWFo4" (and 11 more) found at _acme-challenge.justatest.org

That Incorrect TXT record "q15fvekMS5Miv8hG4WBH7KiRhoU2tVVYnyAPVVjWFo4" is an older TXT for that domain which I had left in the DNS file (it hadn't caused a problem before this).

BTW I would LOVE to get this process automated! auto renewal worked when I first installed Letsencrypt but maybe two years ago stopped working and I couldn't get it to work again - so I resorted to manual renewal - which has been okay up to now.

If I could fix this issue I'll then dedicate time to trying to configure auto renewal again.

Regards,
jp

Yeah, each new cert request uses a new TXT value. After you update your DNS with the value for the current run you need to wait for your DNS servers to sync (world-wide) before proceeding.

Personally, with so many domain names in the same cert I think you should work on getting automated renewal working rather than bothering with your manual method. It will be very tedious. Have you always used wildcard names in your cert?

That "redirect loop" error is usually an error in the web server (Tomcat for you). But, some firewalls make "fake" redirects to block bots which Let's Encrypt sees as a possible loop and stops. Have you changed your firewalls since you last got a good cert.

4 Likes

Okay Mike, I'll try and configure the automated renewal following the docs you shared earlier.

Do I need beforehand to revoke the current certificates as the Certbot instructions says to:
"Remove certbot-auto and any Certbot OS packages"?
I presume that I run:
"sudo certbot certonly --standalone" as there is no Tomcat plugin?
What happens if there's an already-existing letsencrypt certificate (expires in a few days)?

Many thanks,
jp

No, the certificates you have can remain (should remain).

Those sentences are just for the Certbot software install.

No, I wouldn't run that. One is that Standalone uses an HTTP Challenge which does not allow wildcard names. And, two, it requires exclusive use of port 80 while it runs which means Tomcat would need to be stopped/started each time. That is not ideal.

If you need wildcard names you need to use the DNS Challenge

3 Likes

Mike, I find that this quickly gets quite confusing.

  1. Up to now I've been doing the manual renew for multiple wildcarded domains. The renewal process always instructed me to do BOTH dns challenges and http challenges. Would you know if something has changed in the recent months that makes this no longer possible? The '*' wildcard hadn't been a problem before. I have no problem with setting up dns or http challenges.

  2. The renewal process tells me to leave old TXT entries in the dns files but on my last attempt the process complained about an old TXT entry (which hadn't caused any issue up to now). These conflicting instructions/results are pretty opaque to my poor understanding.

  3. If I shouldn't use "certbot certonly --standalone", you say you wouldn't run that. But then you don't say what you would run in its place. Apologies, I'm left feeling really stupid that I don't get this.

AFAIK there have been no configuration changes to the Tomcat installation or the server this year that might indicate where the problem lies.

Thanks for any futher suggestions that I should consider.
jp

No, I don't know of any changes. I asked if you have changed firewalls since your last good cert. I said the error from "Secondary ... redirect loop" can be caused by certain firewalls.

Or, made any changes to your firewall config? Such as geographic based restrictions?

No, it shouldn't be saying that. You should delete TXT records after you get a cert. They are not needed after the cert is issued. And, if too many are allowed to accumulate Let's Encrypt will eventually fail because the packet size gets too large.

I personally recommend automating the DNS Challenge. You need that for your wildcard names anyway and once those work the other names should be just as easy. It looks like you use the same DNS servers for all those names. I didn't check them all but I assume so.

4 Likes

I appreciate that heads-up Mike. I went the route you suggested I take; and with the heavy lifting of claude walking me through the process now have auto renew:
Mon 2025-12-22 06:58:00 GMT 6h left n/a n/a snap.certbot.renew.timer snap.certbot.renew.service

Many thanks!
jp

2 Likes

A good way to test the auto-renew is with:

sudo certbot renew --dry-run

It will not affect your existing certs. It is just a test. It will use the same method and options you used to get the cert against the Let's Encrypt staging system.

If this works you should be fine for automated renewals :slight_smile:

3 Likes