Not able to renew certificate the first time after successfully updated it for the last 2 years or so

Help
1

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: devhomeit.ca

I ran this command: certboot -v renew

It produced this output:

Processing /etc/letsencrypt/renewal/devhomeit.ca.conf

Certificate is due for renewal, auto-renewing...
Plugins selected: Authenticator manual, Installer None
Renewing an existing certificate for *.devhomeit.ca
Performing the following challenges:
dns-01 challenge for devhomeit.ca
Running manual-auth-hook command: /root/Certbot-Godaddy/certbot-godaddy-auth.sh
Hook '--manual-auth-hook' for devhomeit.ca ran with output:
Creating TXT record _acme-challenge.devhomeit.ca for certificate renewal with value zu3YxsoVDHUpZoGe7pRlkM6IlbZ0lDXV2-x9UmB63zA
HTTP/1.1 403 Forbidden
Content-Type: application/json
Content-Length: 78
Vary: origin
X-Request-Id: cW7XUZXWA97X3FkW3CxPir
X-DataCenter: US_EAST_1
Expires: Mon, 10 Jun 2024 11:20:12 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Mon, 10 Jun 2024 11:20:12 GMT
Connection: close
Set-Cookie: _abck=C7760F3A1BBB97AAF51FF8476CDD17C3~-1~YAAQD+g3F6byXvCPAQAA1Y7gAQzatVvqZJqafxmm90pNa7H0Uvd1qgqsA6bfQqlP8IaNobhrzbGNPjgsexor8xsCQs/PJHJgLqQRUwPoLyd3ZbZlOrpD7sZRa/uyWuWaMzRY0qVs9Mpf+rxu2uzsm+oz/r0SCK1t/DW1pUKVU6Hpvu7KmTNEeM0trKJUCu/vI1CrOp9zS6fFA2rUhSGS/sGRCgPgl7q72MAjHEEH0qvZLA5WfuWVQwQByhPmJMdP4phlT/rE8O3hSIiFCiPR32ChYhr6RekrfMLYeTEJqG+1jGhiYEAT2M/FeztACL9Ytc9r8u5PpdqT5XyvtJQvM5nQayPg+Fd9A9TyquiZCEmXO/Ds0APObYUh~-1~-1~-1; Domain=.godaddy.com; Path=/; Expires=Tue, 10 Jun 2025 11:20:12 GMT; Max-Age=31536000; Secure
Set-Cookie: bm_sz=C35B6F9048A8730C441D171A50350A83~YAAQD+g3F6fyXvCPAQAA1Y7gARglSXuBkuqHEBgVrRVzdED/GlKhYZoveggIa713r2SFqcaVzZ0aO8DXxbqV9vMVsUhMWuVsR2lEymEn6c/wx3FemnZiRPjY4DunMOE1KLlc6mfMfv/9Acnuw7Baml4W90rb+uqwYWp+edi78B4KgpBs7oGuicXpvPq7LFb3T6CUrh2JalCy7c8IEeYOy1eATlaG3cMiPrNVEj786wwFnLCF9mWtV//rHpb17YtWUms4cxGlk2dEQGs8O5U4XiTF2uN2O83NCyloEtOfxWyhCm8AKC+oElLZ6pNzEvIqkMh47s7xFrCoOWZDKvfG0y92zQb/BQQZxcYZCQ==~4535863~3356978; Domain=.godaddy.com; Path=/; Expires=Mon, 10 Jun 2024 15:20:12 GMT; Max-Age=14400

{"code":"ACCESS_DENIED","message":"Authenticated user is not allowed access"}
Hook '--manual-auth-hook' for devhomeit.ca ran with error output:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed

0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 189 100 78 100 111 248 353 --:--:-- --:--:-- --:--:-- 601
Waiting for verification...
Challenge failed for domain devhomeit.ca
dns-01 challenge for devhomeit.ca

Certbot failed to authenticate some domains (authenticator: manual). The Certificate Authority reported these problems:
Domain: devhomeit.ca
Type: unauthorized
Detail: Incorrect TXT record "..." found at _acme-challenge.devhomeit.ca

Hint: The Certificate Authority failed to verify the DNS TXT records created by the --manual-auth-hook. Ensure that this hook is functioning correctly and that it waits a sufficient duration of time for DNS propagation. Refer to "certbot --help manual" and the Certbot User Guide.

Cleaning up challenges
Running manual-cleanup-hook command: /root/Certbot-Godaddy/certbot-godaddy-cleanup.sh
Hook '--manual-cleanup-hook' for devhomeit.ca ran with output:
Replacing TXT records for _acme-challenge.devhomeit.ca with park values
HTTP/1.1 200 OK
Content-Length: 0
Vary: origin
X-Request-Id: gH7nQdd1VX7PmpDJ16c8SY
X-DataCenter: US_EAST_1
Expires: Mon, 10 Jun 2024 11:20:43 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Mon, 10 Jun 2024 11:20:43 GMT
Connection: keep-alive
Set-Cookie: _abck=DB66BB264A1F2D3164ED9C3A42F7EA87~-1~YAAQK+g3FyVvNr2PAQAA7wnhAQyavLmjKN3MhPYzg8HLLEa9TNCWu8Zwk1+AXKuZnFwTu0rMszQQCkUjGDQ0pdVrGH8DpzIkh+szeWRHK+dMYhIV054UYA/+kAYP99Wa5knZbwZv6r+eVGIb0MYAOG5eIkzeRX1Y95uaU7z4XQnXPN/mQPeHXdTA6gFSVMBk8uJ/wHazzafrHhjIZ/zvLS7rmYIZGvZPh7nhd/lhPs0PDXYMwBqI6wkVax5ClEyqbTjKyd2NDlSbfq/Fbcn9yqq9CMOgCafx6nnBYi3pN4lCpRBJNbUey0eNFEF2Me5VPfGWIm/1DmotcSsZedI5qOZgu0KNIfSZgUkpER0N36xeQ+7z3ehs8zck~-1~-1~-1; Domain=.godaddy.com; Path=/; Expires=Tue, 10 Jun 2025 11:20:43 GMT; Max-Age=31536000; Secure
Set-Cookie: bm_sz=2D21443C37C82F00C05D5D3143843C34~YAAQK+g3FyZvNr2PAQAA7wnhARiG9v/HUOZsiMT98NC2Q6DEZGHfBOx48U6hXjf70EVF7K0YhNix/e+Z+aJ6kLz91NbSoWKr8JwaU55GI7S8M8jKUPMCbf2ZjAcjPY6r7aH560ZipkNzbXj/LEahx2PI7cdql5VEstFCiQevlSFhkTrC3nt9VQFnR1ex+nI4bItFNwBPiUVvFA+uytSjYZr1mbkunCXyFlFN+29FKDwrVjrCX6ZotqTe4XH3FPslc+ymTDqHXcX74IlGkTcjjp1NyDB1g5Qr3fnM4Xcxfyq5QyxGsF/GwI9DuANIf/B616ElGiKvQHe39Okg1hMGhheSm+7uRd1FMnS4WQ==~4342850~4403513; Domain=.godaddy.com; Path=/; Expires=Mon, 10 Jun 2024 15:20:43 GMT; Max-Age=14400
Hook '--manual-cleanup-hook' for devhomeit.ca ran with error output:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed

0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 71 0 0 100 71 0 185 --:--:-- --:--:-- --:--:-- 185
Failed to renew certificate devhomeit.ca with error: Some challenges have failed.

All renewals failed. The following certificates could not be renewed:
/etc/letsencrypt/live/devhomeit.ca/fullchain.pem (failure)

Running post-hook command: /root/Certbot-Godaddy/certbot-renew-post-hook.sh
1 renew failure(s), 0 parse failure(s)
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.

My web server is (include version): I am running it from haproxy server with nginx installed, nginx version: nginx/1.26.0

The operating system my web server runs on is (include version):
NAME="Fedora Linux"
VERSION="39 (Server Edition)"

My hosting provider, if applicable, is: godaddy.com

I can login to a root she
ll on my machine (yes or no, or I don't know): Yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): No

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): certbot 2.11.0

2

GoDaddy has changed the requirements for usage of their DNS API. I thought there was a good information thread, but I can't find it.

If you search for godaddy and dns you can find many threads about this issue.

You might need to change DNS provider if you really require the dns-01 challenge.

3

It is this one

4

Thanks for your reply. This is not good news and I am going to change the DNS provider.
Which one you can recommend to avoid the problems? Is there another option to continue to renew the certificate without dns-01 option?

5

Thanks for your reply. This is not good news and I am going to change the DNS provider.
Which one you can recommend to avoid the problems?
Is there another option to continue to renew the certificate without dns-01 option?

7

If I let's say create individual certificates, not a wild card one, can I still renew them with Godaddy API?

8

No. You no longer have access to the GoDaddy API. You cannot use it for anything.

You can use an HTTP-01 challenge without needing to update any DNS.

9

Cloudflare is free and well-supported by Certbot, here, and its own docs and community.

You can review the Certbot docs for the other DNS provider it supports. And, other ACME Clients readily support a large number of other providers.

10

Note that Cloudflare, as a DNS registar, has limited support for ccTLDs.

But that limitation does not matter as a DNS provider, which are two different things.

11

It will work for me, at least until I change DNS provider.
I will read documentation to better understand how to create cert with HTTP-01 challenge.

12

This is helpful information. For now I will getting away from wild card certification and try to use
HTTP-01 challenge.

13

I tried to create http-01 challenge certificate, but I have some issue.
Do I need to open another thread or can ask question here?

14

Whichever you prefer. It's okay to continue here. Seeing your original cert and history might help avoid duplication of effort.

although, the original questions / answers are different for HAProxy / nginx and HTTP Challenge than what you were doing. So, be sure to answer them either here or in new thread.

Also explain how you have HAProxy configured for nginx. It can terminate TLS itself or pass-thru HTTPS requests to nginx.

15

I did some mistake and exceeded limit of retries so I have wait for one hour to repeat.
Error creating new order :: too many failed authorizations recently: see Failed Validation Limit - Let's Encrypt
My haproxy terminates TLS. The certificates are in /etc/haproxy/certs directory.
I created new DNS entry nc.devhomeit.ca, but I did not define it in haproxy so the link was shooting to nowhere. I have also nginx installed on the same haproxy, but it is down and disabled.
Now I configured ncloud.devhomeit.ca in haproxy and will try to create certifcate again.

16

I am doing something wrong and not sure how to proceed.
I tried to create certificate for another domain, piwigo.devhomeit.ca
haproxy re-directs traffic to this machine, but I have the follow issue:
Detail: 70.24.9.24: Fetching http://piwigo.devhomeit.ca/.well-known/acme-challenge/kZAEVcaJFu_A6JWEUTr4NLd2NS6Kq3Wm6zS2_bXordo: Error getting validation data
My router on 70.24.9.24 forwarding request to haproxy and haproxy forwarding to some internal machine.

17

Yeah, an HTTP Challenge behind a load balancer (reverse proxy, or CDN) can be challenging.

First, when testing you should use the Let's Encrypt Staging system. With Certbot you just add --dry-run which works with various options.

But, if HAProxy is terminating HTTPS why do you even need HTTPS between it and nginx? Are those backend servers on the same local network? Or even the same machine? If so, isn't that secure enough that HTTP would be okay?

Then, for the certs needed by HAProxy itself there is this option

I realize this isn't a direct replacement for the method you were using. But, sometimes it is worth evaluating new options.

18

Thanks Mike, I am open to any options, including creating certificates outside of haproxy and then transferring it back. Any solution is fine to me.
In my setup haproxy listening on port 443 and then redirecting traffic to other VMs on the same subnet to different ports, like 80 or 3000.
What do you think is the easiest method to create stand alone certificates?

19

So it sounds like you only need certs on HAProxy then and not the backend services. Is that right?

I think the easiest is to switch DNS Providers to something like Cloudflare. That allows you to keep using the same wildcard "infrastructure" you already have.

Another option is to follow that HAProxy blog. That should be a credible solution as it comes directly from them (not some random blogger).

Do you have anything listening on port 80? If not you could use Certbot's standalone option. Although, you need to setup A (and/or AAAA) records in the public DNS so that Let's Encrypt can resolve each specific domain name.

The --standalone option is for an HTTP Challenge so does not support wildcard certs. It requires exclusive use of port 80 and needs to "see" the incoming HTTP request. See Certbot docs but it's like:

sudo certbot certonly --standalone -d (domain) --dry-run

Once that works remove --dry-run to get a production cert. I assume you have to change HAProxy setup somewhat to know which cert to serve based on the incoming URL and SNI.

20

I will try both options tonight. I have nothing listening on port 80 on haproxy server.
I tried to create certs with sudo certbot certonly --standalone -d (domain) --dry-run, but I am getting errors, because there is no such url: https://piwigo.devhomeit.ca/.well-known/acme-challenge/TFj-OA45R9TVF5GHoUxLplhlRRP-Wuw_uNWocWvkFqQ

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator standalone, Installer None
Simulating a certificate request for piwigo.devhomeit.ca
Performing the following challenges:
http-01 challenge for piwigo.devhomeit.ca
Waiting for verification...
Challenge failed for domain piwigo.devhomeit.ca
http-01 challenge for piwigo.devhomeit.ca

Certbot failed to authenticate some domains (authenticator: standalone). The Certificate Authority reported these problems:
Domain: piwigo.devhomeit.ca
Type: connection
Detail: 70.24.9.24: Fetching http://piwigo.devhomeit.ca/.well-known/acme-challenge/TFj-OA45R9TVF5GHoUxLplhlRRP-Wuw_uNWocWvkFqQ: Error getting validation data

Hint: The Certificate Authority failed to download the challenge files from the temporary standalone webserver started by Certbot on port 80. Ensure that the listed domains point to this machine and that it can accept inbound connections from the internet.

Cleaning up challenges
Some challenges have failed.

This is just one example. The host piwigo listening on port 80 on vm with the same subnet.
I think I am doing something completely wrong.

23

Sorry, my bad. Was using a root domain from different thread :frowning: