Failed to renew certificate

Help
1

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: mrs.dsg.dk

I ran this command: certbot renew

It produced this output:
Saving debug log to /var/log/letsencrypt/letsencrypt.log

Processing /etc/letsencrypt/renewal/mrs.dsg.dk.conf

Failed to renew certificate mrs.dsg.dk with error: HTTPSConnectionPool(host='acme-v02.api.letsencrypt.org', port=443): Max retries exceeded with url: /directory (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1133)')))

All renewals failed. The following certificates could not be renewed:
/etc/letsencrypt/live/mrs.dsg.dk/fullchain.pem (failure)

1 renew failure(s), 0 parse failure(s)
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.

My web server is (include version): apache (httpd)

The operating system my web server runs on is (include version): RHEL9.4

My hosting provider, if applicable, is:

I can login to a root shell on my machine (yes or no, or I don't know): Yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): certbot 2.11.0

2

Hi @shriram1992,

Using the online tool Let's Debug yields these results https://letsdebug.net/mrs.dsg.dk/2324962

ANotWorking
Error
mrs.dsg.dk has an A (IPv4) record (109.236.189.11) but a request to this address over port 80 did not succeed. Your web server must have at least one working IPv4 or IPv6 address.
Get "http://mrs.dsg.dk/.well-known/acme-challenge/letsdebug-test": dial tcp 109.236.189.11:80: connect: connection refused

Trace:
@0ms: Making a request to http://mrs.dsg.dk/.well-known/acme-challenge/letsdebug-test (using initial IP 109.236.189.11)
@0ms: Dialing 109.236.189.11
@16ms: Experienced error: dial tcp 109.236.189.11:80: connect: connection refused

The HTTP-01 challenge of the Challenge Types - Let's Encrypt is being used; and it states "The HTTP-01 challenge can only be done on port 80."

Best Practice - Keep Port 80 Open

2 Likes
3

You have two IP addresses in your DNS. One of them reaches an RHEL Apache server. The other fails entirely. You need to remove the failing IP address from your DNS.

If you plan to use both then you need to fix the failing IP address so it can connect. But, you also should then use a DNS Challenge. Coordinating two IP to respond properly to an HTTP Challenge takes extra care and is difficult.

That said, your original problem " '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1133)')))" is usually something wrong in your system's local CA Trusted Root Store. This is an error outbound from your server to the Let's Encrypt API server.

What does this show?

curl -v https://acme-v02.api.letsencrypt.org/directory

Here is the problem with HTTP requests inbound to your server

mrs.dsg.dk.	0	IN	A	109.236.189.11
mrs.dsg.dk.	0	IN	A	109.236.189.10

curl -I http://109.236.189.10
HTTP/1.1 200 OK
Server: Apache/2.4.62 (Red Hat Enterprise Linux) OpenSSL/3.2.2
Last-Modified: Tue, 15 Jun 2021 08:31:17 GMT

curl -I http://109.236.189.11
curl: (7) Failed to connect to 109.236.189.11 port 80 after 97 ms: 
Connection refused
4 Likes
4

Thanks for the response, i ll check the possibility and get back on this

5

Thanks for the response. Here is the output of curl -v https://acme-v02.api.letsencrypt.org/directory:

  • Trying 172.65.32.248:443...
  • Connected to acme-v02.api.letsencrypt.org (172.65.32.248) port 443 (#0)
  • ALPN, offering h2
  • ALPN, offering http/1.1
  • CAfile: /etc/pki/tls/certs/ca-bundle.crt
  • TLSv1.0 (OUT), TLS header, Certificate Status (22):
  • TLSv1.3 (OUT), TLS handshake, Client hello (1):
  • TLSv1.2 (IN), TLS header, Certificate Status (22):
  • TLSv1.3 (IN), TLS handshake, Server hello (2):
  • TLSv1.2 (IN), TLS header, Finished (20):
  • TLSv1.2 (IN), TLS header, Unknown (23):
  • TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
  • TLSv1.2 (IN), TLS header, Unknown (23):
  • TLSv1.3 (IN), TLS handshake, Certificate (11):
  • TLSv1.2 (IN), TLS header, Unknown (23):
  • TLSv1.3 (IN), TLS handshake, CERT verify (15):
  • TLSv1.2 (IN), TLS header, Unknown (23):
  • TLSv1.3 (IN), TLS handshake, Finished (20):
  • TLSv1.2 (OUT), TLS header, Finished (20):
  • TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
  • TLSv1.2 (OUT), TLS header, Unknown (23):
  • TLSv1.3 (OUT), TLS handshake, Finished (20):
  • SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
  • ALPN, server accepted to use h2
  • Server certificate:
  • subject: CN=acme-v02.api.letsencrypt.org
  • start date: Nov 20 15:24:57 2024 GMT
  • expire date: Feb 18 15:24:56 2025 GMT
  • subjectAltName: host "acme-v02.api.letsencrypt.org" matched cert's "acme-v02.api.letsencrypt.org"
  • issuer: C=US; O=Let's Encrypt; CN=R11
  • SSL certificate verify ok.
  • Using HTTP2, server supports multi-use
  • Connection state changed (HTTP/2 confirmed)
  • Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
  • TLSv1.2 (OUT), TLS header, Unknown (23):
  • TLSv1.2 (OUT), TLS header, Unknown (23):
  • TLSv1.2 (OUT), TLS header, Unknown (23):
  • Using Stream ID: 1 (easy handle 0x55ec1b715030)
  • TLSv1.2 (OUT), TLS header, Unknown (23):

GET /directory HTTP/2
Host: acme-v02.api.letsencrypt.org
user-agent: curl/7.76.1
accept: /

Also i would like to let you know that the error i pasted in this case description had occured when we used our internal proxy to reach acme., since our source is DNS forwarder server, we can reach the acme site without proxy as well. and here is the error we get when i run "certbot renew" without proxy:

**

Processing /etc/letsencrypt/renewal/mrs.dsg.dk.conf

Renewing an existing certificate for mrs.dsg.dk

Certbot failed to authenticate some domains (authenticator: apache). The Certificate Authority reported these problems:
Domain: mrs.dsg.dk
Type: connection
Detail: During secondary validation: 109.236.189.11: Fetching http://mrs.dsg.dk/.well-known/acme-challenge/1SMY_LPShwSgJaTDn0yG604wiBIVGPByyv9diTTroeg: Connection refused

Hint: The Certificate Authority failed to verify the temporary Apache configuration changes made by Certbot. Ensure that the listed domains point to this Apache server and that it is accessible from the internet.

Failed to renew certificate mrs.dsg.dk with error: Some challenges have failed.

so its clear that 109.236.189.11 is not reachable from internet . we dont have apache running on 109.236.189.11. so it wont be reachable directly or there is an internal load balancer which is taking care of the job. but i want to renew the certificate in 109.236.189.10 which is working/reachable from internet. also we are using DNS challenge not http to my knowledge, as we have . so whats stopping the renewal please? why is it looking for 109.236.189.11?

Thanks and sorry if i have posted some silly question :wink:

6

hoo! it worked after installing apache in our other server where public IP .11 is.

Previously it was handled by loadbalancer to my knowledge. now we have enabled httpd on .11 to access directly and it worked.

Thanks all for your comments, let me keep this post open for comments for few days to have some healthy conversations ahead :slight_smile:

7

The Apache authenticator is an HTTP Challenge. The Let's Encrypt auth servers will send HTTP requests to validate this challenge. It sends these to any of the IP addresses in the public DNS. Each of your servers must be able to reply with the proper challenge value. In fact, today five HTTP requests will be made from various global points for each HTTP Challenge. And, currently at least 4 of those must reply with the correct value to succeed. Each of these 5 HTTP requests chooses either of the IP in your DNS. So, likely some going to both.

The --apache authenticator will only prepare the Apache system that Certbot runs on. So, you could redirect the HTTP request from the alternate Apache to the Certbot system.

Or, use a DNS Challenge. It does not rely on HTTP flows but instead by placing TXT record values in the public DNS. You still have to distribute the cert to both your systems. This works best if your DNS provider has an API so the needed TXT records can be automatically added / deleted.

2 Likes