Certificate Renewal Failed

jaskaran · September 22, 2021, 2:47am

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is:digitalisland.co.nz

I ran this command:certbot renew

It produced this output: Attempting to renew cert from /etc/letsencrypt/renewal/xyz.digitalisland.co.nz.conf produced an unexpected error: Some challenges have failed.. Skipping.

My web server is (include version): Nginx1.16

The operating system my web server runs on is (include version): CentOS Linux release 7.7

My hosting provider, if applicable, is:

I can login to a root shell on my machine (yes or no, or I don't know): Yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): certbot 1.3.0

Nginx config-

server {
server_name xyz.digitalisland.co.nz;
listen 80;
server_tokens off;
root /var/www/html;
try_files $uri $uri/ =404;

location /.well-known {
}

location / {
    add_header Content-Type text/plain;
    return 200 '$remote_addr\r\n';
}

rg305 · September 22, 2021, 2:51am

Hi @jaskaran, welcome to the LE community forum

The emptiness if that has my head spinning...
I suppose you have tested that to work already.

If that is not the problem, then we may need to see more of the certbot logs (for clues).
/var/log/letsencrypt/letsencrypt.log
And also the renewal config file.
Something like:
/etc/letsencrypt/renewal/YOUR.DOMAIN.conf

jaskaran · September 22, 2021, 2:58am

Sep 22 14:42:48 openvpn: probes/124.197.42.218:63184 peer info: IV_COMP_STUBv2=1
Sep 22 14:42:48 openvpn: probes/124.197.42.218:63184 peer info: IV_TCPNL=1
Sep 22 14:42:48 openvpn: probes/124.197.42.218:63184 Outgoing Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key
Sep 22 14:42:48 openvpn: probes/124.197.42.218:63184 Incoming Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key
Sep 22 14:42:48 openvpn: probes/124.197.42.218:63184 Control Channel: TLSv1.2, cipher TLSv1/SSLv3 ECDHE-RSA-AES256-GCM-SHA384, 2048 bit RSA
Sep 22 14:46:42 openvpn: TCP connection established with [AF_INET]172.31.4.91:50490
Sep 22 14:46:42 openvpn: 172.31.4.91:50490 Non-OpenVPN client protocol detected
Sep 22 14:46:42 openvpn: 172.31.4.91:50490 SIGTERM[soft,port-share-redirect] received, client-instance exiting
Sep 22 14:52:59 openvpn: probes/202.3.80.50:44284 TLS: tls_process: killed expiring key
Sep 22 14:53:00 openvpn: probes/202.3.80.50:44284 TLS: soft reset sec=0 bytes=269854/-1 pkts=2432/0
Sep 22 14:53:01 openvpn: probes/202.3.80.50:44284 VERIFY OK: depth=1,
Sep 22 14:53:01 openvpn: probes/202.3.80.50:44284 VERIFY OK: depth=0,
Sep 22 14:53:01 openvpn: probes/202.3.80.50:44284 peer info: IV_VER=2.4.6
Sep 22 14:53:01 openvpn: probes/202.3.80.50:44284 peer info: IV_PLAT=linux
Sep 22 14:53:01 openvpn: probes/202.3.80.50:44284 peer info: IV_PROTO=2
Sep 22 14:53:01 openvpn: probes/202.3.80.50:44284 peer info: IV_LZ4=1
Sep 22 14:53:01 openvpn: probes/202.3.80.50:44284 peer info: IV_LZ4v2=1
Sep 22 14:53:01 openvpn: probes/202.3.80.50:44284 peer info: IV_LZO=1
Sep 22 14:53:01 openvpn: probes/202.3.80.50:44284 peer info: IV_COMP_STUB=1
Sep 22 14:53:01 openvpn: probes/202.3.80.50:44284 peer info: IV_COMP_STUBv2=1
Sep 22 14:53:01 openvpn: probes/202.3.80.50:44284 peer info: IV_TCPNL=1
Sep 22 14:53:01 openvpn: probes/202.3.80.50:44284 Outgoing Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key
Sep 22 14:53:01 openvpn: probes/202.3.80.50:44284 Incoming Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key
Sep 22 14:53:01 openvpn: probes/202.3.80.50:44284 Control Channel: TLSv1.2, cipher TLSv1/SSLv3 ECDHE-RSA-AES256-GCM-SHA384, 2048 bit RSA
Sep 22 14:53:13 sudo: jas : TTY=pts/1 ; PWD=/home/jas ; USER=root ; COMMAND=/bin/cat /var/log/letsencrypt
Sep 22 14:53:18 sudo: jas : TTY=pts/1 ; PWD=/home/jas ; USER=root ; COMMAND=/bin/cat /var/log/letsencrypt/
Sep 22 14:53:43 sudo: jas : TTY=pts/1 ; PWD=/home/jas ; USER=root ; COMMAND=/bin/vi /var/log/letsencrypt/
Sep 22 14:53:55 sudo: jas : TTY=pts/1 ; PWD=/home/jas ; USER=root ; COMMAND=/bin/vi /var/log/syslog
Sep 22 14:55:03 openvpn: probes/103.241.89.13:33050 TLS: tls_process: killed expiring key
Sep 22 14:55:04 openvpn: probes/103.241.89.13:33050 TLS: soft reset sec=0 bytes=265584/-1 pkts=2394/0
Sep 22 14:55:05 openvpn: probes/103.241.89.13:33050 VERIFY OK: depth=1, CN=Digital Island CA
Sep 22 14:55:05 openvpn: probes/103.241.89.13:33050 VERIFY OK: depth=0, CN=probes
Sep 22 14:55:05 openvpn: probes/103.241.89.13:33050 peer info: IV_VER=2.4.6
Sep 22 14:55:05 openvpn: probes/103.241.89.13:33050 peer info: IV_PLAT=linux
Sep 22 14:55:05 openvpn: probes/103.241.89.13:33050 peer info: IV_PROTO=2
Sep 22 14:55:05 openvpn: probes/103.241.89.13:33050 peer info: IV_LZ4=1
Sep 22 14:55:05 openvpn: probes/103.241.89.13:33050 peer info: IV_LZ4v2=1
Sep 22 14:55:05 openvpn: probes/103.241.89.13:33050 peer info: IV_LZO=1
Sep 22 14:55:05 openvpn: probes/103.241.89.13:33050 peer info: IV_COMP_STUB=1
Sep 22 14:55:05 openvpn: probes/103.241.89.13:33050 peer info: IV_COMP_STUBv2=1
Sep 22 14:55:05 openvpn: probes/103.241.89.13:33050 peer info: IV_TCPNL=1
Sep 22 14:55:05 openvpn: probes/103.241.89.13:33050 Outgoing Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key
Sep 22 14:55:05 openvpn: probes/103.241.89.13:33050 Incoming Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key
Sep 22 14:55:05 openvpn: probes/103.241.89.13:33050 Control Channel: TLSv1.2, cipher TLSv1/SSLv3 ECDHE-RSA-AES256-GCM-SHA384, 2048 bit RSA
Sep 22 14:55:34 sudo: jas : TTY=pts/1 ; PWD=/home/jas ; USER=root ; COMMAND=/bin/vi /var/log/syslog.1.gz
Sep 22 14:55:44 sudo: jas : TTY=pts/1 ; PWD=/home/jas ; USER=root ; COMMAND=/bin/cat /var/log/syslog
Sep 22 14:56:22 sudo: jas : TTY=pts/1 ; PWD=/home/jas ; USER=root ; COMMAND=/bin/certbot renew
Sep 22 14:56:58 sudo: jas : TTY=pts/1 ; PWD=/home/jas ; USER=root ; COMMAND=/bin/cat /var/log/syslog

--------------------------------
/etc/letsencrypt/renewal/YOUR.DOMAIN.conf

renew_before_expiry = 30 days

version = 1.3.0
archive_dir = /etc/letsencrypt/archive/xyz.digitalisland.co.nz
cert = /etc/letsencrypt/live/xyz.digitalisland.co.nz/cert.pem
privkey = /etc/letsencrypt/live/xyz.digitalisland.co.nz/privkey.pem
chain = /etc/letsencrypt/live/xyz.digitalisland.co.nz/chain.pem
fullchain = /etc/letsencrypt/live/xyz.digitalisland.co.nz/fullchain.pem

Options used in the renewal process

[renewalparams]
authenticator = webroot
renew_hook = systemctl reload nginx.service
account = 4d4712915e912a1fd9ba31b46502822e
server = https://acme-v02.api.letsencrypt.org/directory
[[webroot_map]]
xyz.digitalisland.co.nz = /var/www/qqq

rg305 · September 22, 2021, 3:04am

If I read your vhost config and renewal.conf files correctly...
There seems to be a line missing from the renewal.conf (add at the bottom):
myip.digitalisland.co.nz = /var/www/html

Then retry certbot in test mode:
certbot renew --dry-run

rg305 · September 22, 2021, 3:19am

Name:    monitoring.digitalisland.co.nz
Address: 103.13.11.1

Name:    syslog.digitalisland.co.nz
Address: 103.13.11.1

Both fail with "firewall" problem.
You must have a functional HTTP site before it can be secured (via HTTP authentication).

curl -v 103.13.11.1
* Rebuilt URL to: 103.13.11.1/
*   Trying 103.13.11.1...
* TCP_NODELAY set
* Connected to 103.13.11.1 (103.13.11.1) port 80 (#0)
> GET / HTTP/1.1
> Host: 103.13.11.1
> User-Agent: curl/7.58.0
> Accept: */*
>
* Recv failure: Connection reset by peer
* stopped the pause stream!
* Closing connection 0
curl: (56) Recv failure: Connection reset by peer

jaskaran · September 22, 2021, 3:46am

So you mean connection from LetEncrypt to 103.13.11.1 failed because of that its not getting renewed ?

rg305 · September 22, 2021, 3:48am

Yes.
The HTTP authentication request is unable to reach your site.

Please confirm the IP, with:
curl -4 ifconfig.co

jaskaran · September 22, 2021, 3:50am

yes, ip is 103.13.11.1

jaskaran · September 22, 2021, 3:56am

can you confirm what IPs we need to open for letsencrypt in our firewall

Is there any change on these.

rg305 · September 22, 2021, 3:57am

That is the IP that would be used by LE to validate.

Is there a firewall?
Is it open on port 80 to your server?

jaskaran · September 22, 2021, 4:00am

yes there is firewall and port 80 is already open.

Even the error you sent is saying - Connected to 103.13.11.1 (103.13.11.1) port 80** (#0)

curl -v 103.13.11.1

Rebuilt URL to: 103.13.11.1/
Trying 103.13.11.1...
TCP_NODELAY set
Connected to 103.13.11.1 (103.13.11.1) port 80 (#0)

GET / HTTP/1.1

rg305 · September 22, 2021, 4:07am

Not open from my IP.
Not from Let's Debug: Let's Debug (letsdebug.net)

Please try:
certbot renew --dry-run
Then show the LE log file.
/var/log/letsencrypt/letsencrypt.log

jaskaran · September 22, 2021, 4:08am

I cant see log file /var/log/letsencrypt/letsencrypt.log

rg305 · September 22, 2021, 4:10am

hmm...

Try:
sudo find / -name letsencrypt.log

jaskaran · September 22, 2021, 5:07am

Just got to know US requests blocked in Firewall and i am assuming LetsEncrypt request is coming from US ?

Can you confirm what all URLs or IPs we can allow in firewall ?

All the LetsEncypt URLs/IP which will send request to our server on port 80 ?

rg305 · September 22, 2021, 5:21am

No.
LE doesn't publish the IPs used; And it is expected that they can change without notice.
See the FAQ: FAQ - Let's Encrypt (letsencrypt.org)

rg305 · September 22, 2021, 5:36am

On reread of your question...
If you can whitelist URLs, then there will always be the unique URL, that will always start with:
http://YOUR.DOMAIN/.well-known/acme-challenge/
The URL (file) ending will change with every renewal request.

jaskaran · September 22, 2021, 9:09pm

Thanks for your help, For now we have fixed it by changes in FW policy.

jaskaran · September 23, 2021, 10:16pm

Hi,

Sorry another question - I got to know that LetsEncrypt Root CA is going to expire on 30 September.

So will there be any impact in our certs/Websites ? Do we need to do anything ?

rg305 · September 23, 2021, 10:20pm

Not likely.
All ACME clients have been provided with the new longer validity chain now for many weeks.
[I always check to ensure that my systems are in fact serving that newer chain.]

Topic		Replies	Views
Cert Renewal Fails Help	6	2750	November 29, 2021
Certificate Renewal Issue Help	5	418	December 29, 2023
All renewal attempts failed Help	7	1638	June 1, 2021
Unable to renew cert Help	2	753	December 1, 2021
Renew certificate Help	7	589	April 8, 2023

Certificate Renewal Failed

renew_before_expiry = 30 days

Options used in the renewal process

Related Topics