Auto renew didn't respect the config file


#1

Help templete

My domain is: https://qbt.rsb.sparvojo.pw

I ran this command: certbot renew

It produced this output:

Cert is due for renewal, auto-renewing...
Plugins selected: Authenticator standalone, Installer None
Renewing an existing certificate
Performing the following challenges:
tls-sni-01 challenge for qbt.rsb.sparvojo.pw
Cleaning up challenges
Attempting to renew cert (qbt.rsb.sparvojo.pw) from /etc/letsencrypt/renewal/qbt.rsb.sparvojo.pw.conf produced an unexpected error: Problem binding to port 443: Could not bind to IPv4 or IPv6.. Skipping.
All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/qbt.rsb.sparvojo.pw/fullchain.pem (failure)

-------------------------------------------------------------------------------

All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/qbt.rsb.sparvojo.pw/fullchain.pem (failure)
-------------------------------------------------------------------------------
Running post-hook command: /etc/letsencrypt/renewal-hooks/post/10haproxy.sh
1 renew failure(s), 0 parse failure(s)

I ran this command: certbot renew --tls-sni-01-port 54321

It produced this output:

Cert is due for renewal, auto-renewing...
Plugins selected: Authenticator standalone, Installer None
Renewing an existing certificate
Performing the following challenges:
tls-sni-01 challenge for qbt.rsb.sparvojo.pw
Waiting for verification...
Cleaning up challenges
Attempting to renew cert (qbt.rsb.sparvojo.pw) from /etc/letsencrypt/renewal/qbt.rsb.sparvojo.pw.conf produced an unexpected error: Failed authorization procedure. qbt.rsb.sparvojo.pw (tls-sni-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Incorrect validation certificate for tls-sni-01 challenge. Requested b937035686cb5daff5ed7df5ff502e5a.a6b05e085e321fc31847d0888bd6cd14.acme.invalid from 54.38.167.31:443. Received 2 certificate(s), first certificate had names "qbt.rsb.sparvojo.pw". Skipping.
All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/qbt.rsb.sparvojo.pw/fullchain.pem (failure)

-------------------------------------------------------------------------------

All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/qbt.rsb.sparvojo.pw/fullchain.pem (failure)
-------------------------------------------------------------------------------
Running post-hook command: /etc/letsencrypt/renewal-hooks/post/10haproxy.sh
1 renew failure(s), 0 parse failure(s)

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: qbt.rsb.sparvojo.pw
   Type:   unauthorized
   Detail: Incorrect validation certificate for tls-sni-01 challenge.
   Requested
   b937035686cb5daff5ed7df5ff502e5a.a6b05e085e321fc31847d0888bd6cd14.acme.invalid
   from 54.38.167.31:443. Received 2 certificate(s), first certificate
   had names "qbt.rsb.sparvojo.pw"

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address.

I ran this command: certbot renew --preferred-challenges http --http-01-port 54321

It produced this output:

Cert is due for renewal, auto-renewing...
Plugins selected: Authenticator standalone, Installer None
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for qbt.rsb.sparvojo.pw
Waiting for verification...
Cleaning up challenges

-------------------------------------------------------------------------------
new certificate deployed without reload, fullchain is
/etc/letsencrypt/live/qbt.rsb.sparvojo.pw/fullchain.pem
-------------------------------------------------------------------------------

-------------------------------------------------------------------------------

Congratulations, all renewals succeeded. The following certs have been renewed:
  /etc/letsencrypt/live/qbt.rsb.sparvojo.pw/fullchain.pem (success)
-------------------------------------------------------------------------------
Running post-hook command: /etc/letsencrypt/renewal-hooks/post/10haproxy.sh

My web server is (include version): HAProxy 1.8.4

The operating system my web server runs on is (include version): Ubuntu 16.04 LTS

I can login to a root shell on my machine (yes or no, or I don’t know): yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel): no

Additional config/log

HAProxy config (Partial):

global
	ssl-default-bind-ciphers ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384
	ssl-default-bind-options no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets
	ssl-default-server-ciphers ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384
	ssl-default-server-options no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets

frontend qbt-http
	bind :80
	reqadd X-Forwarded-Proto:\ http
	acl letsencrypt-acl path_beg /.well-known/acme-challenge/
	use_backend letsencrypt-backend if letsencrypt-acl
	default_backend qbt-backend

frontend qbt-https
	bind :443 ssl crt /etc/haproxy/certs/qbt.rsb.sparvojo.pw.pem alpn h2,http/1.1 ecdhe secp384r1
	reqadd X-Forwarded-Proto:\ https
	acl letsencrypt-acl path_beg /.well-known/acme-challenge/
	use_backend letsencrypt-backend if letsencrypt-acl
	default_backend qbt-backend

	# CSP
	http-response set-header Content-Security-Policy "default-src 'self'; style-src 'self' 'unsafe-inline'; img-src 'self' data:; script-src 'self' 'unsafe-inline'; object-src 'none';"

	# Referrer Policy
	http-response set-header Referrer-Policy no-referrer-when-downgrade

	# HSTS
	http-response set-header Strict-Transport-Security max-age=15768000

backend qbt-backend
	redirect scheme https if !{ ssl_fc }
	server qbt 127.0.0.1:8080 check

backend letsencrypt-backend
	server letencrypt 127.0.0.1:54321

renewal config:

# renew_before_expiry = 30 days
version = 0.21.1
archive_dir = /etc/letsencrypt/archive/qbt.rsb.sparvojo.pw
cert = /etc/letsencrypt/live/qbt.rsb.sparvojo.pw/cert.pem
privkey = /etc/letsencrypt/live/qbt.rsb.sparvojo.pw/privkey.pem
chain = /etc/letsencrypt/live/qbt.rsb.sparvojo.pw/chain.pem
fullchain = /etc/letsencrypt/live/qbt.rsb.sparvojo.pw/fullchain.pem

# Options used in the renewal process
[renewalparams]
authenticator = standalone
must_staple = True
http01_port = 54321
installer = None
account = fc131284e7942e6382a51ecd1f8467cc
rsa_key_size = 4096
pref_challs = http-01, # Not sure about this line, it may added by certbot after the successful renewal since I used different parameters in manual renew process

My Problem

Today I check the syslog and discovered that certbot unable to renew while it works (dry run) few months ago when I set it up. I searched the forum and seems many other have this problem before. I tried to follow the solutions but all failed. In a manual renew with forcing to use HTTP challenge, it have no problem. Then I discovered that I did enforce it in my config but the renew process didn’t respect it, thus cause the renewal failed. How can I make the renew process respect the config file?


#2

I think LE will only connect (at least initially) via ports 80 or 443.
It might follow a redirection from those ports to another port…



#3

I believe that OP’s problem is that Certbot is not using the intended challenge method when running non-interactively.

The :54321 proxy pass-through looks fine to me for HTTP-01.

OP, if you run certbot renew --dry-run with no other arguments , does it succeed?


#4

It respect the config file under dry run.

output:

user@box01:~$ sudo certbot renew --dry-run
Saving debug log to /var/log/letsencrypt/letsencrypt.log

-------------------------------------------------------------------------------
Processing /etc/letsencrypt/renewal/qbt.rsb.sparvojo.pw.conf
-------------------------------------------------------------------------------
Cert not due for renewal, but simulating renewal for dry run
Plugins selected: Authenticator standalone, Installer None
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for qbt.rsb.sparvojo.pw
Waiting for verification...
Cleaning up challenges

-------------------------------------------------------------------------------
new certificate deployed without reload, fullchain is
/etc/letsencrypt/live/qbt.rsb.sparvojo.pw/fullchain.pem
-------------------------------------------------------------------------------

-------------------------------------------------------------------------------
** DRY RUN: simulating 'certbot renew' close to cert expiry
**          (The test certificates below have not been saved.)

Congratulations, all renewals succeeded. The following certs have been renewed:
  /etc/letsencrypt/live/qbt.rsb.sparvojo.pw/fullchain.pem (success)
** DRY RUN: simulating 'certbot renew' close to cert expiry
**          (The test certificates above have not been saved.)
-------------------------------------------------------------------------------
Running post-hook command: /etc/letsencrypt/renewal-hooks/post/10haproxy.sh

However, this is executed after the cert renewed so I have no idea what the result will be before renewing.


#5

I don’t know why that would work and live renewal wouldn’t. It looks fine/correct. You can also try it without --dry-run and with --force-renew for an even more realistic test, I guess.

The fact that you ran the following at all is a red flag to me:

It may have inadvertently changed the challenge from HTTP to TLS-SNI, though I don’t really understand that part of Certbot and can’t say for sure.


#6

Interesting, I tried using --force-renew, and it respect the config file that use HTTP challenge and port, but it report that it can’t bind the port then failed the whole task.

output:

user@box01:~$ sudo certbot renew --force-renew
Saving debug log to /var/log/letsencrypt/letsencrypt.log

-------------------------------------------------------------------------------
Processing /etc/letsencrypt/renewal/qbt.rsb.sparvojo.pw.conf
-------------------------------------------------------------------------------
Plugins selected: Authenticator standalone, Installer None
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for qbt.rsb.sparvojo.pw
Cleaning up challenges
Attempting to renew cert (qbt.rsb.sparvojo.pw) from /etc/letsencrypt/renewal/qbt.rsb.sparvojo.pw.conf produced an unexpected error: Problem binding to port 54321: Could not bind to IPv4 or IPv6.. Skipping.
All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/qbt.rsb.sparvojo.pw/fullchain.pem (failure)

-------------------------------------------------------------------------------

All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/qbt.rsb.sparvojo.pw/fullchain.pem (failure)
-------------------------------------------------------------------------------
Running post-hook command: /etc/letsencrypt/renewal-hooks/post/10haproxy.sh
1 renew failure(s), 0 parse failure(s)

#7

I’m going to consider to use DNS challenge instead of HTTP or TLS SNI challenge, and hoping it can solve this debug hell.


#8

That means there is something already bound to port 54321
netstat -pant | grep 54321


#9

Ok, now it works but I have no idea what application will bind 54321 port though. Hope it will work next renew. Thanks for your help.


#10

You can get different results between the staging and production servers about the availability of TLS-SNI challenges because they separately apply the rules about challenge availability, which can then produce different behavior from Certbot.


#11

The config is automatically updated after you successfully renew with different parameters - are you sure it was correct before that?


#12

Oh that’s new, I don’t know certbot will update the config if renew successful with different parameters. Maybe the last line perf_challs = http-01 was added by the program because I copy it after the successful renewal. I’m unsure for now. All in all, the lack of documentations cause problems and make debugging difficult.


#13

I don’t know about staging since all of my operation done on the production server.


#14

The --dry-run option always uses the staging server, though.


#15

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.