Echec renouvellement certificat depuis peu

Help Aide (en français)
1

Je peux lire des réponses en Anglais : oui

Mon nom de domaine est :
fileshare.acroe-ica.org
fileshare.eastndc.eu
www-test.eastndc.eu

J’ai exécuté cette commande :
sudo certbot renew --pre-hook “service haproxy stop” --post-hook “service haproxy start” --renew-hook “/root/haproxy_letsencrypt/concat_certificate.sh”

Elle a produit cette sortie :
Saving debug log to /var/log/letsencrypt/letsencrypt.log

Processing /etc/letsencrypt/renewal/fileshare.acroe-ica.org.conf

Cert is due for renewal, auto-renewing…
Plugins selected: Authenticator standalone, Installer None
Running pre-hook command: service haproxy stop
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for fileshare.acroe-ica.org
http-01 challenge for fileshare.eastndc.eu
http-01 challenge for www-test.eastndc.eu
Waiting for verification…
Cleaning up challenges
Attempting to renew cert (fileshare.acroe-ica.org) from /etc/letsencrypt/renewal/fileshare.acroe-ica.org.conf produced an unexpected error: Failed authorization procedure. fileshare.acroe-ica.org (http-01): urn:acme:error:connection :: The server could not connect to the client to verify the domain :: Fetching http://fileshare.acroe-ica.org/.well-known/acme-challenge/Pv7YptWBxv1BxMIyXBQXKGKfYr2hE2YJ5PoOCHOO3pI: Connection refused, fileshare.eastndc.eu (http-01): urn:acme:error:connection :: The server could not connect to the client to verify the domain :: Fetching http://fileshare.eastndc.eu/.well-known/acme-challenge/7Xe9O74ONsLq7sngB9XAgB95Sp51RBcdBK8J0VVVamk: Connection refused, www-test.eastndc.eu (http-01): urn:acme:error:connection :: The server could not connect to the client to verify the domain :: Fetching http://www-test.eastndc.eu/.well-known/acme-challenge/OYGKlvBi4EC5gdNTimRQ4Id06oqjSn_rnXTrY-i_hAs: Connection refused. Skipping.
All renewal attempts failed. The following certs could not be renewed:
/etc/letsencrypt/live/fileshare.acroe-ica.org/fullchain.pem (failure)

All renewal attempts failed. The following certs could not be renewed:
/etc/letsencrypt/live/fileshare.acroe-ica.org/fullchain.pem (failure)

Running post-hook command: service haproxy start
1 renew failure(s), 0 parse failure(s)

IMPORTANT NOTES:

Mon serveur Web est (inclure la version) :
haproxy

Le système d’exploitation sur lequel mon serveur Web s’exécute est (version incluse) :
Linux srv-203 4.4.0-146-generic #172-Ubuntu SMP Wed Apr 3 09:00:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Je peux me connecter à un shell root sur ma machine (oui ou non, ou je ne sais pas) :
Oui !

J’utilise un panneau de configuration pour gérer mon site (non, ou fournit le nom et la version du panneau de configuration) :
non

J’utilise let’s encrypt sous ha proxy.
Jusqu’à récemment, la mise à jout des certificats se passait bien.
Depuis quelque jours, échec.

Quelques remarques :

** mon serveur haproxy est je crois bien joignable en HTTP. **
Exemple : http://www-test.eastndc.eu/

** à ma connaissance, aucune modification n’a été faite sur mes accès réseau **

** il ne me semble pas que le problème soit du à l’abandon de tls-sni-01 **
Mais comment puis-je le vérifier ?

merci par avance pour votre aide !!

2

Normal HTTP connections to your server work fine, as you observed. However, if I try to request http://fileshare.eastndc.eu/.well-known/acme-challenge/test I get a 503 instead of the usual redirect to HTTPS.

Perhaps your haproxy is configured to forward ACME challenge requests to Certbot? There are a few tutorials around that recommend setting things up this way, so maybe you followed one of those. If so, you don't need to stop haproxy during the renewal (that's the main reason for doing it that way). Try running the certbot command without the --pre-hook and --post-hook options.

If that doesn't help, could you share your haproxy configuration and also the certbot configuration from /etc/letsencrypt/renewal/fileshare.acroe-ica.org.conf please?

3

Thank you for your feedback John.

Sorry i could not answer yesterday : day off in France...

This morning, I discovered that some of the target servers (the server to which HAPROXY forward the https requests) has incorrect time. Also, Firefox indicates that there may be a time-setting problem : https://www-test.eastndc.eu/
Perhaps this could relate to the issue I have ? Dunno why, though...

I forgot to provide certbot version : certbot 0.31.0 [uptodate as far as I checked]

To answer your questions :

Perhaps your haproxy is configured to forward ACME challenge requests to Certbot?

To my knowledge, this is what was done (by a colleague, who is not here anymore)

Try running the certbot command without the --pre-hook and --post-hook options.

I obtain the same exact problem.

could you please share your haproxy configuration and also the certbot configuration from /etc/letsencrypt/renewal/fileshare.acroe-ica.org.conf please?

Here is the HAPROXY .cfg file :

global
log /dev/log local0
log /dev/log local1 notice
chroot /var/lib/haproxy
stats socket /run/haproxy/admin.sock mode 660 level admin
stats timeout 30s
user haproxy
group haproxy
daemon

#Default SSL material locations
ca-base /etc/ssl/certs
crt-base /etc/ssl/private

#Default ciphers to use on SSL-enabled listening sockets.
#For more information, see ciphers(1SSL). This list is from:
#Hardening Your Web Server’s SSL Ciphers
ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS
ssl-default-bind-options no-sslv3

defaults
log global
mode http
option httplog
option dontlognull
timeout connect 5000
timeout client 50000
timeout server 50000

#option forwardfor : permet au backend de savoir qui est le client réel (au lieu de voir HAProxy comme client)
option forwardfor

#option http-server-close : optimisation des performances en fermant les connexions tcp, tout en les maintenant dans l'état keep-alive
option http-server-close

errorfile 400 /etc/haproxy/errors/400.http
errorfile 403 /etc/haproxy/errors/403.http
errorfile 408 /etc/haproxy/errors/408.http
errorfile 500 /etc/haproxy/errors/500.http
errorfile 502 /etc/haproxy/errors/502.http
errorfile 503 /etc/haproxy/errors/503.http
errorfile 504 /etc/haproxy/errors/504.http

#------------------------------------------------------------------------------
#------------------------------------------------------------------------------
#------------------------------------------------------------------------------

#Frontend HTTP : listen port 80 / aiguillage vers les Backends selon l'URL d'entrée
frontend www-http
bind *:80
reqadd X-Forwarded-Proto:\ http

#aiguillage localhost letsencrypt
acl acl_path_letsencrypt path_beg /.well-known/acme-challenge/
use_backend letsencrypt-localhost-backend if acl_path_letsencrypt

#aiguillage vers NAS Fileshare
acl http_acl_host_fileshare hdr(host) -i fileshare.acroe-ica.org fileshare.eastndc.eu
use_backend nas-fileshare if http_acl_host_fileshare

#aiguillage vers WWW eastndc.eu
acl acl_host_www hdr(host) -i www-test.eastndc.eu
use_backend www-eastndc if acl_host_www

default_backend www-backend

#------------------------------------------------------------------------------

#Frontend HTTPS : listen port 443 / aiguillage identique vers les Backends selon l'URL d'entrée
#Prise en compte des certificats mentionnés après la directive bind *:443
frontend www-https
bind *:443 ssl crt /etc/haproxy/certs/fileshare.acroe-ica.org.pem
#Si plusieurs fichiers certificats, il faut les assembler selon la syntaxe ci-dessous
#bind *:443 ssl crt /etc/haproxy/certs/fileshare.acroe-ica.org.pem crt /etc/haproxy/certs/test.fileshare.eastndc.eu.pem
reqadd X-Forwarded-Proto:\ https

#aiguillage localhost letsencrypt
acl acl_path_letsencrypt path_beg /.well-known/acme-challenge/
use_backend letsencrypt-localhost-backend if acl_path_letsencrypt

#aiguillage vers NAS Fileshare
acl https_acl_host_fileshare hdr(host) -i fileshare.acroe-ica.org fileshare.eastndc.eu
use_backend nas-fileshare if https_acl_host_fileshare

#aiguillage vers WWW eastndc.eu
acl acl_host_www hdr(host) -i www-test.eastndc.eu
use_backend www-eastndc if acl_host_www

default_backend www-backend

#------------------------------------------------------------------------------
#------------------------------------------------------------------------------
#------------------------------------------------------------------------------

#Backend localhost:8888 spécifique pour le renouvellement des certificats let's encrypt
backend letsencrypt-localhost-backend
server letsencrypt 127.0.0.1:8888

#Backend du NAS-Fileshare
backend nas-fileshare
redirect scheme https if !{ ssl_fc }
server www 147.171.151.196:5000 check

#Backend du site web Drupal Eastndc
backend www-eastndc
#redirect scheme https if !{ ssl_fc }
server www 147.171.151.202:80 check

#Backend par défaut
backend www-backend
#redirect scheme https if !{ ssl_fc }
#server www 147.171.151.202:80 check
errorfile 500 /etc/haproxy/errors/500.http

And here is the letsencrypt fileshare.acroe-ica.org.conf file :

#renew_before_expiry = 30 days
version = 0.22.2
archive_dir = /etc/letsencrypt/archive/fileshare.acroe-ica.org
cert = /etc/letsencrypt/live/fileshare.acroe-ica.org/cert.pem
privkey = /etc/letsencrypt/live/fileshare.acroe-ica.org/privkey.pem
chain = /etc/letsencrypt/live/fileshare.acroe-ica.org/chain.pem
fullchain = /etc/letsencrypt/live/fileshare.acroe-ica.org/fullchain.pem

#Options used in the renewal process
[renewalparams]
authenticator = standalone
account = 96240ca70e8c08700a0f2b1fc8064d06
installer = None
http01_port = 8888
renew_hook = /root/haproxy_letsencrypt/concat_certificate.sh
post_hook = service haproxy start
pre_hook = service haproxy stop

Thank you much,
Nicolas

4

Okay, looks like that’s set up more or less as I thought, but the pre and post hooks are saved in the configuration so just omitting them from the command line doesn’t help.

Can you try deleting the last two lines from fileshare.acroe-ica.org.conf and then run certbot renew --dry-run please?

And if that works, then run certbot renew without the --dry-run

If you want to reload haproxy automatically after renewing (since you’ll no longer be stopping and starting it), it’s probably best to do that at the end of your /root/haproxy_letsencrypt/concat_certificate.sh script.

1 Like
5

Hello again John.

Well, I suspected also these two lines in fileshare.acroe-ica.org.conf.

Indeed, we did have to add the pre- and post- hook to stop/restard haproxy because of a conflict on port 443 between haproxy and certbot.
Our configuration functioned OK for months. I don’t know what caused it to fail this month.

Anyway, your last proposal solved the problem.
And the certificate are now operational, eg https://www-test.eastndc.eu/

Many, many thanks to you !
Nicolas

2 Likes