Certbot renewal fails with binding error or authentication failure

[lammikko]

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. https://crt.sh/?q=example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: slow.tarmo.cz

I ran this command: certbot renew -q

It produced this output:

Challenge failed for domain slow.tarmo.cz
Attempting to renew cert (slow.tarmo.cz) from /usr/local/etc/letsencrypt/renewal/slow.tarmo.cz.conf produced an unexpected error: Some challenges have failed.. Skipping.
Challenge failed for domain tarmo.cz
Attempting to renew cert (tarmo.cz) from /usr/local/etc/letsencrypt/renewal/tarmo.cz.conf produced an unexpected error: Some challenges have failed.. Skipping.
All renewal attempts failed. The following certs could not be renewed:
  /usr/local/etc/letsencrypt/live/slow.tarmo.cz/fullchain.pem (failure)
  /usr/local/etc/letsencrypt/live/tarmo.cz/fullchain.pem (failure)
2 renew failure(s), 0 parse failure(s)

My web server is (include version): Apache/2.4.46 (FreeBSD)

The operating system my web server runs on is (include version): FreeBSD 12.1-RELEASE-p10

My hosting provider, if applicable, is:

I can login to a root shell on my machine (yes or no, or I don't know): Yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): No

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): certbot 1.5.0

[lammikko]

The above is if Apache is stopped. If Apache is started, the output is:

Attempting to renew cert (slow.tarmo.cz) from /usr/local/etc/letsencrypt/renewal/slow.tarmo.cz.conf produced an unexpected error: Problem binding to port 80: Could not bind to IPv4 or IPv6.. Skipping.
Attempting to renew cert (tarmo.cz) from /usr/local/etc/letsencrypt/renewal/tarmo.cz.conf produced an unexpected error: Problem binding to port 80: Could not bind to IPv4 or IPv6.. Skipping.
All renewal attempts failed. The following certs could not be renewed:
  /usr/local/etc/letsencrypt/live/slow.tarmo.cz/fullchain.pem (failure)
  /usr/local/etc/letsencrypt/live/tarmo.cz/fullchain.pem (failure)
2 renew failure(s), 0 parse failure(s)

[_az]

Hi @lammikko,

Could you please run certbot renew without the -q? That would give more fruitful output, including the actual validation failures.

As a tentative answer, it seems like firewalling could be a potential cause?

I can connect to your IP on port 80 from some hosts, but not from others, including not from https://letsdebug.net/slow.tarmo.cz/306547.

[lammikko]

Thank you, @_az,

That actually solved my problem. My firewall blocked incoming connections from wherever the challenge connection to .well-known/acme-challenge happened to come from. I stopped pf, ran the renewal, it succeeded, and then started pf again.

Can I bundle the stopping of two services in the cron script?

As in, change
certbot renew -q --pre-hook 'service haproxy stop' --post-hook 'service haproxy start'
to something like
certbot renew -q --pre-hook 'service haproxy stop && service pf stop' --post-hook 'service haproxy start && service pf start'?

[_az]

Sure, you can also throw some executable scripts into /usr/local/etc/letsencrypt/renewal-hooks/pre/ and /usr/local/etc/letsencrypt/renewal-hooks/post/ to stop and start pf, respectively.