Certbot deploy hook failure is hidden very well

There was a problem with certificate renewal last night:

:
:
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/jenkins01.example.com.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Renewing an existing certificate for jenkins01.example.com
Hook 'deploy-hook' reported error code 1
Hook 'deploy-hook' ran with error output:
 Job for jenkins.service failed because the control process exited with error code.
 See "systemctl status jenkins.service" and "journalctl -xe" for details.
:
:

Obviously certbot recognized there was a problem, but at the end it reported

Congratulations, all renewals succeeded: 
  /etc/letsencrypt/live/... (success)
  /etc/letsencrypt/live/... (success)
  /etc/letsencrypt/live/jenkins01.example.com/fullchain.pem (success)
  /etc/letsencrypt/live/... (success)
  /etc/letsencrypt/live/... (success)
  /etc/letsencrypt/live/... (success)
  /etc/letsencrypt/live/... (success)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

This is weird. Did I miss some flag to tell certbot 2.1.0 to not ignore deploy hook failures?

What do you want Certbot to do if a deploy hook fails? It successfully renewed the certificate, it's not going to try the deployment again on subsequent runs. That's the behavior of Certbot and you will need to adapt your usage around it.

There is a related issue at certbot should exit non-zero if any deploy hook failed · Issue #8528 · certbot/certbot · GitHub.

3 Likes

As indicated in the subject line, certbot should not hide the failure in tons of output, but report it visibly in the summary at the end. In the current implementation I have to run the deploy scripts on my own to not miss a deployment failure.

Yes, I think it would be quite possible to track deploy script errors and report them in the final summary. Good idea.

3 Likes

On the other hand, it's a bit confusing to think whether it should attempt to preserve a copy of the error output from those deploy-hook failures in order to redisplay it (I think probably no, because it could be voluminous, but some users may be mad that Certbot told them some of their deploy-hooks were failing but not why, even though this is more information than Certbot would give them today).

3 Likes

Complex hooks can do their own logging and alerting. And probably should. Certbot reporting a general exit code error would still be helpful.

3 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.