Certbot post-hook invoked only in dry-run mode

Help
1

Hi,

Using certbot-auto my certificates are renewed correctly, but post hook is not being triggered. However, running renew in --dry-run mode the post-hook is executed as it can be seen in the logs.

Works, but no hook information in the logs:

certbot-auto renew --noninteractive --no-self-upgrade --post-hook "service nginx reload"

Production log last entries:

KOqkqm57TH2H3eDJAkSnh6/DNFu0Qg==
-----END CERTIFICATE-----
2020-10-16 19:08:02,577:DEBUG:acme.client:Storing nonce: 0003HtifmEfWMkdVK9YDH4EAl2vSwJyFJIhX1VSxvLVcHAY
2020-10-16 19:08:02,578:DEBUG:certbot._internal.storage:Writing new private key to /etc/letsencrypt/archive/example.com/privkey2.pem.
2020-10-16 19:08:02,578:DEBUG:certbot._internal.storage:Writing certificate to /etc/letsencrypt/archive/example.com/cert2.pem.
2020-10-16 19:08:02,578:DEBUG:certbot._internal.storage:Writing chain to /etc/letsencrypt/archive/example.com/chain2.pem.
2020-10-16 19:08:02,578:DEBUG:certbot._internal.storage:Writing full chain to /etc/letsencrypt/archive/example.com/fullchain2.pem.
2020-10-16 19:08:02,658:DEBUG:certbot._internal.plugins.selection:Requested authenticator nginx and installer nginx
2020-10-16 19:08:02,658:DEBUG:certbot._internal.cli:Var installer=nginx (set by user).
2020-10-16 19:08:02,658:DEBUG:certbot._internal.cli:Var authenticator=nginx (set by user).
2020-10-16 19:08:02,659:DEBUG:certbot._internal.storage:Writing new config /etc/letsencrypt/renewal/example.com.conf.new.
2020-10-16 19:08:02,662:DEBUG:certbot._internal.reporter:Reporting to user: Congratulations! Your certificate and chain have been saved at:
/etc/letsencrypt/live/example.com/fullchain.pem
Your key file has been saved at:
/etc/letsencrypt/live/example.com/privkey.pem
Your cert will expire on 2021-01-14. To obtain a new or tweaked version of this certificate in the future, simply run certbot-auto again. To non-interactively renew all of $
2020-10-16 19:08:02,662:DEBUG:certbot._internal.reporter:Reporting to user: If you like Certbot, please consider supporting our work by:
Donating to ISRG / Let's Encrypt: Donate - Let's Encrypt
Donating to EFF: Support EFF's Work on Let's Encrypt | Electronic Frontier Foundation

However, in dry run mode, the Running post-hook command line is included in the log:

n5Z5MqkYhlMI3J1tPRTp1nEt9fyGspBOO05gi148Qasp+3N+svqKomoQglNoAxU=
-----END CERTIFICATE-----
2020-10-16 21:05:45,389:DEBUG:acme.client:Storing nonce: 0003XdjTtTkGC6v8TTNCw2FCVehwQm3STQo7XYTYWmpJuPc
2020-10-16 21:05:45,390:DEBUG:certbot._internal.renewal:Dry run: skipping updating lineage at /etc/letsencrypt/live/example.com
2020-10-16 21:05:45,390:DEBUG:certbot._internal.updater:Skipping renewal deployer in dry-run mode.
2020-10-16 21:05:46,408:DEBUG:certbot._internal.updater:Skipping updaters in dry-run mode.
2020-10-16 21:05:46,409:DEBUG:certbot._internal.renewal:no renewal failures
2020-10-16 21:05:46,409:INFO:certbot.compat.misc:Running post-hook command: service nginx reload

As a consequence of this inconsistent behavior nginx is not being reloaded after a new certificate is issued, so a manual reload is needed.

Anything wrong in my commands?
Any idea why the post hook command being executed only in dry run mode?

Any help would appreciated.

My web server is (include version):
nginx/1.17.3

The operating system my web server runs on is (include version):
Ubuntu 18.04 LTS

I can login to a root shell on my machine (yes or no, or I don't know):
yes

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):

$ certbot-auto --version
certbot 1.9.0

1 Like
2

I think this really confused me.
In theory, when your certbot use Installer nginx, it will automatically renew certificate and deploy / apply to your nginx instance by reloading.

In the dry-run log you shared, there's a line:

2020-10-16 21:05:45,390:DEBUG:certbot._internal.updater:Skipping renewal deployer in dry-run mode.

This should be the point where your nginx webserver is reloaded. (again, because of the nginx authenticator). I think the post-hook applied because those activities are skipped. But i'm not sure if there's any conflict or so when both commands are specified with the same purpose (to reload your nginx)

2 Likes
3

Hi and welcome to the community!

--post-hook and --deploy-hook run at different times (for different reasons)
I think what you are trying to do is more of a --deply-hook strategy.

[&2* readers: Get involved; Be heard. It starts with: if you read something you like, then like it :heart:]

3 Likes
4

@stevenzhu your expectation is correct when the --nginx parameter is used.
As I don't want certbot to touch my nginx sites configuration files, I intentionally do not use it.
Thanks for pointing out on the Skipping... line! I had totally missed it. :wink:

2 Likes
5

@rg305 the --deploy-hook did the trick.
Thank you very much for your help!

2 Likes
6

Just a P.S. Your production log states that installer=nginx is set. I'm really curious why that doesn't kick in.
Happy to see it's resolved.. Sorry i completely missed you are using post-hook not deploy-hook.

3 Likes
7

I am more confused as these posts go on.

--dry-run is supposed to call --pre-hook and --post-hook, and not call --deploy-hook.

--post-hook is supposed to run unconditionally.

Is it possible there is a typo in your invocation?

2 Likes
8

Found nginx set as the authenticator and installer in /etc/letsencrypt/renewal/my-site.conf - although I can't remember had set that option:

[renewalparams]
authenticator = nginx
installer = nginx
account = ...
server = ...

In another server/site also using certbot-auto I found the same entries + a post hook.

[renewalparams]
authenticator = nginx
installer = nginx
account = ...
server = ...
post_hook = service nginx reload

Does certbot update this file with the last execution params?

2 Likes
9

@jvanasco you're right, in my original question I was using post-hook with dry-run.

When combining --dry-run with --deploy-hook a new line is added to the log stating that the hook will be skipped, as expected:

Waiting for verification...
Cleaning up challenges
Dry run: skipping deploy hook command: service nginx reload
new certificate deployed with reload of nginx server; fullchain is ...

2 Likes
10

Trying to read between the lines a bit, I think there is indeed some confusing behavior in Certbot here.

In the first instance, we create a certificate like so:

certbot certonly --standalone -d example.com --post-hook "/bin/true"

Scenario #1: Renewal

If we run renewal like this (the "normal" way):

certbot renew --cert-name example.com --force-renewal

The post-hook will get executed. Great, that's what we expect.

Scenario #2: Renewal by means of running the original command again

If we once again run the following, but omit --post-hook:

certbot certonly --standalone -d example.com

Certbot will prompt us:

You have an existing certificate that has exactly the same domains or certificate name you requested and isn't close to expiry.
(ref: /etc/letsencrypt/renewal/example.com.conf)

What would you like to do?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: Keep the existing certificate for now
2: Renew & replace the cert (may be subject to CA rate limits)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate number [1-2] then [enter] (press 'c' to cancel):

Of course, we will select option 2 (renew), and we will expect that the original --post-hook parameter will be preserved.

However, it is not the case. The --post-hook will not be preserved, and what's more, it will be stripped from the renewal parameters file in future cases.

I haven't looked too closely, but I think this is the case because option (2) is not really renewal. It's more like "replace without preserving old renewal parameters". I am not sure whether this is an intentional design or not, but I will ask someone else on the team.

Back to OP's problem
I suspect strongly that this log ...

... was not from running certbot renew, but rather it was from a "Scenario #2" renewal.

My thinking here is simply that some of the logging which makes an appearance, cannot appear in a certbot renew invocation (such as the donation prompt).

2 Likes
11

@_az the first log entry I originally posted was indeed from the renew command, executed via a cron job. So it's pretty much your Scenario #1 without the --force parameter but including the --post-hook.

certbot-auto renew --noninteractive --no-self-upgrade --post-hook "service nginx reload"

Also note my usage of the --noninteractive flag, in case that's useful.

2 Likes
12

For the moment, I have to assume you are mistaken on this point, because of the way that code is written.

If you are willing to post the whole log file, we can verify one way or the other.

2 Likes
13

@_az Thanks for pointing me to the source code!

I was mistaken. Sorry:

I have multiple sites on this server and one of them had a certificate just issued minutes after the renew cron job had ran. So the log entries that I posted were actually from the certonly action instead of renew, as can be seen in the first lines of the now correct log file:

2020-10-16 19:07:56,872:DEBUG:certbot._internal.main:certbot version: 1.9.0
2020-10-16 19:07:56,873:DEBUG:certbot._internal.main:Arguments: ['-m', 'email@example.com', '--domains', 'my-site.com', '--nginx', '--non-interactive', '--agree-tos', '--k
eep-until-expiring']

This also explains what @stevenzhu has noticed: the usage of the nginx installer, as the new certificate was issued with the command line below:

certbot-auto certonly -m email@example.com
 --domains my-site.com
 --nginx --non-interactive --agree-tos
 --keep-until-expiring

So it seems the root cause is: my original issuing command did not use a --deploy-hook so this explains why nginx hadn't been reloaded after issuing the new certificate. Therefore I have no evidence it's not being reloaded on renewal.

I apologize for the confusion.
Thanks everyone for your help!

4 Likes
14

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.