Caution: certbot certonly doesn't use renewal configuration information

[raf]

This was going to be a question about why is it OK, that when renewing a certificate with the certonly subcommand, it doesn't run pre/post hooks (only deploy hooks) but the renew subcommand runs all of them. It didn't make sense to me that they wouldn't be run for any renewal, even when done via certonly, which is recommended by the user guide when you want to renew only one certificate instead of all of them. The user guide does only say that pre/post hooks are run with the renew subcommand, it doesn't mention certonly in relation to them, but that seemed wrong.

I think the user guide is giving bad guidance in this case. If you want to renew a single certificate, the only reliably correct to do it is to use the renew subcommand with --cert-name (and possibly --force-renewal).

Do not use the certonly subcommand with --cert-name (and possibly --force-renewal).

The user guide recommends this, but it doesn't always work, and the consequences can be bad. The certonly and renew subcommands are very different (even when --force-renewal is used with both so they are both renewing).

• certonly doesn't run pre/post hooks
• certonly doesn't seem to consult some [renewalparams] in the renewal config file
• certonly removes some [renewalparams] from the renewal config file

This means that, if you did run/certonly/renew --reuse-key in the past, and have "reuse_key = True" in your renewal config file, and you have it there for a reason (e.g. DANE), the renew subcommand will honour it, but the certonly subcommand will ignore it and create a new keypair, and delete "reuse_key = True" from the renewal config file. This could be disastrous.

It also ignores the stored authenticator parameter as well, which is an inconvenience.

And there might be other parameters that it ignores and then removes.

But presumably, it does use the "account" parameter or it wouldn't work at all. It's strangely selective about which [renewalparams] it uses and which ones it doesn't use (and then discards).

Admittedly, it's fine if you don't need pre/post hooks for the renewal to work, and if you don't have anything important in your [renewalparams], but the user guide doesn't know that. It should provide recommendations that always work, or explain when their recommendations do and don't work.

And I could have also supplied --reuse-key along with certonly, and it would have done what I wanted (I just checked), but I knew that it had been saved in the renewal config file, and so it never occurred to me that I would have to supply it again. Certbot already knew I wanted to reuse the key. It was in the renewal config file. That's what persistence means.

Should I log an issue for the user guide to be changed so as to not recommend certonly for renewals?

Should I log an issue for certonly to be changed to act exactly like renew (and so do renewals properly) when it is renewing a certificate, rather than obtaining an entirely new one that wasn't there before?

I think the second is the better option (fewer surprises), but the first might be actioned quicker and help people using older versions of certbot.

On the other hand, the existing certonly behaviour could be desirable as a way of erasing, via the command line, all previously persisted [renewalparams] from the renewal config file. But it's not documented as having that purpose. Although some users may have noticed it and be using it in that way.

It might seem obvious that it's better to use renew than certonly for renewals, and no doubt, that's clear in hindsight, but the problem is that the user guide recommends it, and the certonly subcommand appears to go along with it (by accepting the --force-renewal option), just badly.

Any thoughts?

cheers,
raf

[Osiris]

@raf

the user guide

Which user guide are you refering to?

Also, --force-renewal shouldn't be necessary if the certificate is up for renewal. E.g. <30 days until expiry.

Further more, there shouldn't be any issue with certonly, unless you've found a bug in certbot. But I must say, your post is rather long and I have a hard time extracting key points from all the text.

[raf]

I'm referring to this user guide:

User Guide — Certbot 1.19.0.dev0 documentation

Specifically, the manpage section near the bottom where it says:

renew:
The 'renew' subcommand will attempt to renew all certificates (or more
precisely, certificate lineages) you have previously obtained if they are
close to expiry, and print a summary of the results. By default, 'renew'
will reuse the options used to create obtain or most recently successfully
renew each certificate lineage. You can try it with --dry-run first. For
more fine-grained control, you can renew individual lineages with the
certonly subcommand. Hooks are available to run commands before and
after renewal; see User Guide — Certbot 1.19.0.dev0 documentation for
more information on these.

The sentence that I think needs to change is this one:

For more fine-grained control, you can renew individual lineages with the certonly subcommand

I think it should say:

For more fine-grained control, you can renew individual lineages withe the --cert-name option.

Also, it might be good to document the fact that the certonly subcommand replaces any persisted renewal configuration parameters, since that is part of what it does.

[Osiris]

I think perhaps the "fine-grained control" which is mentioned refers to the fact that with certonly you can indeed "modify" the contents of the certificate renewal configuration, e.g. add/remove hostnames et cetera. Although I agree the current documentation wording leaves a few things to the imagination. Perhaps mention certonly as wel as --cert-name?

Note that it should also be possible to overwrite renewal configuration parameters using renew. E.g., if you want to add the must staple extension, you can do: certbot renew --must-staple which should both add the extension to the certificates as wel as add it to the renewal configuration file.

[raf]

I don't think that's correct. Once you have used --reuse-key with run/certonly/renew, and reuse_key = True is added to the renewal config file, it can't be removed via the renew subcommand. There would need to be a --no-reuse-key option for that to be possible. I thought there was no way to do it at all via the command line, but it seems that it can be done with the certonly subcommand. But that's not a (clearly) documented feature of the certonly subcommand. It seems like more of a side effect. And it's a bit of a sledgehammer. If there are any other stored renewalparams, it removes them all (replacing them all with whatever you supply on the certonly command line).

I think that the term "fine-grained control" there only refers to the ability to renew less than the whole set of certificate lineages. I wouldn't read any more into it than that. But as I said, --cert-name lets you do the same thing but without the surprises. Neither subcommand provides fine-grained control in the sense of being able to turn individual parameters on and off individually. They can be turned on individually, but they can't be turned off individually. To turn parameters off, you need to wipe them all out with certonly, and start again. All of this would be fine, if the renewal config file was documented and there was a certbot-checkconf tool to tell you if you made a mistake. But if they want people to make edits purely via the command line, like the documentation states, then the command line interface should support all the possibilities anyone might need. But I'm getting off-topic. Sorry.

[griffin]

Welcome to the Let's Encrypt Community

I see that you have discovered the oddities of the certbot guide. Sorry for the confusion. You're not alone in the chorus of saying that it desperately needs clarification. Hi! I'm the choir director.

@raf

when renewing a certificate with the certonly subcommand

You already know the following by now, but I'm going to expound them here for later audiences.

You can't "renew" a certificate with the certonly subcommand because renew is itself a subcommand.

These are the (mutually exclusive) subcommands of certbot (with run being the default when no subcommand is specified).

certonly
install
run
renew
certificates
delete
enhance
revoke
register
update_account
unregister

certbot certonly doesn't run the stored pre, post, or deploy hooks associated with the renewal configuration of a certificate because the renew subcommand isn't being run. That doesn't stop you from specifying those hooks via their associated parameters when using certonly. Indeed, that's how those hooks get in the renewal configuration in the first place.

If you want to only renew a single certificate, the proper syntax is:

certbot renew --cert-name NAMEOFCERTTORENEW

Avoid using --force-renewal like a flesh-eating plague. It almost never results in anything good.

[griffin]

@raf

For more fine-grained control, you can renew individual lineages with the certonly subcommand.

Why is that terrible line in the guide?

[griffin]

@raf

Do not use the certonly subcommand with --cert-name

You often want to use --cert-name with certonly to be certain of which lineage's renewal parameters you are overwriting with certonly's execution.

[griffin]

@raf

even when --force-renewal is used with both so they are both renewing

The (awful) --force-renewal parameter has no specific affiliation with the renew subcommand.

First, a huge clarification:

@griffin

Let's Encrypt...

• sends an expiration email twenty days, ten days, and one day before a certificate expires
• will not send an expiration email for a renewed certificate
• considers a certificate to be renewed if a newer duplicate of that certificate has been issued
• considers a certificate to be a duplicate of another certificate if both certificates have the exact same set of subject alternative names (SANs)

A renewal certificate is just a new certificate that happens to be a duplicate certificate.

The (awful) --force-renewal parameter simply tells certbot to acquire a duplicate certificate even if the current certificate is more than 30 days from expiration. If said current certificate doesn't exist, the awful parameter does nothing (meaning that a new certificate is simply acquired).

[griffin]

@raf

certonly doesn't run pre/post/deploy hooks

Yes, it does, if they're specified via command-line parameters.

[griffin]

@raf

This means that, if you did run/certonly/renew --reuse-key in the past, and have "reuse_key = True" in your renewal config file, and you have it there for a reason (e.g. DANE), the renew subcommand will honour it, but the certonly subcommand will ignore it and create a new keypair, and delete "reuse_key = True" from the renewal config file. This could be disastrous.

It also ignores the stored authenticator parameter as well, which is an inconvenience.

And there might be other parameters that it ignores and then removes.

But presumably, it does use the "account" parameter or it wouldn't work at all. It's strangely selective about which [renewalparams] it uses and which ones it doesn't use (and then discards).

Admittedly, it's fine if you don't need pre/post hooks for the renewal to work, and if you don't have anything important in your [renewalparams], but the user guide doesn't know that. It should provide recommendations that always work, or explain when their recommendations do and don't work.

And I could have also supplied --reuse-key along with certonly, and it would have done what I wanted (I just checked), but I knew that it had been saved in the renewal config file, and so it never occurred to me that I would have to supply it again. Certbot already knew I wanted to reuse the key. It was in the renewal config file. That's what persistence means.

The moral of this story:

• Use renew to use the renewal parameters.
• Use run or certonly to set the renewal parameters.

Failure to understand that paradigm leads to disaster.

@raf

Should I log an issue for the user guide to be changed so as to not recommend certonly for renewals?

Possibly, though it's just the tip of the iceberg.

@raf

Should I log an issue for certonly to be changed to act exactly like renew (and so do renewals properly) when it is renewing a certificate, rather than obtaining an entirely new one that wasn't there before?

Absolutely not. That would be truly terrible and make certbot unusable.

@raf

I think the second is the better option (fewer surprises), but the first might be actioned quicker and help people using older versions of certbot.

Understanding the moral above is all that's necessary.

@raf

On the other hand, the existing certonly behaviour could be desirable as a way of erasing, via the command line, all previously persisted [renewalparams] from the renewal config file. But it's not documented as having that purpose. Although some users may have noticed it and be using it in that way.

It might seem obvious that it's better to use renew than certonly for renewals, and no doubt, that's clear in hindsight, but the problem is that the user guide recommends it, and the certonly subcommand appears to go along with it (by accepting the --force-renewal option), just badly.

You got it.

[griffin]

@raf

For more fine-grained control, you can renew individual lineages withe the --cert-name option.

Also, it might be good to document the fact that the certonly subcommand replaces any persisted renewal configuration parameters, since that is part of what it does.

I couldn't agree more. Once I have the time, I hope to create a PR to overhaul the entire guide. For now though, these changes would be helpful.

[griffin]

@raf

Once you have used --reuse-key with run/certonly/renew, and reuse_key = True is added to the renewal config file, it can't be removed via the renew subcommand. There would need to be a --no-reuse-key option for that to be possible. I thought there was no way to do it at all via the command line, but it seems that it can be done with the certonly subcommand. But that's not a (clearly) documented feature of the certonly subcommand. It seems like more of a side effect. And it's a bit of a sledgehammer. If there are any other stored renewalparams, it removes them all (replacing them all with whatever you supply on the certonly command line).

I think that the term "fine-grained control" there only refers to the ability to renew less than the whole set of certificate lineages. I wouldn't read any more into it than that. But as I said, --cert-name lets you do the same thing but without the surprises. Neither subcommand provides fine-grained control in the sense of being able to turn individual parameters on and off individually. They can be turned on individually, but they can't be turned off individually. To turn parameters off, you need to wipe them all out with certonly, and start again. All of this would be fine, if the renewal config file was documented and there was a certbot-checkconf tool to tell you if you made a mistake. But if they want people to make edits purely via the command line, like the documentation states, then the command line interface should support all the possibilities anyone might need.

Bullseye!

As a note, certbot certonly absolutely can renew a certificate (by acquiring a duplicate certificate then updating the corresponding, existing lineage). Just be aware that certonly doesn't use the existing renewal parameters for that lineage, but sets the renewal parameters for that lineage going forward.

[Osiris]

@raf

I don't think that's correct. Once you have used --reuse-key with run/certonly/renew, and reuse_key = True is added to the renewal config file, it can't be removed via the renew subcommand. There would need to be a --no-reuse-key option for that to be possible.

True, it only works for adding stuff currently.

@raf

I thought there was no way to do it at all via the command line, but it seems that it can be done with the certonly subcommand. But that's not a (clearly) documented feature of the certonly subcommand. It seems like more of a side effect. And it's a bit of a sledgehammer. If there are any other stored renewalparams, it removes them all (replacing them all with whatever you supply on the certonly command line).

You're absolutely correct. In that sense I personally wouldn't use "certonly" as an alternative for "renewal", as it clearly functions completely differently. Although technically the statement is correct: issuing a certificate with the same hostnames is the definition of a renewal (as "renewals" are technically brand new certificates anyway) and that's perfectly possible using certonly, but it has many caveats: any previously used option would need to be used with the """renewal""" too indeed. This should be emphasised in the documentation IMHO.

@raf

I think that the term "fine-grained control" there only refers to the ability to renew less than the whole set of certificate lineages. I wouldn't read any more into it than that.

That can also be done with renew in combination with --cert-name as you've also stated, so I believe they actually mean adding/removing certain options for a "renewal" when using certonly. For example, your --reuse-key can be removed by using certonly but not with renew.

That said I too think certbot is lacking many user-friendly options for renewal configuration manipulation. In my opinion it should even be possible to easily modify the renewal configuration file WITHOUT actually issuing a certificate. For example, "toggle" the reuse-key option. On, off, on, off, whatever the user wants, without issuing a certificate.

[raf]

Thanks for the feedback. It's much appreciated.

@griffin

Avoid using --force-renewal like a flesh-eating plague. It almost never results in anything good.

Thanks for the advice. I'm sure it's almost always correct, but for the obscure thing I'm doing (DANE), it's important maybe once a year (and in an emergency). Don't ask.

@Osiris

In my opinion it should even be possible to easily modify the renewal configuration file WITHOUT actually issuing a certificate

This gets mentioned in the issue tracker, and it might happen one day, but apparently there are difficulties. But they're thinking about it. I'd also like that, but I'd also like the option of a documented renewal config file format and a certbot-checkconf tool to tell me if I've made an invalid change. That might be quicker and easier than overcoming the difficulties they refer to in the issue tracker. But in lieu of that, a command line interface that can perform any valid config file edit (and nothing else) would be great.