Hey,
I've created a TLSA record manager that hooks into certbot internals to handle renewals. It successfully deals with rotating private keys so one doesn't need to pin (reuse) a private key — resulting in an improved OPSEC.
Feel free to use it: karolyi/daneupdate: DANE TLSA records updater using certbot - daneupdate - KSOL Gitea
You shouldn't replace certbot renew.
You have no way to control when certbot renew is going to be run, or what's going to run it: cron, systemd, something else? Also, what happens if you have more than one cert, and only use dane on some?
You might want to implement the changes as their own crontab line, or you might look up the at scheduling utility while using certbot's deploy hooks.
Not sure if you pin the end entity certificate or the intermediate, also.
Well, you could read the readme because a lot of your criticisms were addressed there before you've commented here. 
It's a tool I built for myself primarily, but others are free to use it, given it suits their needs.
Yes, you readme shouldn't require me to open the example config to understand what you're doing.
There is sufficient documentation in the readme about how to get it working, and I even hint there at things you were questioning here.
In the end, I suppose it's not for you so just ignore it.
Bumping for interest, so the bot doesn't close the topic.
Is there a reason why you haven't packaged this for distribution on pypi?
In terms of the package...
I am impressed this is properly typed and the code itself.
However, I am worried about the integration against Certbot internals and leveraging the private methods (For the non-python developers out there, functions with a leading underscore are typically internal APIs are unsupported and subject to change). This strategy is quite fragile and prone to break.
My suggestions for wider adoption:
1- Release on PyPi. If you need help getting the packaging structure for that, let me know.
2- Consider using CookieCutter to generate the sample configuration automatically. (see GitHub - cookiecutter/cookiecutter: A cross-platform command-line utility that creates projects from cookiecutters (project templates), e.g. Python package projects, C projects.). With cookiecutter, you could probably have a subcommand or separate command to generate this file.
3- Consider decoupling from Certbot code. At a minimum, you should be doing some version pinning and probably have a testsuite that you can run against each Certbot version to ensure the defs are still there and type signatures still match.
Overall, great work. Thanks for writing this and sharing with the greater community.
Hey,
thanks for your compliments; I'm working with python since 2002, hence the code quality.
My concerns against using certbot internals were the same as yours. However, since certbot uses a lot of plugins and configurations I don't want to implement separately, I figured sticking with the pinned versions of certbot should suffice, when the code is tested and working. How you test it currently, is a different question. : )
For the latter, some automated tests would have been nice (and PRs are welcome on them), I didn't take the time for those. As long as you use a 'nearby' certbot version that's probably installed on the system anyways, it should be okay.
My other concern while working with certbot internals was, certbot has loads of various command line flags, which I haven't implemented, and probably won't either. One has to go there with the currently available ones, and maybe extend them with ones wanted by adopters of the project.
The release on pypi won't happen because of the license. They aren't for free speech, which I am. That'll stay that way and I'm fine with that.
Thanks for the tip on the configuration generator, I'll look into it later and see if the project can be helped with it.
Fair response.
FWIW, if you ever consider decoupling our code and invoking Certbot directly, have you ever used psutil? I greatly prefer it to the stdlib's subprocess module.
The whole point of making my project Python-based was to be able to hook into certbot internals : ) Other projects are implemented in bash and/or don't use certbot: I wanted to keep the infrastructure of certbot functional, even when it costed me months to untangle the internals.
I'm afraid that unless certbot will have a way to put the newly fetched certificates somewhere else where the processes don't pick them up automatically, I can't decouple from using certbot internals. If you look at the functions and classes at cert.py, that entire module is responsible for circumventing RenewableCert's built-in certificate placement.
Generally, I prefer avoiding any external command caller utility, on accounts of them being a potential security problem. Python has lots of batteries built-in already, and you can use external modules that most of the time will achieve the same thing, with a little programming. : )
@karolyi willing to go to all that effort would suggest going with a memory safe language ACME implementation client as the base instead of anything Python-based. Here is a list of GO based ACME Client Implementations - Let's Encrypt
Yes, I know about alternative ACME clients.
But, since I was already using certbot (in my case with the nginx plugin in particular), I wanted to make my tool with certbot kept used.
I suggest being very very careful about versions of the
they really need to be absolutely in sync with the Certbot version you are using.
The Certbot --deploy-hook only runs when a new cert is issued. Was there a reason your script couldn't run as this hook? You could copy the just-issued cert from ../live/.. to your new folder structure and manage that as you have done.
The only difference is the triggering mechanism (hook versus renew replacement). Using a documented interface should be more reliable than relying on internals.
After thinking about @karolyi 's needs, --config-dir might be a better solution for this particular need.
--config-dir will override the default /etc/letsencrypt directory as being the base for reading and writing files. Certbot could be invoked with --config-dir=/etc/certbot-daneupdate to have a totally isolated interface that will not affect the other processes that monitor /etc/letsencrypt.
The daneupdate command could then migrate the certificates into the normal certbot directory once it is done, or just leave them as-is.
Maybe but it is messier. You must then use --config-dir for every Certbot command even just certbot certificates --config-dir (X). There's been more than a few people seeking help here for problems caused by that inconsistency.
In general I prefer letting Certbot do its thing with its usual folders. If you want something special done with the certs clone them and do it in your own space. Anyway, that was my thinking.
One skilled person could manage using --config-dir perhaps. But, as a tool suggested for a broader audience it seems fragile.
One skilled person could manage using --config-dir perhaps. But, as a tool suggested for a broader audience it seems fragile.
The tool @karolyi maintains is designed to replace using Certbot - instead of invoking certbot renew, users invoke the daneupdate command which handles the DANE pre/post work and wraps the Certbot functionality. The confusion @9peppe had above is from the line in the readme stating "It entirely replaces the certbot renew command" - the daneupdate does not alter certbot to replace the renew subcommand, but is a drop-in replacement for invoking Certbot, as in cerbot renew, entirely.
i.e the commandline would change as:
- (certbot-3.12.2) $ certbot renew
+ (daneupdate-3.12.2) $ daneupdate
Instead of interacting with Certbot's internals and private defs as the current system does, another approach would be for the daneupdate command to invoke Certbot with the necessary flags (such as) --config-dir. Under this approach, end users would never invoke Certbot – all the usage that requires --config-dir would happen programmatically through the daneupdate command.
Using --deploy-hook would not work for this setup for a few reasons, but primarily because there would still be a race condition between (1) when Certbot writes the files to disk and (2) when the --deploy-hook moves them. It is possible that a mixture of using a custom --cert-name in conjunction with --deploy-hook could avoid these problems, but I think that starts to overcomplicate things far more than an isolated --config-dir environment would - and all of this still needs to be wrapped into the pipeline of functionality the tool provides via a certbot plugin or invoking Cerbot through a subprocess, so there really isn't a concern for end-users getting confused.
@karolyi wants to avoid subprocesses, so this is likely not a solution for them, but I do think it is the least fragile and most manageable approach.
I don't see any inconsistency there, and that is pretty obvious. Some people are just rushed or stupid, and useful tools should not be abandoned or discouraged because of that.
Edit: The --deploy-hook idea I have is this- use a custom --cert-name=dane-{NAME}, then use a deploy hook to migrate that cert into whatever directory that needs to be. That would allow the cert to eventually be installed as if it were --cert-name={NAME} but without getting caught by process that is monitoring that directory for changes between originally writing the file to disk and whatever actions need to be taken beforehand in the deploy hook. I think should address the concerns , but I very well could be wrong as I'm not very familiar with the DANE integration and only skimmed the codebase.
I don't think we are that far apart. I understood the original design. I was intentionally asking about a different strategy.
I wanted to avoid the tightly-coupled integration with Certbot.
An outline:
Certbot component
Dane Update
What is a "good fresh cert"?
Side Notes
Anyway, it's just an idea.
For someone like myself who is ignorant about DANE, can anyone explain what action does the ACME client (or the cert update process overall) need to perform to update DANE certs?
From my limited understanding the main requirement is to add/update a TLSA record with a SHA256 hash of the new certificate, which seems to me like a deployment hook/task against your respective DNS API. Are there any other requirements for the certificate switch over?
I haven't done DANE, but in my guess propergation delay make webservers should not use new certificate before it propergated globally, so normal deploy hook like reloading webserver need to be done by dane deploy hook after it checked propergation