Using Hashicorp Vault to save and restore certbot files


#1

I would like to preserve all certbot state in the Hashicorp’s Vault system, and then restore that state to check for renewal. This way cert renewal could be done on any Jenkins worker. I could preserve the whole state as a gzip archive, but I would rather just store the needed files as individual keys, thus being able to use them directly in other software like Terraform. Which files are required to store and restore for the certbot to be able to work?

I use route53 DNS method, manually using the certificate files afterwards. Thanks!!


#2

Hi @nyurik,

Can this method preserve symbolic link structure?

Certbot expects all of the files in these four directories

/etc/letsencrypt/renewal
/etc/letsencrypt/archive
/etc/letsencrypt/live
/etc/letsencrypt/accounts

to be preserved in order to perform renewal. Other files aren’t necessary. Most of the files in /etc/letsencrypt/live are symbolic links that point to items in /etc/letsencrypt/archive; these symbolic links, including both their names and their destinations are necessary because Certbot uses them to keep track of renewal status.


#3

I actually had this exact idea a week or two ago, but after some investigation, it seemed like Certbot uses unabstracted filesystem operations in too many places to fully replace the persistence mechanism.

It seems like you could just have a script that publishes to Vault as a --deploy-hook to do parts of, though - the script would have access to the names and the certificate and private keys.

I’m not sure how Vault is about binary storage, but you could also tar up the directory preserving symlinks, base64 it, and restore it back?


#4

Hi @schoen, thanks for a quick reply! So I guess I only need to preserve config dir, not the work dir or log dir, right? I was hoping to just copy individual files to the state store, without symlinks, logs, or temporary files, and restore them into a new directory right before attempting a renewal. Of course I could preserve the whole dir by gziping it into a single value, but it seems hackish.


#5

You can basically do this, but the symlinks are part of the structure that tells Certbot what the current version of a certificate is in order to decide whether to renew. (Among other things, Certbot will expect to follow a symlink in /etc/letsencrypt/live/example.com/cert.pem in order to find the X.509 file to read to determine when the certificate is expiring. Because of our implementation of file versioning, this object and the other three items within the same directory must be symlinks.)

There might be another ACME client that represents certificate state in a way that’s more easily compatible with what you want to do.


#6

@schoen thanks for the explanation. I am not sure about other clients because I need Route53 plugin as well, and certbot is a well known/respected/versatile system I would rather use. Finding an alternative doesn’t seem to be trivial. I will try to tar czpvf config.tgz config/ and store it as a binary or base64 value, and restore it with tar -xhzvf config.tgz when needed.

P.S. Do I need to remove older certificates by hand, or will it only keep up to N certs?


#7

Currently it keeps an unlimited number of old certificates and private keys.


#8

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.