Hi. I'm setting up a DANE-friendly Certbot workflow.
See DNS-based Authentication of Named Entities - Wikipedia
for a short introduction, or RFC6698, RFC7671, RFC7672 for the gory details.
I'm only interested in TLSA 3 1 1 records. They only refer to the public key
itself and nothing else up the chain. This isn't about TLSA 2 1 1 records,
or any other kind of TLSA record.
I think I have a good idea of what needs to happen, but I'd like
confirmation that what I am attempting can work, and I have a question about
certbot usage that I don't know the (official) answer to yet. Any advice
would be greatly appreciated.
Normally, certbot automatically renews a certificate as it approaches
expiry, and at the same time, it replaces (using symlinks) the existing
certificate with the new certificate. Each time this happens, by default,
there is a new keypair as well as a new certificate.
For DANE, I need to be able to create the replacement keypair and
certificate well in advance of the existing keypair and certificate expiring
and being deactivated. So I need two keypairs and two certificates for the
same set of domains in existence at the same time. And I need to be able to
decide when to deactivate the old keypair/certificate and activate the new
keypair/certificate. That will be done on a different schedule (of my
choosing) to certbot's schedule for renewing certificates.
The replacement keypair/certificate needs to be created well in advance so
as to give me plenty of time to create the corresponding TLSA records
(unless I automate that), and to give remote caching DNS servers enough time
to have access to both the old and the new TLSA records before the new
keypair/certificate becomes active (i.e. at least a few DNS TTLs worth of
time, however long that is). But rather than thinking about TTLs, it's
recommended to just have two keys with published TLSA records at all times:
the current one, and the next one. That makes it possible for even emergency
key rollovers to be as unexciting as possible.
While both certificates are automatically renewing, I want them to keep
their existing keypairs. That way, the certificates can still renew every
three months, but the keypairs, and hence the corresponding TLSA records,
can stay the same until I choose to rollover the keypair. That might be
manually triggered, or I might automated it (e.g. annually). I'm not
expecting the key rollover to be handled by certbot. I'll automate or
script it myself.
So, this is what I think I need to do:
- Instruct certbot to create a keypair/certificate (the original)
- Instruct certbot to create another keypair/certificate (the duplicate)
- Instruct certbot to reuse the original keypair when renewing
- Instruct certbot to reuse the duplicate keypair when renewing
- Designate the original certificate as current (with symlinks)
- Designate the duplicate certificate as next (with symlinks)
- Publish TLSA 3 1 1 records for the original key
- Publish TLSA 3 1 1 records for the duplicate key
- Configure services to use the current keypair/certificate
These two keypairs/certificates will act as a set of current/next
keypairs/certificates for the set of domains. Certbot will automatically
renew the certificates every three months without creating new keypairs each
time.
When I want to rollover the key from the original to the duplicate:
- Swap the current/next roles (symlinks) and reload any affected services
- Remove old TLSA 3 1 1 records for the just deactivated (original) key
- Instruct certbot to renew the next (original) certificate with a new keypair
- Publish new TLSA 3 1 1 records for the just created next (original) key
When I want to rollover the key from the duplicate to the original:
- Swap the current/next roles (symlinks) and reload any affected services
- Remove old TLSA 3 1 1 records for the just deactivated (duplicate) key
- Instruct certbot to renew the next (duplicate) certificate with a new keypair
- Publish new TLSA 3 1 1 records for the just created next (duplicate) key
What I'd like help with is the actual certbot commands to do all of this
(see below), and how to arrange for the automatic renewals to reuse the
keypair, but for the less frequent key rollover forced renewals to create a
new keypair.
My understanding is that command line options used with the certbot renew or
certonly subcommands (such as --reuse-key) cause undocumented magic to
happen to the renewal config file so that its effect persists to subsequent
renewals. And I know that that this persistence can be suppressed with the
--disable-renew-updates option. But I can't see how to set up --reuse-key
persistently, but then temporarily disable it when doing the key rollover
forced renewal, so that it goes back to reusing the key again afterwards for
normal automated renewals.
I expect to need to create both certificates (one the usual way, and one
with --duplicate), then immediately renew them both with --reuse-key, so
that subsequent automatic renewals keep the same keypairs [Correction:
--reuse-key can be supplied when initially creating both
keypairs/certificates]. Then, when doing a key rollover, renew the next
certificate with --force-renewal and --no-reuse-key and
--disable-renew-updates, so as to create a new keypair with the new
certificate, but then go back to reusing the key for subsequent automatic
renewals.
But I can't find a --no-reuse-key option in the documentation.
It looks like, once you have used --reuse-key, you can't go back to not
reusing the key (at least, not via the certbot command line). Is that the
case or am I missing something?
What's the best way to temporarily override the persistent effect of an
earlier use of --reuse-key? Do I just have to delete the "reuse_key = True"
from the renewal config file beforehand, and then put it back afterwards? Or
is there a command line option for the renew/certonly subcommands that has
the effect of removing the "reuse_key = True" from the renewal config file
that was put there by an earlier use of --reuse-key? If so, could I use that
with --disable-renew-updates so that the creation of a new keypair is done
once, but isn't made persistent, so that subsequent automatic renewals will
go back to reusing the existing key?
Removing "reuse_key = True" from the renewal config file and putting it back
again does seem to work, but I'd rather not rely on undocumented behaviour.
It'll also be good when there's a documented way to update the renewal
config file without also performing an otherwise unwanted certificate
renewal.
Another thing to note is that, after creating the duplicate
keypair/certificate, any subsequent use of the --expand option, to change
which domains are included in these certificates, will need to also use the
--cert-name option so as to prevent ambiguity, and be performed on both the
original and the duplicate certificates.
This is what I think the workflow will look like in detail:
# Create the original keypair/certificate (reusing the keypair thereafter)
certbot run --apache --reuse-key \
-d example.org -d www.example.org -d mta-sts.example.org -d mail.example.org
# Create the duplicate keypair/certificate (and choose the new cert-name)
certbot certonly --apache --duplicate --reuse-key --cert-name example.org-duplicate \
-d example.org -d www.example.org -d mta-sts.example.org -d mail.example.org
# Designate the original/duplicate keypairs/certificates as current/next via symlinks
mkdir /etc/letsencrypt/next
mkdir /etc/letsencrypt/current
ln -s /etc/letsencrypt/live/example.org /etc/letsencrypt/current/example.org
ln -s /etc/letsencrypt/live/example.org-duplicate /etc/letsencrypt/next/example.org
# Publish TLSA records to DNS for both keys (one per hostname/port/key combination)
# See postfix-tls(1) (look for output-server-tlsa) for one way to generate these. e.g.:
# postfix tls output-server-tlsa -h mail.example.org /etc/letsencrypt/current/example.org/privkey.pem
# postfix tls output-server-tlsa -h mail.example.org /etc/letsencrypt/next/example.org/privkey.pem
_443._tcp IN TLSA 3 1 1 ... (for /etc/letsencrypt/current/example.org)
_443._tcp.www IN TLSA 3 1 1 ... (for /etc/letsencrypt/current/example.org)
_443._tcp.mta-sts IN TLSA 3 1 1 ... (for /etc/letsencrypt/current/example.org)
_25._tcp.mail IN TLSA 3 1 1 ... (for /etc/letsencrypt/current/example.org)
_465._tcp.mail IN TLSA 3 1 1 ... (for /etc/letsencrypt/current/example.org)
_587._tcp.mail IN TLSA 3 1 1 ... (for /etc/letsencrypt/current/example.org)
_993._tcp.mail IN TLSA 3 1 1 ... (for /etc/letsencrypt/current/example.org)
_995._tcp.mail IN TLSA 3 1 1 ... (for /etc/letsencrypt/current/example.org)
_443._tcp IN TLSA 3 1 1 ... (for /etc/letsencrypt/next/example.org)
_443._tcp.www IN TLSA 3 1 1 ... (for /etc/letsencrypt/next/example.org)
_443._tcp.mta-sts IN TLSA 3 1 1 ... (for /etc/letsencrypt/next/example.org)
_25._tcp.mail IN TLSA 3 1 1 ... (for /etc/letsencrypt/next/example.org)
_465._tcp.mail IN TLSA 3 1 1 ... (for /etc/letsencrypt/next/example.org)
_587._tcp.mail IN TLSA 3 1 1 ... (for /etc/letsencrypt/next/example.org)
_993._tcp.mail IN TLSA 3 1 1 ... (for /etc/letsencrypt/next/example.org)
_995._tcp.mail IN TLSA 3 1 1 ... (for /etc/letsencrypt/next/example.org)
# Configure web/mail services to use the current certificate
/etc/letsencrypt/current/example.org/privkey.pem
/etc/letsencrypt/current/example.org/fullchain.pem
# A year later..., rollover from the original to the duplicate
# Swap role symlinks (would be faster/better in a language other than shell)
rm /etc/letsencrypt/next/example.org
rm /etc/letsencrypt/current/example.org
ln -s /etc/letsencrypt/live/example.org-duplicate /etc/letsencrypt/current/example.org
ln -s /etc/letsencrypt/live/example.org /etc/letsencrypt/next/example.org
Reload any affected services (e.g. dovecot)
# Remove old TLSA records from DNS
Remove TLSA records (for the old key that is now in /etc/letsencrypt/next/example.org)
# Replace the newly next keypair/certificate (original)
# Before: Remove "reuse_key = True" from the renewal config file
perl -pi -e 's/reuse_key = True\n//' /etc/letsencrypt/renewal/example.org.conf
# Create the new next keypair/certificate (original)
certbot certonly --force-renewal --cert-name example.org
# After: Restore "reuse_key = True" to the renewal config file
echo 'reuse_key = True' >> /etc/letsencrypt/renewal/example.org.conf
# Q: Is there a certbot-approved way to temporarily override reuse_key = True?
# Publish TLSA records to DNS for new next key
_443._tcp IN TLSA 3 1 1 ... (for /etc/letsencrypt/next/example.org)
_443._tcp.www IN TLSA 3 1 1 ... (for /etc/letsencrypt/next/example.org)
_443._tcp.mta-sts IN TLSA 3 1 1 ... (for /etc/letsencrypt/next/example.org)
_25._tcp.mail IN TLSA 3 1 1 ... (for /etc/letsencrypt/next/example.org)
_465._tcp.mail IN TLSA 3 1 1 ... (for /etc/letsencrypt/next/example.org)
_587._tcp.mail IN TLSA 3 1 1 ... (for /etc/letsencrypt/next/example.org)
_993._tcp.mail IN TLSA 3 1 1 ... (for /etc/letsencrypt/next/example.org)
_995._tcp.mail IN TLSA 3 1 1 ... (for /etc/letsencrypt/next/example.org)
# A year later..., rollover from the duplicate to the original
# Swap role symlinks (would be faster/better in a language other than shell)
rm /etc/letsencrypt/next/example.org
rm /etc/letsencrypt/current/example.org
ln -s /etc/letsencrypt/live/example.org /etc/letsencrypt/current/example.org
ln -s /etc/letsencrypt/live/example.org-duplicate /etc/letsencrypt/next/example.org
Reload any affected services (e.g. dovecot)
# Remove old TLSA records from DNS
Remove TLSA records (for the old key that is now in /etc/letsencrypt/next/example.org)
# Replace the newly next keypair/certificate (duplicate)
# Before: Remove "reuse_key = True" from the renewal config file
perl -pi -e 's/reuse_key = True\n//' /etc/letsencrypt/renewal/example.org-duplicate.conf
# Create the new next keypair/certificate (duplicate)
certbot certonly --force-renewal --cert-name example.org-duplicate
# After: Restore "reuse_key = True" to the renewal config file
echo 'reuse_key = True' >> /etc/letsencrypt/renewal/example.org-duplicate.conf
# Q: Is there a certbot-approved way to temporarily override reuse_key=True?
# Publish TLSA records to DNS for new next key
_443._tcp IN TLSA 3 1 1 ... (for /etc/letsencrypt/next/example.org)
_443._tcp.www IN TLSA 3 1 1 ... (for /etc/letsencrypt/next/example.org)
_443._tcp.mta-sts IN TLSA 3 1 1 ... (for /etc/letsencrypt/next/example.org)
_25._tcp.mail IN TLSA 3 1 1 ... (for /etc/letsencrypt/next/example.org)
_465._tcp.mail IN TLSA 3 1 1 ... (for /etc/letsencrypt/next/example.org)
_587._tcp.mail IN TLSA 3 1 1 ... (for /etc/letsencrypt/next/example.org)
_993._tcp.mail IN TLSA 3 1 1 ... (for /etc/letsencrypt/next/example.org)
_995._tcp.mail IN TLSA 3 1 1 ... (for /etc/letsencrypt/next/example.org)
# And repeat annually, to rollover back and forth between the original and the duplicate
If anyone can suggest a better way to temporarily override "reuse_key = True",
or see anything I've done wrong, or point out anything I've misunderstood,
any advice would be greatly appreciated.
The certbot commands themselves all seem to work, as does the nasty
temporary overriding of "reuse_key = True" in the renewal config file. So
I'm hopeful that I can automate this, or at least mostly script it, without
much trouble.
But I suggest that a --no-reuse-key option be added to certbot's command
line interface. I've opened an issue on github for it. In fact any option
that causes a persistent change to the renewal config file should have an
opposite. There's an old adage that comes to mind:
Any feature that can't be turned off is a bug. :-)
It would make the post-rollover creations of new keypairs/certificates
possible without having to resort to clumsy behind-the-scenes file
modifications. Ideally, they would look like this:
# Replace the newly next keypair/certificate (original)
certbot certonly --force-renewal --no-reuse-key --disable-renew-updates \
--cert-name example.org
# Replace the newly next keypair/certificate (duplicate)
certbot certonly --force-renewal --no-reuse-key --disable-renew-updates \
--cert-name example.org-duplicate
cheers,
raf
