LetsEncrypt + DANE

The most recent LE+DANE thread I could find is here.

But there’s also this more recent post which describes several certbot options I’ve never seen, which I suspicion are new.

I think it’s likely that certbot is under heavy development and may have some useful new options WRT DANE (and other functions). That’s the reason for this thread. {trawl, trawl}

You are indeed correct that you need certonly in order to do the renewal with a custom CSR.

As for auto renew scripts, as I posted in an earlier thread, you might try mine or any of several other examples online to see if they can be modified to suit your needs.

It’s complex. And clearly not for RHEL. It’ll take me a while to check every command for translation. There are so many things that can go wrong, and I have to make sure I’m aware of it if they do.

Why ECDSA? That was promulgated with input from one of those three-letter agencies.

It will become a larger and larger issue as DANE is more accepted, and clearly the LE devs did not intend for certs to go a year or more with no renewal. Hopefully tomorrow they’ll chime in.

Modifying the script for RHEL/CentOS variants should mostly be a matter of changing the paths to the certificates and keys. But of course, reviewing foreign code before using it is best practice.

As for why I use ECDSA, simply for the smaller, more efficient key sizes. I’d gladly exchange the NIST curves for Ed25519, but that isn’t supported in TLS or x509 standards yet, and probably won’t be available for new certificates without an update to the CA/B Forum Baseline requirements.

Also, to ensure you don’t have to update your DNS records every 90 days, you could just renew the certificate using the same private key. That’s basically what I do, and rotate the key annually. I feel that’s a decent security/convenience trade-off, but if you desire more frequent key changes, you might be able to automate the DNS updates as well, depending on who your provider is and if their API supports it.

So nothing. No input from devs?

This was the thread I was trying to point you to: Please avoid “3 0 1” and “3 0 2” DANE TLSA records with LE certificates. It seems like the missing piece for you is easy automated renewal without rotating keys. There’s a Certbot issue open for that.

1 Like

This is correct. To renew the certificate just return the same certbot command you ran with --csr in the first place. There is no need to regenerate the CSR every time, as Let’s Encrypt ignores the stuff in it that could possibly change anyway.

When people say certbot doesn’t support renewing certificates issued with --csr, they’re referring to the fact that certbot renew doesn’t go over all your old CSRs and renew them. You just have to renew them manually or put your full CSR command into a cron job instead of running certbot renew (or in addition to it, if you have certificates issued without --csr).

@schoen was working on a pull request for key reuse but it is not finished yet. At the current time, using --csr is the only way to achieve key reuse for HPKP and DANE.

RSA has been promulgated by more three letter agencies for longer than ECDSA so by this argument you should prefer ECDSA to RSA. :stuck_out_tongue_winking_eye:

In reality, they have conflicting goals in securing their own stuff and getting into ours. One of their goals in undermining standards processes is to trick you into using less secure stuff because you think the more secure stuff is less secure because they say it’s more secure. Don’t fall into their reverse psychology trap! Look at what other cryptography people say and base your judgements on that, not whether the government approves of it or not.

Hm, I’m using a systemd timer. (They’re encouraging us to move to that over cron)

So my letsencrypt.service would look like:

[Unit]
Description=Service for SSL cert renewals

[Service]
#EnvironmentFile=/etc/sysconfig/certbot
Type=oneshot
ExecStart=/usr/bin/certbot certonly --csr /etc/letsencrypt/csr-quantum-equities.com.csr -d quantum-equities.com
ExecStart=/usr/bin/certbot certonly --csr /etc/letsencrypt/csr-delphi-real-estate.com.csr -d delphi-real-estate.com --apache

# Security
PrivateTmp=yes
InaccessibleDirectories=/bin /boot /home /media /mnt /opt /root /sbin /sys
ReadOnlyDirectories=/lib /lib64 /usr
CapabilityBoundingSet=~CAP_SYS_PTRACE
DeviceAllow=/dev/null rw
NoNewPrivileges=yes

(I can get away with multiple ExecStarts since this is Type=oneshot)

Is this all I need, and not a script?

Is it correct that even with just -d {domain}.com, that since the csr includes also www.{domain}.com and mail.{domain}.com, that all three would be in the cert?

And would --apache operate to restart Apache afterward?

lol, fair enough.

In making my statement about ECDSA and RSA above, I am relying on the Word Of Schnier. I’ll stick with RSA for now.

You shouldn’t need to pass -d at all, as certbot should ignore it and use what’s in the CSR.

You need --apache in the first ExecStart line as well, or --webroot -w /var/www/html, or whichever authentication method you are using.

You also want to set WorkingDirectory to the directory you want your certificate to be stored in, since certbot saves the certificate in the current working directory. systemd would otherwise default to using the root user’s home directory, which you make inaccessible later in your unit file.

Also remember that this will issue a certificate every time, so the corresponding timer unit should be set to run every two months and you should follow up on any errors if it fails. It’s possible the scripts linked to upthread implement some retrying logic to reduce manual effort.

YIKES, lots of gotchas there.

So I’ll remove all the -d’s and will add --apache to each line.

Problem with setting WorkingDirectory is that it’s different for each domain. I don’t know that I can have multiple ones in one .service . A different .service for each domain? Hm, seems like certbot has --chain-path ${chain} --cert-path ${cert} options. Would those work per line? And would this be the /live/ directory or the /archive/?

I’ll change my .timer to:

[Unit]
Description=Timer for SSL cert renewals

[Timer]
OnCalendar=01-03,03-03,05-03,07-03,09-03,11-03
Persistent=true

[Install]
WantedBy=timers.target

There doesn’t seem to be a ‘bimonthly’ option. This will trigger at 00:00:00 which is fine with me, but if I need multiple .service files I’d better use RandomizedDelaySec=.

Yup, that’s what you want.

It looks a lot like the files in /live/, except they aren’t symlinks to /archive/. Your private key would be missing unless you manually place it there.

I wouldn’t recommend reusing /etc/letsencrypt/{live, archive}. Even though certbot should ignore directories that don’t have a corresponding renewal configuration file it’s possible this could still confuse some code paths in certbot. Instead just use a directory of your own creation.

You can abbreviate this slightly and this makes it easier to specify a time other than midnight if you want:

OnCalendar=*-01,03,05,09,11-03 00:00:00

RandomizedDelaySec is a great idea even if you have one unit because Let’s Encrypt’s infrastructure tends to be hit the hardest on the hour. Using it helps them out and slightly reduces the chance you’ll run into temporary outages or overcapacity issues.

1 Like

Nice, thanks. So now my .timer is:

[Unit]
Description = Timer for SSL cert renewals

[Timer]
# Realtime timer
OnCalendar = *-01,03,05,09,11-03 00:00:00
RandomizedDelaySec = 3600
Persistent = true

[Install]
WantedBy = timers.target

So I’d like to date the certs, and get an email when something goes wrong. This points to the need for a script.

And I think a sensible place to put the certs is /etc/pki/tls/certs, chains, and private.

Proposal (shamefully plagiarized from here):

DATE=$(date +%Y-%m-%d)

##
# certbot handling
#
# first it cannot replace certs, so ensure new locations (date suffix)
# each time mean the certificate is unique each time.  Next, it's
# really chatty, so the only way to tell if there was a failure is to
# check whether the certificates got updated and then get cron to
# email the log
##

# Quantum-Equities.com
DIR=/etc/pki/tls
DIR2=/etc/letsencrypt/live/quantum-equities.com
CERT="${dir}/certs/cert-${date}.pem"
FULLCHAIN="${dir}/chains/fullchain-${date}.pem"
CHAIN="${dir}/chains/chain-${date}.pem"
CSR=/etc/letsencrypt/csr-quantum-equities.com.csr
OUT=/tmp/certbot-QE.out

certbot certonly --webroot --csr ${CSR} --preferred-challenges http-01 -w /srv/www/QE --fullchain-path ${FULLCHAIN} --chain-path ${CHAIN} --cert-path ${CERT} > ${OUT} 2>&1

if [ ! -f ${FULLCHAIN} -o ! -f ${CHAIN} -o ! -f ${CERT} ]; then
    cat ${out}
    exit 1;
fi

# link into place

# cert only (apache needs)
ln -sf ${CERT} ${DIR2}/cert.pem
# cert with chain (stunnel needs)
ln -sf ${FULL} ${DIR2}/fullchain.pem
# chain only (apache needs)
ln -sf ${CHAIN} ${DIR}/chain.pem

# ... next domain ...

# reload the services
systemctl reload HTTPD
systemctl reload postfix
systemctl restart stunnel4

I don’t understand the need for these things – the docs aren’t clear to me:
–webroot
–preferred-challenges http-01
-w /srv/www/QE
} ${OUT} 2>&1

Unfortunately I can’t test it without disrupting the existing certs. Although, since they aren’t getting changed maybe it wouldn’t hurt?

I believe that I finally have my update script right, but I can’t test it because, “There were too many requests of a given type :: Error creating new cert :: too many certificates already issued for exact set of domains: mail.quantum-equities.com,quantum-equities.com,www.quantum-equities.com

I thought it was an hour wait, but joke’s on me.

!/bin/bash

DATE=$(date +%Y-%m-%d)

##
# certbot handling
#
# first it cannot replace certs, so ensure new locations (date suffix)
# each time mean the certificate is unique each time.  Next, it's
# really chatty, so the only way to tell if there was a failure is to
# check whether the certificates got updated and then get cron to
# email the log
##

# Quantum-Equities.com
DIR=/etc/pki/tls
DIR2=/etc/letsencrypt/live/quantum-equities.com
CERT="${DIR}/certs/quantum-equities.com_cert-${DATE}.pem"
FULLCHAIN="${DIR}/chains/quantum-equities.com_fullchain-${DATE}.pem"
CHAIN="${DIR}/chains/quantum-equities.com_chain-${DATE}.pem"
CSR=/etc/letsencrypt/csr-quantum-equities.com.csr
OUT=/tmp/certbot-QE.out

certbot certonly --webroot --csr ${CSR} --preferred-challenges http-01 -w /srv/www/QE --fullchain-path ${FULLCHAIN} --chain-path ${CHAIN} --cert-path ${CERT} > ${OUT} 2>&1

if [ ! -f ${FULLCHAIN} -o ! -f ${CHAIN} -o ! -f ${CERT} ]; then
    cat /tmp/certbot-QE.out | mail -s "TLS Cert Update Fail for quantum-equities.com" postmaster@quantum-equities.com
fi

# link into place

# cert only (apache needs)
ln -sf ${CERT} ${DIR2}/cert.pem
# cert with chain (stunnel needs)
ln -sf ${FULL} ${DIR2}/fullchain.pem
# chain only (apache needs)
ln -sf ${CHAIN} ${DIR}/chain.pem

How long does it take to reset? I’ve deleted my certs and chains so am dead right now.

In your service file, you use --apache, which tells certbot to use TLS-SNI-01 authentication with the running Apache webserver. The --webroot option tells certbot to use HTTP-01 authentication by placing a file in the directory /srv/www/QE which must be the one used to serve files for the domain. (The --preferred-challenges option is redundant and unnecessary.)

The --apache method is easier to use and doesn’t require your script to know where the public web files are kept for particular domains. If that’s what you were using before you can replace all three of these options with just this one.

> ${OUT} redirects the standard output stream to the filename stored in $OUT. 2>&1 redirects the standard error stream (file descriptor #2) to the standard output stream (file description #1), so ultimately the output of both streams is saved in the log file $OUT.

The wait is two weeks.

https://letsencrypt.org/docs/rate-limits/

You can work around the issue by adding an additional domain name to the certificate, or splitting off the mail subdomain to its own certificate. Any difference in the set of domain names will ensure it is not counted as a duplicate certificate.

Keep in mind there is also a 20 certificates per registered domain limit that isn’t so easy to work around. If you need to test things, pass --dry-run to certbot, which instructs it to use the staging servers Let’s Encrypt runs that issue untrusted test certificates.

2 Likes

Thank you T.C. Very clear and helpful responses to my questions.

I’ll revert to --apache and restore my cert and chains from backup, and hope my script works when the time comes.

Make sure to test your script before the time comes, also as @Patches mentioned always make use of the --dry-run flag

Here is my experience with DANE and Let’s Encrypt
"DANE technology — security thru DNS" - https://kostikov.co/tehnologiya-dane-bezopasnostj-cherez-dns
Article in Russian but I believe config language is the same everywhere :wink:

It seems I’ve pretty much got everything running. Have a possible SELinux problem but must wait for January to find out. My script works with --dry-run.

But I can’t receive SMTP email. Anyone who sends to me gets a “Connection timed out” to port 25. No wonder, I only have port 587 open.

Am I supposed to plumb 25 to 587 in the router? Would LE cert encryption be understood? Why don’t outside SMTP servers bother to try 587?

Port 587 is the submission port. It’s only for users with mailboxes on the server to send mail with. Port 25 is the port mail servers use to exchange mail. Mail servers don’t retry on port 587 because they are not supposed to.

By default, most mail server software restricts port 587 to authenticated outgoing submission as well so you may not even be able to forward port 25 to port 587 without changing your mail server configuration.

Your certificate will be recognized as valid for any port listening on the domain. Mail servers that enforce DANE will look up the TLSA record for TCP port 25 at the hostname(s) listed in your MX record.