Certificate renewal turned all RSA into EC certificates: Temporary hiccup or bug?

Yesterday, after a regular renewal of my subscriber certificates, Apache and Postfix suddenly stopped working. Either service offers both EC and RSA certificates for maximum compatibility with the public Internet and after yesterday's renewal the RSA certificates also became EC certificates such that Apache and Postfix suddenly had two certificates of the same type. As a result, the OpenSSL library was not able to handle incoming TLS requests.

I was able to revert to the previous state, but the question is whether this sudden conversion of RSA certificates to EC certificates was a temporary hiccup or does it point to a major change in the Letsencrypt client which I am unaware of? The incident happened yesterday around 20:02 CEST and last week I upgraded to Certbot 2.10.0.

Postfix configuration details

For example, my Postfix configuration looks like

smtpd_tls_chain_files =
  /etc/letsencrypt/live/server.my-domain.tld:smtps-ec/privkey.pem,
  /etc/letsencrypt/live/server.my-domain.tld:smtps-ec/fullchain.pem,
  /etc/letsencrypt/live/server.my-domain.tld:smtps-rsa/privkey.pem,
  /etc/letsencrypt/live/server.my-domain.tld:smtps-rsa/fullchain.pem

The EC certificate was originally requested with

certbot certonly \
  -a webroot -i None \
  --cert-name 'server.my-domain.tld:smtps-ec' \
  -w /var/www/my-domain.tld/server -d server.my-domain.tld \
  --key-type ecdsa --elliptic-curve secp384r1 \
  --deploy-hook 'systemctl reload postfix'

and the corresponding configuration file looks like

$ cat server.my-domain.tld\:smtps-ec.conf
[renewalparams]
account = XXXXXX
key_type = ecdsa
elliptic_curve = secp384r1
renew_hook = systemctl reload postfix
authenticator = webroot
webroot_path = /var/www/my-domain.tld/server,
server = https://acme-v02.api.letsencrypt.org/directory

The RSA certificate was originally requested with

certbot certonly \
  -a webroot -i None \
  --cert-name 'server.my-domain.tld:smtps-rsa' \
  -w /var/www/my-domain.tld/server -d server.my-domain.tld \
  --key-type rsa --rsa-key-size 2048 \
  --deploy-hook 'systemctl reload postfix'

and the corresponding configuration file looks like

$ cat server.my-domain.tld\:smtps-rsa.conf
[renewalparams]
account = XXXXXX
key_type = rsa
rsa_key_size = 2048
renew_hook = systemctl reload postfix
authenticator = webroot
webroot_path = /var/www/my-domain.tld/server,
server = https://acme-v02.api.letsencrypt.org/directory

The Apache configuration is similar, but I believe you get the gist. The problem with "double" EC certificates was that OpenSSL was not able to match the private key and the corresponding certificate based on key type and I got a lot of "TLS error: secret key does not match certificate ".

After all RSA certificates suddenly had turned into EC certificates, too, a second, forced renewal also created EC certificates only. The only solution was to revoke all certificates, clean out Certbot configuration directory and start from scratch. Now, everything is fine again.

But the nagging question remains whether I will run into the same issue again in 90 days.

@nagmat84 that is expected see Certbot 2.0.0 Release

1 Like

Certbot 2 should only be using ECDSA as defaults for new certificates, though, not for renewing existing ones.

5 Likes

There was a bug in older versions of Certbot 2 (fixed in v2.5.0) that sometimes wouldn't preserve the key type depending on the exact upgrade path and maybe some other circumstances. You say that you recently updated to 2.10, what is the sequence of versions before that, and where in that sequence were these certificates last renewed?

4 Likes

Depending on the versions you switched from and to, there was a bug that did not properly pick up on existing certificates using RSA keys and converted them to EC as you described. It bit me and was quickly fixed after I reported it here. If you haven't yet, you can reissue the certificates as RSA, and they will renew correctly.

3 Likes

Not really. I upgraded from 2.9.0. to 2.10.0 not 2.0.0. Also, only the default key type for new certificates changed with 2.0.0 from RSA to EC.

I upgraded from Certbot 2.9.0. Last time the certificates were renewed approx. 80 days ago with that Certbot version.

But it is good to know that this as been a known issue and it obviously bit me. But it also gives me the confidence that it won't happen again in approx. 90 days from now, after I re-created all certificates.

I did as written in my post.

4 Likes

The implication was read between the lines as there was a bug along the way to 2.10.
I guess this is one I should have belabored instead of being overly succinct. :man_shrugging:

2 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.