Zimbra renewal - Problems with R3

My domain is: mymailserver.com (a mail server, not web-accessible)

still got the error:

-- Deploying certificate (zmcertmgr)
** Verifying 'cert.pem' against '/opt/zimbra/ssl/zimbra/commercial/commercial.key'
Certificate 'cert.pem' and private key '/opt/zimbra/ssl/zimbra/commercial/commercial.key' match.
** Verifying 'cert.pem' against 'chain.pem'
ERROR: Unable to validate certificate chain: cert.pem: C = US, O = Let's Encrypt, CN = R3
error 10 at 1 depth lookup:certificate has expired
OK

So I run:

openssl s_client -connect mymailserver:443 -servername mymailserver.com
CONNECTED(00000003)
depth=2 C = US, O = Internet Security Research Group, CN = ISRG Root X1
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = R3
verify return:1
depth=0 CN = mymailserver.com
verify return:1
---
Certificate chain
 0 s:/CN=mymailserver.com
   i:/C=US/O=Let's Encrypt/CN=R3
 1 s:/C=US/O=Let's Encrypt/CN=R3
   i:/C=US/O=Internet Security Research Group/CN=ISRG Root X1
 2 s:/C=US/O=Internet Security Research Group/CN=ISRG Root X1
   i:/C=US/O=Internet Security Research Group/CN=ISRG Root X1

My local certificate does not expire until December:

SSL_CERT OK - x509 certificate mymailserver.com from R3 valid until Dec 28 21:27:05 2021 GMT (expires in 89 days)

How do I fix this error below -- is there a way I can use certbot to 'remove' R3 from the chain? Or there is something else I need to do?

ERROR: Unable to validate certificate chain: cert.pem: C = US, O = Let's Encrypt, CN = R3
error 10 at 1 depth lookup:certificate has expired

The operating system my mailserver server runs on is (include version): Ubuntu 16.04 LTS, using snapd for certboot.

I can log in to a root shell on my machine (yes or no, or I don't know): Yes.

The version of my client is (e.g. output of certbot --version): certbot 1.19.0

That chain is not the preferred.
See the EXAMPLE:

---
Certificate chain
 0 s:/CN=community.letsencrypt.org
   i:/C=US/O=Let's Encrypt/CN=R3
 1 s:/C=US/O=Let's Encrypt/CN=R3
   i:/C=US/O=Internet Security Research Group/CN=ISRG Root X1
 2 s:/C=US/O=Internet Security Research Group/CN=ISRG Root X1
   i:/O=Digital Signature Trust Co./CN=DST Root CA X3
---

The R3 cert being used is the expired one.

1 Like

Yes, you are right, but how do I fix this? I followed the certbot/ZImbra directions I mentioned and this is what I got. How do Ι change my:

i:/C=US/O=Internet Security Research Group/CN=ISRG Root X1``

to:

i:/O=Digital Signature Trust Co./CN=DST Root CA X3

1 Like

Unfortunately, I can't reach the blog site to read the instructions you followed.
But, basically, you have to ensure the chain being used is the correct one.
There are also updates available if you are using OpenSSL (less than version 1.1).

1 Like

Thank you, the same directions also appear here if it helps: Installing a LetsEncrypt SSL Certificate - Zimbra :: Tech Center

I was hoping to be able to use certbot to fix the the chain (e.g. force --preferred-chain, etc). Is the answer to nuke the existing certificate and issue a brand new one?

My openssl is v.1.0.2g so I can update it although I don't quite understand how the chain issue would be related.
Thank you!

1 Like

No, it should already have the right fullchain.pem file.

That needs fixing!
Try:
sudo apt update
sudo apt install libgnutls-openssl27 libgnutls30
sudo apt install ca-certificates

2 Likes

That's never the correct way of handling things........

2 Likes

In the very first block of the instructions to which you've linked, they have you run this command:
cat /tmp/ISRG-X1.pem >> /etc/letsencrypt/live/barrydegraaff.tk/chain.pem

This chain.pem is where your R3 is coming from. It also should already have X1 within it, so the additional concatenate is weird (no, sorry, was thinking of fullchain.pem).

Either way, please check the contents of the new chain.pem - be sure you're not using an old one! Run these within your live/mymailserver.com directory and let us know what the output looks like:

  • ls -lFah chain.pem
  • openssl crl2pkcs7 -nocrl -certfile chain.pem | openssl pkcs7 -noout -print_certs
3 Likes

Thank you, this is what I see:

$ openssl crl2pkcs7 -nocrl -certfile chain.pem | openssl pkcs7 -noout -print_certs
subject=/C=US/O=Let's Encrypt/CN=R3
issuer=/C=US/O=Internet Security Research Group/CN=ISRG Root X1

1 Like

Have you updated/patched OpenSSL?

1 Like

great, now also please run:

  • ls -lFah chain.pem

AND

  • openssl x509 -noout -in chain.pem -subject -serial -dates -fingerprint
2 Likes
  • OpenSSL -- not yet, updating now.
  • The output:

$ ls -lFah chain.pem
lrwxrwxrwx 1 root root 45 Sep 29 19:21 chain.pem -> ../../archive/mymailserver.com/chain13.pem
$ openssl x509 -noout -in chain.pem -subject -serial -dates -fingerprint
subject= /C=US/O=Let's Encrypt/CN=R3
serial=912B084ACF0C18A753F6D62E25A75F5A
notBefore=Sep 4 00:00:00 2020 GMT
notAfter=Sep 15 16:00:00 2025 GMT
SHA1 Fingerprint=A0:53:37:5B:FE:84:E8:B7:48:78:2C:7C:EE:15:82:7A:6A:F5:A4:05

1 Like

This is great - this confirms that your chain.pem contains the still-valid version of R3 issued by X1.

That tells me that our API and certbot have provided you with a good cert and chain that can be used for TLS. It looks likely that Zimbra did not import the chain correctly. Hopefully you've opened some conversations with the Zimbra community?

Looking over the rest of the zmcertmgr instructions, the output line here:
** Appending ca chain '/etc/letsencrypt/live/barrydegraaff.tk/chain.pem' to '/opt/zimbra/ssl/zimbra/commercial/commercial.crt' looks like the stage where it actually imports the chain.

So, you can also verify the version of R3 in that commercial.crt chain file.

2 Likes

Thank you, it is encouraging that certbot provided a good cert+chain. Does this mean that once the chain issue is resolved I can just run certbot renew without any flags ( flags, e.g.--force-renewal --preferred-chain "ISRG Root X1")?

I will take up the chain and loading on Zimbra with the Zimbra folks then, thank you!

1 Like

:weary:

Please don't use --force-renewal. The idea mentioned by @ezekiel earlier was that certbot provided the correct chain already.

1 Like

Zimbra is crazy picky (unnecessarily).
The problem isn't in certbot, it is in the requirements Zimbra imposes on cert verification.
Which usually require it to include the root cert.
In this case, that would be two root certs.
The "broken step" is where Zimbra tries to validate the chain.
Let me hit my test servers with this problem and get back to you.

2 Likes

Thank you, there is a lot of Zimbra weirdness...:frowning:

1 Like

OK try this:

su - zimbra -c "/opt/zimbra/bin/zmcertmgr verifycrt comm privkey.pem cert.pem"

[add the path to files as needed]

1 Like

Also, please upload this file:
/opt/zimbra/ssl/zimbra/commercial/commercial_ca.crt

1 Like

Sure:

# su - zimbra -c "/opt/zimbra/bin/zmcertmgr verifycrt comm /etc/letsencrypt/live/mymailserver.com/privkey.pem /etc/letsencrypt/live/mymailserver.com/cert.pem"
** Verifying '/etc/letsencrypt/live/mymailserver.com/cert.pem' against '/etc/letsencrypt/live/mymailserver.com/privkey.pem'
Certificate '/etc/letsencrypt/live/mymailserver.com/cert.pem' and private key '/etc/letsencrypt/live/mymailserver.com/privkey.pem' match.
** Verifying '/etc/letsencrypt/live/mymailserver.com/cert.pem' against '/opt/zimbra/ssl/zimbra/commercial/commercial_ca.crt'
Valid certificate chain: /etc/letsencrypt/live/mymailserver.com/cert.pem: OK

And the certificate.crt is attached.Processing: commercial.crt...
commercial.txt (5.6 KB)

1 Like