Zimbra unable to get local issuer cert

Help
1

I have the same problem in 4 servers. (from centos 6.5 to redhat 8) , all of them worked flawlessly with letsencrypt until this week, no changes were made on the servers, no updates, no software installation, no script modifications, etc.

To try to solve the problem I changed from X1 to X2 and back, but the problem persist.
same adding the X1 and X2 pem to the chain file, always the same error

ERROR: Unable to validate certificate chain: /etc/letsencrypt/live/mail.narf.com/cert.pem: CN = mail.narf.com
error 20 at 0 depth lookup:unable to get local issuer certificate

2

The chain file doesn’t need the Trusted Anchor Root Certificates.

Edit
I’ve been informed that Zimbra maybe different. :man_shrugging:

2 Likes
3

Hi @fromanm1,

The domain doesn’t seem to exist.

image
image1640×834 147 KB

2 Likes
4

that domain is fake, is just an example.

one of my domains affected with this is jhg.cl (the mail host es mail2.jhg.cl )

5

example.com is the proper domain name to use if you redacted the actual domain name.

As https://example.com/ shows the intended usages

Example Domain
This domain is for use in illustrative examples in documents. You may use this domain in literature without prior coordination or asking for permission.

Also see:

4 Likes
6

@fromanm1

I moved your posts to a new thread. Often problems that seem the same are very different. We have seen Zimbra often enough to be familiar with it.

Would you please describe more about your problem. Specifically the versions of Zimbra and the exact steps you used to create the cert that is failing validation.

3 Likes
7

This is the Chains of Trust - Let's Encrypt
It was “Last updated: Jun 11, 2024”

2 Likes
8

thank you very much my friend!

the procedure is the same in all servers.

  1. stop zimbra services
    su - zimbra -c "zmcontrol stop"

  2. renew the certs (it's a multi domain cert)
    /usr/bin/certbot certonly --standalone -n --agree-tos --register-unsafely-without-email --preferred-chain "ISRG Root X1" -d mail2.jhg.cl -d mail.paulamar.cl -d mail.centraldeoficinas.cl

this ends with "Successfully received certificate."

  1. download and add the ISRG Root X1 to the chain.pem
wget -O /tmp/ISRG-X2.pem https://letsencrypt.org/certs/isrgrootx1.pem
cat /tmp/ISRG-X2.pem > $DIRECTORY/chain.pem

  1. fix perms to allow zimbra to read the cert files

  2. validate the new certs with zimbra
    su - zimbra -c "/opt/zimbra/bin/zmcertmgr verifycrt comm $DIRECTORY/privkey.pem $DIRECTORY/cert.pem $DIRECTORY/chain.pem"

this step fails with the message

ERROR: Unable to validate certificate chain: CN = mail.appnexit.cl
error 20 at 0 depth lookup: unable to get local issuer certificate
error /etc/letsencrypt/live/mail.appnexit.cl/cert.pem: verification failed

this worked flawlessly until a few days, now it fails in several servers.

by example

CentOS release 6.10 (Final)

Release 8.8.10_GA_3039.RHEL6_64_20180928094617 RHEL6_64 FOSS edition, Patch  8.8.10_P8.

CentOS release 6.4 (Final)

Release 8.8.12_GA_3794.RHEL6_64_20190329045002 RHEL6_64 FOSS edition, Patch 8.8.12_P6.

CentOS Linux release 8
Release 8.8.15_GA_3953.RHEL8_64_20200629025823 RHEL8_64 FOSS edition, Patch 8.8.15_P44.
9

Looks like coping ISRG- 2.pem to chain.pem

See here for piping and redirection pipe - With regards to piping commands, what are the greater than (>) and less than (<) symbols called? - Super User

2 Likes
10

yes my friend. cat and redirect the output to the file is the same as copy the file.
I saved the file with other name (just drag from older script), but it's the isrg root X1 pem

wget -O /tmp/ISRG-X2.pem https://letsencrypt.org/certs/isrgrootx1.pem
cat /tmp/ISRG-X2.pem > $DIRECTORY/chain.pem

is the same as

wget -O /tmp/ISRG-X2.pem https://letsencrypt.org/certs/isrgrootx1.pem
cp /tmp/ISRG-X2.pem  $DIRECTORY/chain.pem

PS: I tried with all the pem files publicated on the chain of trust webpage you left me, same result.

11

Your cert should be connecting to ISRG Root X1

Why are you adding X2 rather than X1?

UPDATE:
You might miss this as I see you typing but as Bruce pointed out you are overlaying cert.pem from Let's Encrypt. You should be adding the ISRG Root X1 to it. You should use X1 because you have an RSA cert.

2 Likes
12

because it worked for ages only with the ISRG root x1 and an old chain (that chain was added later by the script).

now it's working.

this is the working script


#!/bin/sh

DOMAIN=example.com
DIRECTORY=/etc/letsencrypt/live/$DOMAIN

su - zimbra -c "zmcontrol stop"

/usr/bin/certbot certonly --standalone -n  --agree-tos --register-unsafely-without-email  --preferred-chain "ISRG Root X1"  --keep-until-expiring --key-type rsa -d $DOMAIN

wget -O /tmp/ISRG-X1.pem https://letsencrypt.org/certs/isrgrootx1.pem
cat /tmp/ISRG-X1.pem >> $DIRECTORY/chain.pem

su - zimbra -c "zmcontrol start"

chgrp -R zimbra $DIRECTORY /etc/letsencrypt/archive
chgrp -R zimbra /etc/letsencrypt/live/
chmod 750 /etc/letsencrypt/live/
find /etc/letsencrypt/archive -type d -exec chmod 750 {} \;  
find /etc/letsencrypt/archive -type f -exec chmod 640 {} \;  

su - zimbra -c "/opt/zimbra/bin/zmcertmgr verifycrt comm $DIRECTORY/privkey.pem $DIRECTORY/cert.pem $DIRECTORY/chain.pem"

cp /etc/letsencrypt/live/$DOMAIN/privkey.pem /opt/zimbra/ssl/zimbra/commercial/commercial.key

su - zimbra -c "/opt/zimbra/bin/zmcertmgr deploycrt comm /etc/letsencrypt/live/$DOMAIN/cert.pem /etc/letsencrypt/live/$DOMAIN/chain.pem"

su - zimbra -c "zmcontrol restart"
13

Just so you know with RSA certs there is no "preferred-chain" anymore. You can leave that option off.

Also, the script you said is now working has >> to append X1 as Bruce pointed out

Your previous example just one >

2 Likes
14

yep, as I told you, it just copied the ISRG because the old chain was added later in the script.

now it's working properly.

thanks guys.

1 Like
15

The chain.pem returned by Certbot is the chain for the cert.pem also returned. There is no "old" or "new" about it. I don't know what you mean by that. If you were wrongly using your own static copy of some chain file this will cause problems. You must use the chain as provided by Let's Encrypt. This has always been the proper way.

Further, Zimbra requires you to add the Let's Encrypt CA root cert to it.

Currently both RSA and ECDSA certs have a default chain that leads to root ISRG Root X1.

For only ECDSA certs you can use --preferred-chain to select ISRG Root X2. But, only recent Zimbra version supports ECDSA with other dependencies. See those docs.

See below topic for the details of LE chains

2 Likes
16

this old chain, same for all servers with all different domains, it worked like this for years until this week.

cat <<EOF >  $DIRECTORY/chain.pem
-----BEGIN CERTIFICATE-----
MIIFFjCCAv6gAwIBAgIRAJErCErPDBinU/bWLiWnX1owDQYJKoZIhvcNAQELBQAw
TzELMAkGA1UEBhMCVVMxKTAnBgNVBAoTIEludGVybmV0IFNlY3VyaXR5IFJlc2Vh
cmNoIEdyb3VwMRUwEwYDVQQDEwxJU1JHIFJvb3QgWDEwHhcNMjAwOTA0MDAwMDAw
WhcNMjUwOTE1MTYwMDAwWjAyMQswCQYDVQQGEwJVUzEWMBQGA1UEChMNTGV0J3Mg
RW5jcnlwdDELMAkGA1UEAxMCUjMwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEK
AoIBAQC7AhUozPaglNMPEuyNVZLD+ILxmaZ6QoinXSaqtSu5xUyxr45r+XXIo9cP
R5QUVTVXjJ6oojkZ9YI8QqlObvU7wy7bjcCwXPNZOOftz2nwWgsbvsCUJCWH+jdx
sxPnHKzhm+/b5DtFUkWWqcFTzjTIUu61ru2P3mBw4qVUq7ZtDpelQDRrK9O8Zutm
NHz6a4uPVymZ+DAXXbpyb/uBxa3Shlg9F8fnCbvxK/eG3MHacV3URuPMrSXBiLxg
Z3Vms/EY96Jc5lP/Ooi2R6X/ExjqmAl3P51T+c8B5fWmcBcUr2Ok/5mzk53cU6cG
/kiFHaFpriV1uxPMUgP17VGhi9sVAgMBAAGjggEIMIIBBDAOBgNVHQ8BAf8EBAMC
AYYwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMBMBIGA1UdEwEB/wQIMAYB
Af8CAQAwHQYDVR0OBBYEFBQusxe3WFbLrlAJQOYfr52LFMLGMB8GA1UdIwQYMBaA
FHm0WeZ7tuXkAXOACIjIGlj26ZtuMDIGCCsGAQUFBwEBBCYwJDAiBggrBgEFBQcw
AoYWaHR0cDovL3gxLmkubGVuY3Iub3JnLzAnBgNVHR8EIDAeMBygGqAYhhZodHRw
Oi8veDEuYy5sZW5jci5vcmcvMCIGA1UdIAQbMBkwCAYGZ4EMAQIBMA0GCysGAQQB
gt8TAQEBMA0GCSqGSIb3DQEBCwUAA4ICAQCFyk5HPqP3hUSFvNVneLKYY611TR6W
PTNlclQtgaDqw+34IL9fzLdwALduO/ZelN7kIJ+m74uyA+eitRY8kc607TkC53wl
ikfmZW4/RvTZ8M6UK+5UzhK8jCdLuMGYL6KvzXGRSgi3yLgjewQtCPkIVz6D2QQz
CkcheAmCJ8MqyJu5zlzyZMjAvnnAT45tRAxekrsu94sQ4egdRCnbWSDtY7kh+BIm
lJNXoB1lBMEKIq4QDUOXoRgffuDghje1WrG9ML+Hbisq/yFOGwXD9RiX8F6sw6W4
avAuvDszue5L3sz85K+EC4Y/wFVDNvZo4TYXao6Z0f+lQKc0t8DQYzk1OXVu8rp2
yJMC6alLbBfODALZvYH7n7do1AZls4I9d1P4jnkDrQoxB3UqQ9hVl3LEKQ73xF1O
yK5GhDDX8oVfGKF5u+decIsH4YaTw7mP3GFxJSqv3+0lUFJoi5Lc5da149p90Ids
hCExroL1+7mryIkXPeFM5TgO9r0rvZaBFOvV2z0gp35Z0+L4WPlbuEjN/lxPFin+
HlUjr8gRsI3qfJOQFy/9rKIJR0Y/8Omwt/8oTWgy1mdeHmmjk7j1nYsvC9JSQ6Zv
MldlTTKB3zhThV1+XWYp6rjd5JW1zbVWEkLNxE7GJThEUG3szgBVGP7pSWTUTsqX
nLRbwHOoq7hHwg==
-----END CERTIFICATE-----

-----BEGIN CERTIFICATE-----
MIIFazCCA1OgAwIBAgIRAIIQz7DSQONZRGPgu2OCiwAwDQYJKoZIhvcNAQELBQAw
TzELMAkGA1UEBhMCVVMxKTAnBgNVBAoTIEludGVybmV0IFNlY3VyaXR5IFJlc2Vh
cmNoIEdyb3VwMRUwEwYDVQQDEwxJU1JHIFJvb3QgWDEwHhcNMTUwNjA0MTEwNDM4
WhcNMzUwNjA0MTEwNDM4WjBPMQswCQYDVQQGEwJVUzEpMCcGA1UEChMgSW50ZXJu
ZXQgU2VjdXJpdHkgUmVzZWFyY2ggR3JvdXAxFTATBgNVBAMTDElTUkcgUm9vdCBY
MTCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBAK3oJHP0FDfzm54rVygc
h77ct984kIxuPOZXoHj3dcKi/vVqbvYATyjb3miGbESTtrFj/RQSa78f0uoxmyF+
0TM8ukj13Xnfs7j/EvEhmkvBioZxaUpmZmyPfjxwv60pIgbz5MDmgK7iS4+3mX6U
A5/TR5d8mUgjU+g4rk8Kb4Mu0UlXjIB0ttov0DiNewNwIRt18jA8+o+u3dpjq+sW
T8KOEUt+zwvo/7V3LvSye0rgTBIlDHCNAymg4VMk7BPZ7hm/ELNKjD+Jo2FR3qyH
B5T0Y3HsLuJvW5iB4YlcNHlsdu87kGJ55tukmi8mxdAQ4Q7e2RCOFvu396j3x+UC
B5iPNgiV5+I3lg02dZ77DnKxHZu8A/lJBdiB3QW0KtZB6awBdpUKD9jf1b0SHzUv
KBds0pjBqAlkd25HN7rOrFleaJ1/ctaJxQZBKT5ZPt0m9STJEadao0xAH0ahmbWn
OlFuhjuefXKnEgV4We0+UXgVCwOPjdAvBbI+e0ocS3MFEvzG6uBQE3xDk3SzynTn
jh8BCNAw1FtxNrQHusEwMFxIt4I7mKZ9YIqioymCzLq9gwQbooMDQaHWBfEbwrbw
qHyGO0aoSCqI3Haadr8faqU9GY/rOPNk3sgrDQoo//fb4hVC1CLQJ13hef4Y53CI
rU7m2Ys6xt0nUW7/vGT1M0NPAgMBAAGjQjBAMA4GA1UdDwEB/wQEAwIBBjAPBgNV
HRMBAf8EBTADAQH/MB0GA1UdDgQWBBR5tFnme7bl5AFzgAiIyBpY9umbbjANBgkq
hkiG9w0BAQsFAAOCAgEAVR9YqbyyqFDQDLHYGmkgJykIrGF1XIpu+ILlaS/V9lZL
ubhzEFnTIZd+50xx+7LSYK05qAvqFyFWhfFQDlnrzuBZ6brJFe+GnY+EgPbk6ZGQ
3BebYhtF8GaV0nxvwuo77x/Py9auJ/GpsMiu/X1+mvoiBOv/2X/qkSsisRcOj/KK
NFtY2PwByVS5uCbMiogziUwthDyC3+6WVwW6LLv3xLfHTjuCvjHIInNzktHCgKQ5
ORAzI4JMPJ+GslWYHb4phowim57iaztXOoJwTdwJx4nLCgdNbOhdjsnvzqvHu7Ur
TkXWStAmzOVyyghqpZXjFaH3pO3JLF+l+/+sKAIuvtd7u+Nxe5AW0wdeRlN8NwdC
jNPElpzVmbUq4JUagEiuTDkHzsxHpFKVK7q4+63SM1N95R1NbdWhscdCb+ZAJzVc
oyi3B43njTOQ5yOf+1CceWxG1bQVs5ZufpsMljq4Ui0/1lvh+wjChP4kqKOJ2qxq
4RgqsahDYVvTH9w7jXbyLeiNdd8XM2w9U/t7y0Ff/9yi0GE44Za4rF2LN9d11TPA
mRGunUHBcnWEvgJBQl9nJEiU0Zsnvgc/ubhPgXRR4Xq37Z0j4r7g1SgEEzwxA57d
emyPxgcYxn/eR44/KJ4EBs+lVDR3veyJm+kXQ99b21/+jh5Xos1AnX5iItreGCc=
-----END CERTIFICATE-----
EOF
17

Yeah, so you hard-coded a static chain file as that is not the one that came from Let's Encrypt for your most recent certs. You should never have been doing that. I explained that prior.

The current Zimbra wiki doesn't say to do that either so I don't know how you ever developed that as a solution. Their wiki says to use the chain.pem that comes with the cert.pem

3 Likes
18

how could I know, I didn't make that script, it's before my time, that's the script that worked for ages in those servers and it failed just now and I have to understand how and why it worked like that and solve the problem.

what's the matter? it's solved now, there is the answer and the script that works correctly, what else do you need on this thread?

it's everithing ok at home?

19

You don't have to be rude.

I was explaining the details so you could understand why it works. If that doesn't matter to you then let it go.

Other people searching in the future may find this thread. I want them to understand what they should do too.

4 Likes
20

Hi @fromanm1,

This looks like a problem

And it looks like you solved that issue here, by append output to the end of the file.

2 Likes