Upgrade of Ubuntu ca-certificates package breaks cert trust chain [SOLVED]

I did not see your use of port 143 until after I posted result of port 25. But, now with 143 I get below. Did you put the certs back?

openssl s_client -connect mail.fmp.com:143 -starttls imap
(parts omitted from info)
Certificate chain
 0 s:CN = *.fmp.com
   i:C = US, O = Let's Encrypt, CN = R3
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Nov  7 06:05:45 2022 GMT; NotAfter: Feb  5 06:05:44 2023 GMT
 1 s:C = US, O = Let's Encrypt, CN = R3
   i:C = US, O = Internet Security Research Group, CN = ISRG Root X1
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Sep  4 00:00:00 2020 GMT; NotAfter: Sep 15 16:00:00 2025 GMT
 2 s:C = US, O = Internet Security Research Group, CN = ISRG Root X1
   i:O = Digital Signature Trust Co., CN = DST Root CA X3
   a:PKEY: rsaEncryption, 4096 (bit); sigalg: RSA-SHA256
   v:NotBefore: Jan 20 19:14:03 2021 GMT; NotAfter: Sep 30 18:14:03 2024 GMT
3 Likes

Yes, I changed it back.

I have the broken ssl/certs file in place again, and will leave it there until 2:30 GMT.

1 Like

For openssl s_client -connect mail.fmp.com:143 -starttls imap I currently get the following, maybe you're still working on it..

CONNECTED(00000003)
40E7A617E67F0000:error:0A00010B:SSL routines:ssl3_get_record:wrong version number:../ssl/record/ssl3_record.c:354:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 534 bytes and written 340 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---
1 Like

No, at the moment I'm not working on it. I'll leave the broken ssl/certs in place until 2:35 GMT (8:35 CST), another 10 minutes or so.

1 Like

So I had a look with Wireshark, and your couriertls is throwing error messages:

NO STARTTLS failed: couriertls: /etc/ssl/certs/7c302982.0: No such file or directory
* NO Error in IMAP command received by server.
....5. NO Error in IMAP command received by server.
.. NO Error in IMAP command received by server.
* NO Error in IMAP command received by server.
* NO Error in IMAP command received by server.

So the problem appears to be with your couriertls expecting some of the files in /etc/ssl/certs that appear to be no longer there. I'm not experienced with courier, so no idea why that happens.

5 Likes

That's VERY good information. I'll run with it and look at the Courier config, or get on the Courier forum and chase it there. Thanks!!!

3 Likes

I can't find any portion of the Courier suite, especially the couriertls unit, which expects a specific file in ssl/certs. This is, oddly enough, another TrustCor certificate, which has been removed as per executive decision by Mozilla, but the mail server should never be asking for a specific file within the SSL subsystem.

2 Likes

You've restarted the services right? I can't see why it would refer to a file that doesn't exist unless it had cached the file paths in memory and now it was trying to load them. Impressively, googling your error returns this very post.

1 Like

Google is finding reference to something called "TLS_CACHEFILE" - ring any bells?

1 Like

That looks like a TrustCor certificate.

2 Likes

Yes, one of the 1st things I always do for mail server problems is check the process table and then kill and restart the entire Courier suite. On rare occasions, something in Courier will get hung up and a full restart of the suite will correct it. In this case, it didn't. It may be that the file access request on
/etc/ssl/certs/7c302982.0 is cached elsewhere. Courier MTA and friends has an active forum, of which I'm a member, so I'll pursue it there.

Google is quick to index tech forums. No surprise thiat this discussion is already in their index.

1 Like

Yep. That's what it is.

I'm not familiar with Wireshark, but I installed it. How did you use it to get the "No such file ..." return from mail.fmp.com:143

1 Like

Trustcor itself is removed from mozilla trust store

https://groups.google.com/a/mozilla.org/g/dev-security-policy/c/oxX69KFvsm4/m/yLohoVqtCgAJ

In line with our policies, Mozilla weighs the risks and benefits to end-user security when deciding whether a CA should be a member of our Root Program. Ordinarily, Mozilla would not directly evaluate the benefit of the CA owner’s other products when considering whether a CA should be a member of our Root Program. However, Trustcor’s quantifying value statement rests heavily on the value of MsgSafe which has suffered from a number of problematic behaviors [9] that undermine the value proposition of MsgSafe, and therefore undermine the purported benefits for the TrustCor CA to be a member of our Root Program.

Our assessment is that the concerns about TrustCor have been substantiated and the risks of TrustCor’s continued membership in Mozilla’s Root Program outweighs the benefits to end users.

In line with our earlier communication, we intend to take the following actions:

  1. Set “Distrust for TLS After Date” and “Distrust for S/MIME After Date” to November 30, 2022, for the 3 TrustCor root certificates (TrustCor RootCert CA-1, TrustCor ECA-1, TrustCor RootCert CA-2) that are currently included in Mozilla’s root store.
  2. Remove those root certificates from Mozilla’s root store after the existing end-entity TLS certificates have expired.
3 Likes

So it seems. This leaves a number of stale symlinks in /etc/ssl/certs. I'm deleting them to see if this clears the problem up.

1 Like

It looks like I fixed it!!

The couriertls program iterates through the files in /etc/ssl/certs. These are symlinks to files in /usr/share/ca-certificates/mozilla/. The package update process removed files in the latter folder, leaving broken symlinks in the former, and couriertls was barfing on the broken symlinks. I removed these manually and it looks as if all is well.

My sincere thanks to everyone who weighed in on this problem.

5 Likes

Lindsay Haisley, by golly. I guess I'll visit your website to see what you are up to. I suspect you are still as old as me, because that can't be fixed, but do you still live in Austin?

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.