Slack.com updated their certs yesterday (3/14/2023) and our PaloAlto firewalls immediately began rejecting them as invalid.
From what I can see in openssl s_client (see below), it looks like their X1 root is signed/issued by X3 (expired) instead of being self-signed. Is this cert chain invalid or am I misunderstanding?
** **% openssl s_client -connect hooks.slack.com:443** **CONNECTED(00000005)** **depth=2 C = US, O = Internet Security Research Group, CN = ISRG Root X1** **verify return:1** **depth=1 C = US, O = Let's Encrypt, CN = R3** **verify return:1** **depth=0 CN = slack.com** **verify return:1** **---** **Certificate chain** ** 0 s:/CN=slack.com** ** i:/C=US/O=Let's Encrypt/CN=R3** ** 1 s:/C=US/O=Let's Encrypt/CN=R3** ** i:/C=US/O=Internet Security Research Group/CN=ISRG Root X1** ** 2 s:/C=US/O=Internet Security Research Group/CN=ISRG Root X1** ** i:/O=Digital Signature Trust Co./CN=DST Root CA X3** **---** **
Since our problem is not in a decryption profile on the palo but rather in a Log Forwarding profile, I don't know that this change would apply to us, but I will look.
But is the fact that slack's chain seems to show X1 issued by X3 wrong? On other letsencypt cert servers, I see X1 signed by X1 (self-signed) which is what I expected would be the right chain.
Ok then on my firewalls, I have R3 and X1 (self-signed) loaded and trusted, but it does not validate. the Palo will not let me load the X1 (cross-signed X3) since it is expired.
Half the sites on the Internet probably use this "long chain" (including this community forum you're using now). Whatever issue you're having, it's either not related or you'd be having problems with a lot more than one site.
If you expand the "Certification Paths" section, you will see it shows two trusted paths.
One of the two [if not both] should work with Palo.
If neither work, then open a ticket with Palo about that.
In light of its being a Palo Alto firewall, maybe it should be "no-way-San-Jose"?
We knew in the past that some devices would have a problem with the long chain, but I think this is the first time someone has reported here having a Palo Alto firewall reject all of them. Could you try asking about it on the Palo Alto forum and see what that community has to say about it?
Yessir, I've posted the question in the PAN forum and have a support case open with them.
Their forum appears to be... less active than this one, to say the least. I appreciate the quick & accurate responses from everyone here. Really helping me to understand more about the situation, and hopefully will help me to teach Palo what they're doing wrong.
That solution is for the SSL decryption system (where you MITM attack your users so you can inspect their encrypted traffic). It doesn't apply to the Log Forwarding service which is the firewall itself just sending traffic/threat/system logs to a webhook.