Hmkay.. Well.. Maybe ask Slack too about their choice of chain.. And Palo Alto..
Besides that not much we can do for OP..
3 Likes
I'm not familiar with the OS for PaloAlto but does your firewall have the ISRG Root X1 (self signed) root in it's trust store (for the outgoing client request to slack)? Assuming it can build alternative trusted cert validation paths it may just be able use that path rather than the expired root path. Chain of Trust - Let's Encrypt
For instance Windows just ignores the path the server presents and builds it's own from leaf onwards, because Windows doesn't trust the expired root either.
3 Likes
Aye, the firewall DOES have X1 in its trusted CA store. And when I point the log forwarding system at a server with the SHORT chain LE Certificate, it returns valid. So the problem is inherent in how the Palo process validates the long chain / X3-referencing certificate.
I'm learning that Tenable.io vulnerability scanner is flagging long-chain LE cert as expired / invalid as well. Advice from Tenable to avoid the false-positive was to use the short-chain unless absolutely necessary.
1 Like
Both companies should learn more about how to validate chains.
6 Likes
The companies are right in some sense to think that there is a "problem", but in current practice I don't think there's any reason that that problem should be considered fatal.
3 Likes
Yep; and that "problem" is Android devices suppliers are not willing to supply security updates that would fix the problem.
2 Likes
Dropping support for devices that haven't applied security updates in 2+ years doesn't seem like a terrible idea.
2 Likes
But I would like Android devices suppliers to provide security updates, I am willing to apply the update if they provided them.
1 Like
The earliest version of Android with built-in support for ISRG Root X1 was released in Dec 2016 which was over 6 years ago at this point.
Regardless, the long chain issues will sort themselves when the extended DST cross-sign expires next year. From https://letsencrypt.org/2020/12/21/extending-android-compatibility.html
What happens when the new cross-sign expires? This new cross-sign will expire in early 2024. Prior to that, perhaps as early as June 2021, we will be making a similar change to what we intended to make this January. When we make that change, subscribers will have the option to continue using DST Root CA X3 by configuring their ACME client to specifically request it. If you are a Let’s Encrypt subscriber who is already working to mitigate the chain switch (e.g. by configuring alternate chains or encouraging your users on old Android devices to install Firefox), continue that work so you’ll be well-positioned in the future.
Seems like the timeline to switch away from the long chain by default has been a bit delayed though.
4 Likes
Curious if anyone can advise why openssl s_client -connect letsencrypt.org:443 is showing non-LetsEncrypt certificate? But in my browser I show LE R3 -> X1
For me, it doesn't?
2 Likes
About Let's Encrypt - Let's Encrypt read about Internet Security Research Group (ISRG)
And here lencr.org - Let's Encrypt
I see this
$ openssl version
OpenSSL 3.0.2 15 Mar 2022 (Library: OpenSSL 3.0.2 15 Mar 2022)
$ openssl s_client -connect letsencrypt.org:443
CONNECTED(00000003)
depth=2 C = US, O = Internet Security Research Group, CN = ISRG Root X1
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = R3
verify return:1
depth=0 CN = lencr.org
verify return:1
---
Certificate chain
0 s:CN = lencr.org
i:C = US, O = Let's Encrypt, CN = R3
a:PKEY: id-ecPublicKey, 256 (bit); sigalg: RSA-SHA256
v:NotBefore: Feb 2 00:00:24 2023 GMT; NotAfter: May 3 00:00:23 2023 GMT
1 s:C = US, O = Let's Encrypt, CN = R3
i:C = US, O = Internet Security Research Group, CN = ISRG Root X1
a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
v:NotBefore: Sep 4 00:00:00 2020 GMT; NotAfter: Sep 15 16:00:00 2025 GMT
2 s:C = US, O = Internet Security Research Group, CN = ISRG Root X1
i:O = Digital Signature Trust Co., CN = DST Root CA X3
a:PKEY: rsaEncryption, 4096 (bit); sigalg: RSA-SHA256
v:NotBefore: Jan 20 19:14:03 2021 GMT; NotAfter: Sep 30 18:14:03 2024 GMT
---
Server certificate
1 Like
Got any "security" devices/software intercepting traffic between you and the Internet? What do you actually see?
Also, it doesn't matter in this instance, but generally you should also include -servername <blah> in the openssl command to send a host header with the request. Some web servers will serve different certs based on that.
5 Likes
On up to date OpenSSL versions that isn't required: s_client will always send the server name from the hostname, unless overwritten by the -servername option.
6 Likes
TIL
Is there a way to explicitly not send a host header for the rare cases you actually want to test that?
3 Likes
Well, Host: is different from SNI (HTTP layer rather than TLS layer). In fact, sending different values in the two protocol layers has been used as part of an anti-censorship technique called domain fronting.
In newer OpenSSL, you can avoid sending SNI with the -noservername option:
-noservername
Suppresses sending of the SNI (Server Name Indication) extension in
the ClientHello message. Cannot be used in conjunction with the
-servername or <-dane_tlsa_domain> options.
8 Likes
As noted, your openssl version is probably old and not sending the server name (1.0.1k for example does not by default).
letsencrypt.org is served by a CDN. A CDN needs to know the server name (domain) to direct to the proper origin server. In this case, the CDN uses a default DigiCert when none is supplied.
5 Likes
What cert do you get?
3 Likes
If I use -noservername (like older versions of s_client do by default), I get one with
subject=C = US, ST = California, L = San Francisco, O = "Netlify, Inc", CN = *.netlify.app
(which is not too surprising to me for the CDN reason that @MikeMcQ alludes to above)
5 Likes