PCI Scan failure

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: nwdw-kpl-fw01.nw-drywall.com

I ran this command: None - Script embedded in Sophos Firewall

It produced this output: Result is successful.

My web server is (include version): Sophos Firewall V21 (unknown web server but likely some Tomcat variant)

The operating system my web server runs on is (include version): Sophos Firewall Operating System (hardened Linux)

My hosting provider, if applicable, is: N/A

I can login to a root shell on my machine (yes or no, or I don't know): Yes, but applications are stripped.

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): No

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): this fails in the firewall's Linux shell.

Sophos Firewall v21 introduced automatic certificate renewals with Let's Encrypt. The functionality is hidden behind their WebUI.

Issue, the ASV, when performing a PCI scan is seeing 2 failures.
Certificate #0 CN=nwdw-kpl-fw01.nw-drywall.com ISSUER:_CN=E5,O=Let's_Encrypt,C=US self signed certificate in certificate chain
Certificate #2 CN=ISRG_Root_X2,O=Internet_Security_Research_Group,C=US is a self signed certificate

I see these certificates are part of the hierarchy. Are they indeed self signed or is there something missing in the chain? Might this be a Let's Encrypt issue, a Sophos issue (maybe missing a cert in the chain), or is the Authorized Scanning Vendor producing a false positive?

I can engage Sophos support if there are missing intermediate certificates in the chain, but in speaking with their initial line of support, I would need some information to back up the request otherwise they don't know what to do. If it's the ASV, what evidence might I supply to them that it's a false positive on their end?

Your system is sending out 3 certs. The first is the normal leaf. The second is its intermediate. That's all good and normal. But, you also send the ISRG Root X2 cert which should not be sent. This is the self-signed Let's Encrypt cert that resides in the client's trusted root store.

A client should ignore any such root when connecting to your server. But you also shouldn't send it. It's wasteful if nothing else.

So, your system is correct in identifying a problem. You should review / adjust whatever is sending that faulty chain.

3 Likes

Hi @MikeMcQ, and thank you for your quick response.

As this is Sophos Firewall doing this with it's built-in scripts, I will open a case with them. What information can I reference to help them understand what they are doing incorrectly and guide them to a fix?

Show them the output of this command and ask them why it happens

echo | openssl s_client -connect DOMAIN:PORT | head -20 

Substitute your domain and (alternate) port in above. They can run it or just show this:

Certificate chain
 0 s:CN = nwdw-kpl-fw01.nw-drywall.com
   i:C = US, O = Let's Encrypt, CN = E5
   a:PKEY: id-ecPublicKey, 384 (bit); sigalg: ecdsa-with-SHA384
   v:NotBefore: Jan 15 07:16:06 2025 GMT; NotAfter: Apr 15 07:16:05 2025 GMT
 1 s:C = US, O = Let's Encrypt, CN = E5
   i:C = US, O = Internet Security Research Group, CN = ISRG Root X2
   a:PKEY: id-ecPublicKey, 384 (bit); sigalg: ecdsa-with-SHA384
   v:NotBefore: Mar 13 00:00:00 2024 GMT; NotAfter: Mar 12 23:59:59 2027 GMT
 2 s:C = US, O = Internet Security Research Group, CN = ISRG Root X2
   i:C = US, O = Internet Security Research Group, CN = ISRG Root X2
   a:PKEY: id-ecPublicKey, 384 (bit); sigalg: ecdsa-with-SHA384
   v:NotBefore: Sep  4 00:00:00 2020 GMT; NotAfter: Sep 17 16:00:00 2040 GMT

3 Likes

Thank you so much! I appreciate your time.

2 Likes

I reviewed Chains of Trust - Let's Encrypt and I'm wondering what certificate Sophos should be using as the root. Do you have a reference I can review?

I ran the openssh query against the firewall I have of the same manufacturer with a LE cert and see the identical issue. Then I ran this query on another distro and don't see the verify error:num19:self signed certificate in certificate chain, however I do see that we are still using ISRG Root X2.

I did reach out to Sophos support and got immediate escalation to their GES team to find a work-around or a solution.

The server shouldn't send out any root. It should send out the leaf and the intermediate. Which are the first and second cert you currently send.

The client will already have, or should have, the root that the leaf and intermediate lead to. That's how the trust works.

Your server looks like a VPN. Is that part of the Sophos product or something else?

2 Likes

It's a VPN portal which needs to be enabled so the client software can get any updates needed and so users can login to setup MFA.

So the server (firewall) shouldn't be sending the third cert. Good to know.

Thank you!

3 Likes

Note that this is E5 signed by X2, not an usual config.

It should not be sending a self-signed ISRG Root X2. It may choose to send a cross-signed ISRG Root X2, which behaves like an intermediate if you want to rely on ISRG Root X1 instead.

(But you would usually see E5/6 signed by X1, skipping over X2)

Correct, that is the alternate chain for linking to the ECDSA X2 root. While it is not the default it is a valid alternate chain. We agree they should not be sending the self signed X2 root.

1 Like

This is getting to be quite the side note, but I think it's worth noting that this would require manual intervention of some sort (since it's not a chain that the ACME server would provide), and the cross-sign will expire later this year in September.

4 Likes

I really appreciate everyone's input to this. It helps me understand what's happening. I am just a server/network/router/firewall guy and sometimes have to update SSL certs. This is my first foray into Let's Encrypt.

3 Likes

Well, now I'm in a going back and forth with Sophos support.

"As per checking with GES, it's an expected output, we give back the root certificate, which shouldn't matter to any device that connect. Based on the RFC for TLS/SSL (RFC5246) the server MAY omit the CA certificate sent in the chain. Majority of web servers will send the full certificate chain including the CA certificate. Kindly please see this article RFC 5246: The Transport Layer Security (TLS) Protocol Version 1.2 which is from the RFC itself. On the SSLshopper shows it passes, please see link SSL Checker . You can also try against any website like google.com and it gives back the full chain as well. I hope this helps."

They also say the certificate chain is valid based on:

This site says it's misconfigured but offers no details.
https://whatsmychaincert.com/?ets-sain-fw01.entrets.com:4443

GoDaddy's test says there is an extraneous cert in the chain, but shows it's valid.
https://ssltools.godaddy.com/views/certChecker

From what I am understanding (or mis-understanding), the ISRG Root X2 shouldn't be provided as the clients should already have it.

What further information might I give to them?

  1. The root shouldn't be in the returned response, just the intermediate. I'm not sure what else you could tell them about that.
  2. But it shouldn't be hurting anything to include it. You might want to follow whatever process your "PCI Scan" has for a false positive, unless there's some specific security risk (that I can't concieve of) that it's trying to flag by complaining about it being included anyway.
3 Likes

Thanks for the quick reply.

Thank you for confirming that the root should not be returned in the response. Is this defined in a published standard?

Since we have hundreds of these firewalls that we manage and shifting over to Let's Encrypt from purchased certificates, we might have to come up with a way to submit false positives for each unit, every year when this comes up.

Well, they referenced RFC 5246; here's what it says:

The sender's certificate MUST come first in the list. Each following certificate MUST directly certify the one preceding it. Because certificate validation requires that root keys be distributed independently, the self-signed certificate that specifies the root certificate authority MAY be omitted from the chain, under the assumption that the remote end must already possess it in order to validate it in any case.

So that says the root "MAY" be there. So that might be evidence to your PCI scan that it shouldn't be a problem to include the root.

But that's for TLS 1.2. The newer version TLS 1.3, in RFC 8446 gives a bit more flexibility:

The sender's certificate MUST come in the first CertificateEntry in the list. Each following certificate SHOULD directly certify the one immediately preceding it. Because certificate validation requires that trust anchors be distributed independently, a certificate that specifies a trust anchor MAY be omitted from the chain, provided that supported peers are known to possess any omitted certificates.

Note: Prior to TLS 1.3, "certificate_list" ordering required each certificate to certify the one immediately preceding it; however, some implementations allowed some flexibility. Servers sometimes send both a current and deprecated intermediate for transitional purposes, and others are simply configured incorrectly, but these cases can nonetheless be validated properly. For maximum compatibility, all implementations SHOULD be prepared to handle potentially extraneous certificates and arbitrary orderings from any TLS version, with the exception of the end-entity certificate which MUST be first.

So in terms of the root, it also just says it MAY be included and doesn't seem to care whether it is or not. That all actually surprises me; I hadn't realized that the standard was so ambivalent about whether the root should be there or not. Maybe I shouldn't have been so emphatic about it. It's certainly not common in practice, but again it shouldn't really hurt anything one way or the other.

3 Likes

So in all reality, this should work just fine the way it is configured. I'll just have to work with the Authorized Scanning Vendors as we come across them. I doubt I'll be able to get Sophos to remove the root cert.

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.