Strange problem with Android

[h3ctic]

We have a problem where users can access IIS/API on inside of firewall with Apple phones but not Android using the same app. It also works on Chrome, Edge etc. So only Android is the problem.

Error given on phone (in the app) is:
Error occured
During the preparations, the following error occured:
The SSL connection could not be established, see
inner exeption.

The app uses only GET so it should be HTTPS only.

Problem started around when the root certificate expired.

Any ideas on how to proceed to figure out this problem would be greatly appreciated!!

(API is located @ [redacted])

[webprofusion]

Your service is currently using Apache (not IIS), I'm guessing you are proxying back to IIS. Your server chain is the android compatible one, so it should work fine for both iOS and Android.

If you skipped the Apache proxy you would indeed have problems with older Android devices becuase it's difficult to serve the android compatible chain on Windows (IIS) So.. It's impossible to support Android 7 and older on IIS? (In a non-hacky way?)

[h3ctic]

Thank you for your reply @webprofusion

Yes, I have a Sophos XG that is presenting the certificate to the clients and protects the webservers on the "inside".

I've been told that the software-devs have been testing with latest Android and you are saying that the server-chain in my certificate is all good(?) - I just now was told that they are also experiencing problems accessing the IIS/API with some Apple devices...which makes it even more strange...

I have no idea on how to proceed to figure out this problem

[webprofusion]

As your Sophos appliance is terminating TLS (handling the https communication with the clients) your actual IIS certificate is not relevant (it's not used by any of the clients, just the appliance).

The chain being served by the Sophos appliance is the 'android compatible' chain, so yes it should work for everything. Newer devices are supposed to know about the new 'ISRG Root X1' root certificate so they trust the certificate based on that, older devices don't trust (know) that certificate but your chain then leads them to the old DST Root CA X3 root certificate, which they do trust even though it has expired.

If you have broad and complex legacy client compatibility requirements I'd suggest testing out an alternative CA (such as ZeroSSL), they have the advantage of being cross signed by an old and as yet unexpired root certificate, so a lot of old devices trust them.

In each instance of reported incompatibility you need to determine the OS version, as you may be trying to support OS versions no longer supported by the manufacturer.

[webprofusion]

Also, if the reports of problems are new-ish devices but the reports a few weeks old you can likely ignore them, the recent Let's Encrypt root certificate expiry/changeover meant some devices needed reboots to clear cached certificate info.

[h3ctic]

Thanks again @webprofusion.

So, in conclusion; the firewall is serving the devices the correct certificate with correct chain and root certificates. The IIS is not communicating directly with the devices.

Then the problem must be either on the firewall itself or in the app they are using (?)...any way of figuring out this? Is there an Android app that can test the https-communication somehow?

Edit: The thing is that the devs of this app is saying that the cause of the problems are either the certificate and/or the firewall and Im not able to figure out a way to test this so that I with confidence can say the problem is because of this or that.....

[MikeMcQ]

@h3ctic I agree with what @webprofusion says. But, I also wonder is the app you talk about a custom one? Is it doing its own certificate validation? If so, they may not be handling the "long chain" that you are serving that ends with an expired certificate. They should see the ISRG Root X1 in the chain and see that it is trusted and stop there.

Here is some further info which may help resolve this:

curl -I (domain)

HTTP/1.1 200 OK
Date: Mon, 25 Oct 2021 12:12:50 GMT
Server: Apache
Cache-Control: private
Content-Length: 2703
Content-Type: text/html; charset=utf-8
X-AspNetMvc-Version: 5.2
X-AspNet-Version: 4.0.30319
Request-Context: appId=cid-v1:00a086d3-09b0-40ec-a46a-0b12c07f36f0
Access-Control-Expose-Headers: Request-Context
X-Powered-By: ASP.NET

And, use this site to see details of the certificate that Apache serves:
https://decoder.link/sslchecker

Show this info to the app developers and get a better description of the error they are having. They could try accessing this "home" page you showed in a browser and also with their app and see what the difference is.

[h3ctic]

Thank you @MikeMcQ ! I've sent this to the devs....Hopefully they can provide some more details.

Edit: Earlier in this discussion I wrote that I've just been told that also Apple-devices were having problems, but they dont, it was a false alarm.

Isnt that a bit strange? That all iphones work on this App, but Android does not?

[webprofusion]

With the Android devices, are they all new ones and/or are they part of a corporate network (or are they just consumer devices)?

[h3ctic]

Thanks @webprofusion

They are new consumer devices. The devs says theyve tested numerous Android devices, updated and rebooted; and they all fail.

[webprofusion]

Ok, let the devs figure it out then. New devices will have the ISRG Root X1 root installed, so those devices will validate the chain up to that cert then say "Hey, I trust ISRG Root X1!", then stop and everything will be great. Developers love a puzzle

[h3ctic]

The devs figured it out, I post it here for reference if someone else is using "Microsoft Xamarin framework". If not updated the frameworks SSL validator will not have the new Lets Encrypt root-certificate as a trusted SSL certificate. The devs here selected to change the validator to Android native instead and it is now working perfectly.

Thanks again guys @webprofusion and @MikeMcQ for helping and I would appreciate if a moderator could edit the post and remove the url:

"(API is located @ Home Page )" from my initial post (I am not able to find any edit-button for it any longer)

Ty to @MikeMcQ for redacting.

Thanks again!

[webprofusion]

Great, glad you got it working. I do remember something now about the Xamarin behavior being different, you'll probably find it mentioned in the forum already.