Having trouble with SAN cert on iOS 9

I’ve been tasked with setting up a test Exchange environment for my company and thought that letsencrypt would be a great fit. I was able to use the wonderful ACMESharp .NET/PowerShell Module to get my SAN cert generated. Once I had the cert, I installed it in my test Exchange server and all seems to be going well. I tested it that OWA works with Firefox, Chrome and IE. However, when I pointed my iPhone running iOS 9.3.1, I get the dreaded “Cannot Verify Server Identity” message. SSL Labs and other online SSL/TLS testing tools says my cert is installed properly. I did notice that although I have the Let’s Encrypt Authority X3 Intermediate installed properly, the signing Root CA, DST Root CA X3 is not presented by the server. Does that indicate that there is something wrong with my installation? If so, what can I do to resolve this? Obviously, this system is running IIS, and is running on Server 2008 (non-R2 - long story on that…)

FWIW - I see that SSL Labs reports:
Apple ATS 9 / iOS 9 R
Protocol or cipher suite mismatch
RSA 2048 (SHA256) | TLS 1.0 | TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA | ECDH secp256r1

However, I get the same results on our production server, only that iOS doesn’t complain there… Production currently is protected by a commercially purchased SAN/UCC certificate.

Thanks in advance for any insight.

That's working as expected - the root certificate should be included on any device that trusts the CA, so servers actually should not do that at all.

A cipher suite mismatch would still be my best guess. Maybe there are some small differences in your IIS configuration. Would you mind sharing your domains so we could take a look at it and compare production with your test environment?

PFG - Thanks for the quick reply. Would you be able to give me a few things to verify between our two environments that would contribute to this problem?

To recap, the only noticeable differences regarding TLS between our Test & Prod environments I’ve found are:

  1. Prod has a commercial UCC/SAN Certificate purchased from a well-known CA.
  2. SSL Labs is reporting that prod is advertising 3 certificates in the chain. (Server, Intermediate, CA.)
  3. SSL Labs is reporting that test is advertising 2 certificates in the chain. (Server, Intermediate, & CA is listed as “in trust store”, not “Sent by server”.)
  4. SSL Labs is reporting that test supports OCSP stapling whereas Prod does not. (I believe this is a byproduct of the functionality supported by the certs securing the server and not the actual server configuration.)
  5. It appears that there has been no modification to the Cypher Suites in Production, and I have not modified this in Test either. (Looking into this, this would be accomplished through group policy - Group Policy -> Computer Configuration -> Policies -> Administrative Templates
    -> Network -> SSL Configuration Settings -> SSL Cipher Suite Order.)
  6. iOS 9.x works in Production whereas it does not trust the cert in Test. (The same cert that works when using current versions of IE, Firefox and Chrome.)

Anything else I can look at?

Thanks in advance.

You can use this script to compare the exact list of ciphers supported by each server. If there’s any difference, that’s probably where the problem lies. If that doesn’t yield anything, I’m pretty much out of ideas as well. :confused:

You may want to make sure you have the latest set of CA root certs, this is typically updated as part of OS upgrades, so maybe make sure your devices has the latest point release of whatever OS version it’s on.

@pfg Thanks for the link to the script. I ran it against both servers, and they had identical output…

@ebekker Hmmm - I'll look into that. The device is running the latest version of iOS, 9.3.1.

Taken from: Which browsers and operating systems support Let's Encrypt

Interestingly enough, Safari on that same iOS device seems to go through just fine when hitting the OWA site. However, because Safari doesn't let you view the certificates in use for that session, it's not really helpful. I assume that Safari would use the same root store as the Apple Mail app does, but I could be wrong on that.

Taken from: List of available trusted root certificates in iOS 9 - Apple Support

DST Root CA X3 is in the trusted store.

I verified the it had the same serial number as listed on the Apple support KB article.

I really think the issue is that the server is only presenting the client with:

  • Let's Encrypt Authority X3
    • {server certificate}
      Rather than:
  • DST Root CA X3
    • Let's Encrypt Authority X3
      • {Server certifiate}

However, the fact that the Let's Encrypt Authority X3 cert was signed by DST Root CA X3 which is supposedly in the device's local root store, shouldn't that make the intermediate LE cert inherently trusted?

That's how the chain works, yes. While it is possible to send root certs by the server, when this is in fact necessary, it would be a bug in the client IMHO. There's no sane reason to require the root cert being send by the server..

@Osiris - Thanks for confirming. That’s what I thought. After running one of the online SSL/TLS testing tools and it reported that the Root was ‘missing’ and flagged it red it got me headed down this path in thinking that perhaps it would be better if the server presented the full chain. I also saw @swisshuttles having a similar issue and his site was only presenting a portion of the chain as well. iOS Mail not trusted?

That SSL/TLS testing tool was:

You can always try ofcourse. :slight_smile:

Well, I've got a hard time trusting a site with lorem ipsum texts as their "?" icon help popups.. :stuck_out_tongue:

Just an update - I’ve also just tested an Android device running Lollipop (5.1.1) and it’s also complaining about the cert.

Has anyone had any luck with LE and Android/iOS mail clients trusting a LE cert?

Which protocol are you using to connect to your mail server? IMAP/POP3, or HTTPS (I suppose that would be MAPI over HTTPS)?

I’m asking because while you used SSL Labs to verify that the certificate chain is complete when your site is accessed via HTTPS, it’s not necessary the same if you’re using one of the other protocols. I’m not sure if that part needs to be configured separately in Exchange. Here’s a quick way to check the chain for any service (not just HTTPS on port 443), using Google’s IMAP server as an example:

openssl s_client -showcerts -connect imap.gmail.com:993

For a service using a certificate from Let’s Encrypt, you should see this line in the output:

1 s:/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
i:/O=Digital Signature Trust Co./CN=DST Root CA X3

To answer your question, yes, the root certificate used by Let’s Encrypt is definitely trusted by (non-ancient) Android and iOS, both for browser use and as an endpoint for apps.

@pfg - thanks for chiming in. As the system is using Exchange Outlook Anywhere (Exchange 2007) it’s HTTPS.

Yes, running that command against my server shows those exact same two lines within the output.

My understanding is while the clients should trust the Let’s Encrypt Authority X3 intermediate, that trust should be inherent as that cert was signed by the DST Root CA X3, which is definitely listed in Apple’s CA Root store (and most likely every other non-ancient keystore.)

I’d recommend using a packet sniffer like Wireshark to capture the TLS connection and look for the served certificates in the stream.
Otherwise I’m afraid people are giving good and well-meant advise, but nontheless we’re not really getting anywhere.

If the correct intermediate certificate is verified in the actuall TLS stream, there’s 100% no doubt anymore about the server configuration.

Again the server seems to work just fine when hitting it with IE/Firefox/Chrome. It’s just the mobile clients that are having issue with it. I would think that if the server configuration was an issue, I’d be seeing a problem with the browsers.

I know this has been suggested earlier, but it's sounding more and more like it's your ciphers.

After configuring my web server to use Let's Encrypt, everything was working except for a single mobile app (ownCloud) which was giving me an "SSL Initialisation failure" error. The error's wording made me think it was a certificate issue, and I spent ages trying various chain/fullchain/cert settings and even tried expanding the protocols the server offered (I originally restricted it to TLS1.2, but tried 1.1 and 1.0 without success).

Turned out to be my ciphers. I added a few more and the problem went away. The mobile app started working perfectly.

I know the symptoms don't make it feel like a cipher problem, and it really feels like a certificate problem, but it couldn't hurt to try expanding your cipher list! If it makes no difference, just go back to what you originally had.

@DarkSteve - Thanks for taking the time to respond to this and share your experience. I’ve been digging into this after trying to force the chain (unsuccessfully), even though I know I should not have to bother with this. I essentially have two systems that are configured identically in terms of server software and cipher suites. I’ve run multiple utilities against both environments and have found that both environments return identical results (with the exception of the obvious differences given that these are different hosts/domains, different CAs,etc.)

I’ve been doing packet captures. Everything seems to go just fine up until the client exchanges keys with the server, the client asks to change the cipher, there’s another Encrypted Handshake Message, and then the client seems to just give up sending a FIN/ACK. At this point, the iPhone is presented with an “Cannot Identify Server Identity” message. I will mention that the Client presents its suite of supported ciphers, which appears they agree to use TLS_RSA_WITH_AES_128_CBC_SHA, which I verified was in the list that the iOS device reported up to the server during the session…

In case you’re willing to do packet captures on the production server: Does it pick the same cipher in the “Server Hello” message?

Just to follow up and let everyone know, I figured out what was going wrong with my test Exchange server running the LE SAN certificate. It turns out that for my test external DNS domain, I had an A record for the root domain pointing to the Exchange server. This was actually different than our production environment. Because the Autodiscover process tries to connect to the root domain first, and my Exchange server responded with a correct response, the client tried to communicate using that session. Problem is that the certificate did not include the root domain FQDN in the SAN. After I changed the root A record, Apple Mail connected without any complaints.

Thanks to everyone who took the time and tried to help. :slight_smile: