Ios app uses retired certificate chain for several users

In mobile app, when I send api request, in correct case I see using of certificates

[Subject]CN=E5, O=Let's Encrypt, C=US
[Issuer]CN=ISRG Root X1, O=Internet Security Research Group, C=US

[Subject]CN=ISRG Root X1, O=Internet Security Research Group, C=US
[Issuer]CN=ISRG Root X1, O=Internet Security Research Group, C=US

But as per production logs, several users receive certificate chain error with
CN=E1, O=Let's Encrypt
CN=ISRG Root X2, O=Internet Security Research Group

As I can see from Chains of Trust - Let's Encrypt, E1 certificate is retired more than 3 monthes ago. So what can be the reason, that iphone still try to use this certificate from trust store? Does iphone don't load new certificates common trust store? Or it can be service issue and web service returns wrong certificate for app(but only in several cases)?

If your server is serving the wrong chain check that you are using the chain generated by your ACME client (usually called fullchain.pem or similar) and not concatenating your own chain together using an old intermediate.

4 Likes

Sounds like an incorrect chain indeed. Please provide the URL of the server which is giving you trouble.

2 Likes

Sorry, could you clarify? So server sometimes(for less than 0.1% of users) returns chain error. Chain generated by acme client should be the same for all. What is own chain - from server or from trusted store on iphone? Why used old intermediate chain?

Host that used Let's encrypt

That domain name is "proxied" at Cloudflare. This means the Cloudflare CDN edge provides the cert and chain for the user-agent (browser or other client).

Cloudflare may use various certs. Perhaps when people reach their CDN from certain parts of the world they get a different cert / chain than others.

This is probably a better question for the Cloudflare community

From my own test server I see below chain. Also use SSL Labs to check server chains.

subject=CN = auth0.com
issuer=C = US, O = Let's Encrypt, CN = E1
notBefore=May 28 15:46:01 2024 GMT
notAfter=Aug 26 15:46:00 2024 GMT

subject=C = US, O = Let's Encrypt, CN = E1
issuer=C = US, O = Internet Security Research Group, CN = ISRG Root X2
notBefore=Sep  4 00:00:00 2020 GMT
notAfter=Sep 15 16:00:00 2025 GMT

subject=C = US, O = Internet Security Research Group, CN = ISRG Root X2
issuer=C = US, O = Internet Security Research Group, CN = ISRG Root X1
notBefore=Sep  4 00:00:00 2020 GMT
notAfter=Sep 15 16:00:00 2025 GMT

subject=C = US, O = Internet Security Research Group, CN = ISRG Root X1
issuer=O = Digital Signature Trust Co., CN = DST Root CA X3
notBefore=Jan 20 19:14:03 2021 GMT
notAfter=Sep 30 18:14:03 2024 GMT
3 Likes

Your SERVER provides a CERTIFICATE that is signed by E5 INTERMEDIATE, which is signed by X2 ROOT (and compatible with X2 CROSS-SIGN, which is signed by X1 ROOT).

The E1 INTERMEDIATE was still used for signing until June 6 2024. There are still active certificates signed by E1 INTERMEDIATE, and that is expected to continue until September 4 2024 (90 days after June 6, the max certificate lifetime).

You did not share your domain name or any information about how you are leveraging Auth0. It is fairly impossible to guess what is going on based on the information you shared, because we have no idea what the mobile app is doing, what is is querying, and what end entity/leaf certificates it is using.

Considering you are having this issue with a small subset of users, my first guess would be there is a bug in your app, and you might have two API endpoints behind a load balancer, one is configured to serve an older certificate with E1 and the other with a newer E5. The error would happen when a second request is serviced by a different backend than the first request.

My second guess is that you might also have the following situation: Your webserver is sending a new Certificate signed by E5, but is still configured to send the chain from the previous certificate that specifies E1. Many web clients/libraries/browsers have a "short circuit" logic, and might ignore the E1 certificate because they previously validated a chain with an E5 certificate and still have that in their cache -- so when they try to validate the E1 certificate you indicated (that would fail), they: look up the SPKI, find the E5 certificate, use the E5 Certificate to build a path to the X2 trust store, and never have to look up the E1 certificate.

You should be debugging the certificates each connection is using - they will have a different serial number and expiration date. That should help you better pinpoint the issues.

So what can be the reason, that iphone still try to use this certificate from trust store? Does iphone don't load new certificates common trust store?

The iPhone does not have the intermediate certificates (E1, E5, etc) in the Trust Store. The iPhone only has the Root certificates (X1, X2) in the trust store. The server sends the iPhone an EndEntity/Leaf Certificate (for your domain) alongside a chain certificate (E1 or E5) to bridge the connection between your Leaf and the Root in the Trust Store.

Or it can be service issue and web service returns wrong certificate for app(but only in several cases)?

This is almost certainly the issue - either a bug in your app or the server.

There are a few other possibilities, but I think these are the most likely.

4 Likes

That domain - Auth0 - is a premium service from Okta. I doubt the OP is from Auth0/Okta, or the issue is caused by the Auth0/Okta platform.

3 Likes

I see your prior post. Good info.

It is annoying when they provide incorrect info.

3 Likes