I think it's probably better to think about the situation as a Directed Graph (in the mathematical sense) in which the nodes are named public keys and the edges are certificates issued by one node. The goal of a verifier is to decide if there are a series of edges that lead from named public key they trust (e.g. ISRG Root X1) to the leaf certificate that a TLS server is showing them to prove its identity. In this understanding it is these named keys which are trusted at the root, not certificates for them.
A verifier that trusts ISRG Root X1 never needs to contemplate the "ISRG Root X1 signed by DST Root CA X3" certificate, because this represents an edge between ISRG Root X1 (trusted) and DST Root CA X3 (also trusted but soon to expire) so who cares?
It can stop once it sees just the "R3 signed by ISRG Root X1" certificate, it knows what ISRG Root X1 is, and it can verify using the public key it knows for that name that this certificate is valid, therefore R3 is trustworthy, therefore the leaf certificate is acceptable.
Unfortunately graph traversal problems are hard and so there is software that gets this wrong.
@tialaramex explains it well, though I would a caveat:
I would say "if there is a series of valid edges the lead from the the end entity certificate to any trust anchor (aka root certificate)." In other words, the path builder component of a verifier shouldn't be looking for a specific trust anchor, but for any one for which there's a path.
A couple of good but dense blog posts on the subject:
Your fullchain/chain includes R3, signed by ISRG Root X1 and ISRG Root X1, signed by DST Root CA X3. This is the current and future chain (for the foreseeable future) - no further changes.
I wonder, if between now and 2024 one could change to use a different, non-letsencrypt CA, from someone else that is both trusted by old Linux distros and old Android.
I fear of making a choice if I care more about "openssl s_client 1.0x" versus "android web browser 4.x..7.x". Cause I really don't know which has the largest impact (api usage, versus humans tap tap).
Is there a way to lookup what trust store CAs 4.x android shipped with?
The fundamental problem is that having trust store updates is really a fundamental part of having security updates, and having any concept of a "secure connection" with a system that doesn't get security updates is an oxymoron. One can work around some issues for some amount of time with tricks like the expired-root-signature that Let's Encrypt is employing, but really the problem isn't with the CA or with the server, it's with the client that doesn't really know anymore which roots are trustworthy. (That is, if an old long-lived root somehow got its key compromised, there's no way for systems that don't get trust store updates to know about it.)
ISRG Root X1 itself only has 14 years of validity left. A system that needs to be handle secured communications over a many-year timeframe without updates really needs something different than the WebPKI can give it.
In theory, sure. But most CAs aren't thrilled with signing roots for what are basically competitors, and there's a lot of cost involved (and a lot of risk, since signing someone else's root means you're taking responsibility for everything that other root does). At this point in time, Let's Encrypt really is ready to stand on its own feet, and I'm guessing the financial/auditing/etc. requirements are too onorous for starting anything beyond their existing relationship with IdenTrust, though if somebody wanted to pay for it maybe they'd figure it out? I'm just guessing, though.
You will find patch for OpenSSL 1.0.2g series as shipped in Ubuntu 16.04 LTS (xenial) and also a PPA built with this update for Xenial.
Testing things with it seems to make everything work. Instead of backporting all the features it really simply only sets the trusted-first flag by default. Please review these changes and let me know if anyone has any concerns about it. To me this makes openssl 1.0.2g be able to limp along with the new letsencrypt default chain, when the host otherwise trusts the ISRG Root X1 CA.
Although basic support for Ubuntu 16.04 LTS (xenial) has now ended, and only Extended Security Maintenance is offered (see Ubuntu Extended Security Maintenance | Security | Ubuntu), this issue is critical enough that openssl 1.0.2g update has now been published to xenial-security to address this compatibility issue.