Including "ISRG Root X1" with my IoT firmware before hand

Currently, we are developing IoT firmware which will be communicating to our backend over HTTPS or MQTT over SSL.

At our backend, we are using LE signed certificates (which are currently using root as "DST CA X3") which is well tested and works.

I am thinking of putting/adding in the "ISRG Root X1" but haven't tested it with our backend servers
(Although have tested connecting to isrg test web page via the firmware and works good :+1:.)

I feel I am blindly trusting that my certificates will be correctly signed using "ISRG Root X1" from Jan 2021 and everything will transition smoothly.

Is this a good idea :thinking:? or is there any way that I can get my LE certs using ISRG root so that I can test it now?

This is critical as once our firmware goes out in the future, and could not connect to our backend due to the SSL trust issue, the device cannot be recovered remotely and has to be physically called back.

4 Likes

This would suggest that your devices already trusts the "ISRG Root X1" root certificate.

Have you already checked your device's trust store? It may be the case that it's modern enough that you don't need to do anything.

One thing to be cautious about (in the long term) is if you currently are using an ECDSA certificate for your MQTT endpoints. Let's Encrypt will eventually be chaining these to "ISRG Root X2", which is not yet in trust stores (afaik). However, until that root is widely trusted (years away probably), there will be a cross-sign which chains to "ISRG Root X1" in the meantime.

4 Likes

No, It does not trust, hence I have added it manually.

Anyways, I guess it is safe to assume that if I can securely connect and communicate with the "ISRG test web page" any future certificates issued to me by LE's certbot should work too.

Thanks for the super quick response

3 Likes

While you're at it, why don't you also include the new ISRG Root X2 (ECDSA) certificate? It isn't in use at the moment, but it will be at some point.

1 Like

If you can't put a full "real" trust store in there, then you want at least ISRG Root X1 and ISRG Root X2 in there, and at least one other CA to use as a backup. ISRG Root X1 is only valid for another 15 years, and (while it's hopefully really unlikely), if Let's Encrypt becomes distrusted (due to a root key compromise or terrible bug in the automated validation or whatever), or if it runs out of funding and suddenly disappears, you're going to want to be able to switch to some other CA provider on short notice.

While I love Let's Encrypt and fully expect it to stick around indefinitely (just to be clear), I personally wouldn't want to entrust my business's well-being on one specific non-profit being around forever.

I think you could use the alternate link (like through certbot's --preferred-chain option) to download one chaining up to ISRG Root X1 now. Even if you can't get it through certbot, you could replace the intermediate being served by your API server, just download from the Certificates page the one under "Let’s Encrypt Authority X3" labeled "Signed by ISRG Root X1" and use that instead. (Though soon you'd need to get the one under "Let’s Encrypt R3" instead once they start using that one as the intermediate.)

4 Likes

This is a really tough situation to be in. I can only recommend that you use Let's Encrypt certificates (or certificates from any publicly trusted CA) with devices that get a regularly updated trust store. All publicly trusted CAs are subject to a variety of requirements, many of which are outside our control and yours. If those requirements change, we might need to change out our root certificates, or issue different kinds of certificates, or revoke all certificates of a given type.

My first recommendation is: Design and test a robust remote-update strategy. This is critical for your users' security.

If that's not possible, my second recommendation is: Generate your own root certificates, and configure your IoT devices to trust those roots. That way you are fully in control of your own fate, and cannot have your devices bricked by changes in the WebPKI. This may seem like an abstract worry, but the archives of https://groups.google.com/forum/#!forum/mozilla.dev.security.policy contain many stories of CAs that have had to revoke certificates or change issuance practices, and have broken large numbers of IoT devices.

Generating your own CA doesn't have to be hard. If your security requirements are low, you can do it in seconds with a tool like https://github.com/jsha/minica/. If you have somewhat higher requirements, you can make your own CA on a Yubikey (https://developers.yubico.com/PIV/Guides/Certificate_authority.html). Or if you don't want to fuss with managing a CA yourself, you can use Google's hosted private CA service: https://cloud.google.com/certificate-authority-service. Just remember if you go the "do it yourself" route, to keep the devices that hold your keys physically safe, and make sure you have robust backups.

5 Likes

Fictional evidence may not be as impressive as real-world evidence, but this is also a major plot point of Vernor Vinge's novel Rainbows End (really!). Maybe if the companies in that story had followed @jsha's advice here, Vinge would have had to find some other kind of infrastructure disruption for his plot!

1 Like

Regarding IoT devices and embedded trust stores (without updates), I yesterday read https://scotthelme.co.uk/impending-doom-root-ca-expiring-legacy-clients/ and it points out some more possible dangers. It's probably a good idea to include a variety of trust roots from various (trustworthy) CAs that won't expire for quite a time, so even if some go out of business or stop issuing for these roots, there are still backups from which you can get certificates that are accepted by your devices. Even including an own root certificate that you are not actively using and whose key is only stored offline and is never used until you eventually really need it since all other methods failed might be a good idea.

2 Likes