To continue this mess, the original server certificate is gone (meaning no backup) . My question now is:
Is there a way to use win-acme to generate a new lets encrypt certificate with a manually added intermediate certificate include in it. I am looking to add retired R3 to a newly generated certificate.
The newly generated certificates seem to include R10 or R11 by default.
This would hopefully allow the devices to connect temporarily without a trip to upgrade the firmware in the field.
No, the certificate would need to be cryptographically signed by the private key of R3 by Let's Encrypt, simply slapping on an intermediate certificate that did not actually sign the certificate doesn't work. But Let's Encrypt has already retired the R3 intermediate. Now, technically it is probably possible (the private key is still being used to sign OCSP responses I believe, so the private key does still exist), but Let's Encrypt does not have the possibility to manually sign certificates. Thus that would mean they'd need to revert their entire certificate issuance system, which signs millions of certs daily, for just your single certificate.
I hope you can appreciate how small the chance is that Let's Encrypt is willing to do that. (I'm not a Let's Encrypt staff member, so I cannot with certainty say that chance is zero. But I'm willing to place a bet it is just that.)
By the way, you don't need the certificate from the backup, as long as you have the corresponding private key still somewhere. Because certificates are publicly known in CT logs. If you also don't have the corresponding private key any longer, you're out of luck.
Thanks again, Osiris. I'll recheck with IT again about the backup and existence of the certificate and/or the private key.
Have a great day.
-Greg
@gmdriscoll you could perhaps try something like Recuva to get the old file back but it depends on disk activity etc. You would be looking in C:\ProgramData\win-acme\acme-v02.api.letsencrypt.org but if you also look at certlm ("Manage computer certificates") it may still have a copy in the local machine certificate store either under Personal or Web Hosting and you may be able to export that (as a PFX, with the private key) depending on your win-acme config.
I think this is the seconds report of an IoT device being hard coded to trust R3 instead of using the root (ISRG Root X1) as the trust anchor, they would need to manually update everything as well.
Keep in mind that intermediates can change overnight, one of the reasons they exist is because they are "sacrificial" in the event of a CA key compromise (which keeps the real CA root key well away from anything that's in daily use).
@gmdriscoll when you get a chance can you also review how you ended up taking this cert validation path - perhaps there is a common developer guide you followed etc that we need to reach out to in order to avoid this in the future. If everyone else did the same then there could be a lot of IoT devices across a range of industries affected.
On the surface, there is not a lot of info as to which certificate in the chain to use when developing an IOT device. Maybe you can expand the idea as to why this is?
So the path that was taken was to make sure that IT secured the server using TLS, which a developer won't have much control over. Then to get a working certificate, query the server using a browser, OpenSSL or other tool. What is returned, in general, is the intermediate certificate from the server. I believe this is standard practice as stated above.
The intermediate certificate generally expires in 3ish years, and the TLS connection to the webserver can be established.
In looking through the 3 browsers, Chrome, Edge, and Firefox, the default one that is shown is either the domain name cert or the intermediate cert. The root is there if you know what to look for. In the case of feeder-bot.com, all 3 certs work to validate the connection. The only real difference shown is when they expire.
Using openssl to query the server from a command line, the root is not returned.
Do best practices for IOT development and web certificates need to be explained better? Absolutely. I am not sure how you would disseminate that info.
This is the ESP IDF programming reference. No mention of certificate heirarchy to use or decision making process.
https://docs.espressif.com/projects/esp-idf/en/stable/esp32/api-reference
Thanks. Certificate trust generally works by trusting a known root certificate. How these get distributed to popular browsers and OSes is often referred to as a "root program" and is a list of roots decided by the OS/browser manufacturer. Servers do not normally actually serve the root because it's the client os (the device calling the server) who's job it is to know/manage roots that it trusts. If a server could decide which roots you should trust then any server could pretend to be any domain and get you to trust it.
Your desktop computer normally has such a list of root certificates it trusts (updating by your standard OS update process) and will in turn find a certificate path to one of these roots to decide if the certificate it sees is trusted, so even if it doesn't know to trusts the intermediate it sees as long as it does trust the root of that intermediate (the issuer of the intermediate) that's going to be ok.
The intermediate is a stunt issuer temporarily acting on behalf of the root and if you use that as effectively your "trusted" (known) root issuer that will temporarily work but it's not related to the expiry of the issuer so much as it is the inclusion of the issuer in your trust store.
Awesome.
The ESP IDF has the ability to add a certificate root bundle which is built from Mozilla's NSS root certificate store. This would take care of worrying about which certificates for the client to trust. Unfortunately, it is over 200k. For an IOT application, this is a lot of memory. By managing this manually, a single certificate is around 2k.
There are several free ACME CAs. See about deploying your own bundle of a few of these. Look at deploying ISRG Root X2 in addition to X1 if you plan for ECDSA support.
That's a great point, I would even suggest starting a little github project targeted at IoT with a minimal set of CA roots that just relate to public ACME CAs (Let's Encrypt, ZeroSSL, BuyPass, Google Trust etc) and let people use it and contribute to it. It could just be a folder of individual certs or a folder per category (acme, non-acme etc) and a script to concatenate them based on preferences.
If you only accept roots into that bundle that are already accepted by either the mozilla or chrome root programs (and evicts any that they evict) then it should be pretty safe - accepting stuff outside that would be problematic.
The benefit of making a project out of it is that you get more eyes on it and so you provide a scrutinized best practice for other IoT developers.
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.