We have an API server with app.example.com domain with Let's Encrypt SSL configured with Nginx, recently the IOS developer asked to apply SSL certificate pinning. I manage the server but I'm not familiar with this mechanism at all because we used to use only certbot certificates.
My first question is what are the best practices to pin certbot certificates? and what is the .pem file we should use in the client app?
The second and the more important question is how should we deal with certbot certificate renewal? I mean the .pem files will be replaced when renewing the certificate, will this affect the client app because the files changed? updating the app with the new .pem file will take time even when we do this automatically. is there a way to guarantee files sync without causing an outage?
I read this but I think I need more clear answers.
That depends a lot on just exactly what you're trying to pin, and why. Reasonable answers could include the Let's Encrypt Root X1, the current Let's Encrypt Intermediates, or your actual end-entity public key (which probably shouldn't be a "certificate" as in any of the pem files output from certbot exactly, but just the public key portion of the cert.pem). With any of those, though, you're going to want to have a plan for how to update the app when any of those change. Let's Encrypt intermediates change regularly, the Let's Encrypt Root changes rarely (but there's only a decade or so left in the current X1, so there will need to be another root eventually), and your end-entity key changes whenever you want it to (though by default certbot will make a new one for every certificate).
If you want to keep the same public key, you can pass the --reuse-key flag. That lets you get a new certificate but if you pinned to the same public key your app should still work. That does mean that your key is longer-lived, now, though, so you would need to plan for how to handle if that were were to get compromised and you needed to replace it on short notice.
And so you might want to pin to multiple possible public keys, so that you can have a "backup" ready for quick replacement.
But really this all comes down to why you're wanting to do pinning, that is: what threats are you trying to protect against by doing so, and what tradeoffs are you willing to make to deal with those threats? If your concern is that some bad actor might get Let's Encrypt to issue a certificate for your api server name, then pinning to Let's Encrypt's root won't really help you. If your concern is that Let's Encrypt might revoke your certificate due to a problem on their side before you can replace it, then maybe
having your TLS client base its security just on the public key it knows is good rather than on the certificate chain to a public CA makes some amount of sense. But for a lot of use cases, you're probably better off using your own private CA instead, or implementing CAA records, rather than trying to sort-of-kind-of manage your own PKI with pinning without really working through all the things involved in running your own private CA.
It's worth checking if this is the desired goal because when you decide you eventually want to use a new private key your app will stop working until it's updated . You will lose a percentage of your users who will just see it's broken and delete it.
In this scenario, you at least have to consider the update/transition mechanism when the server key changes, whether that's by pushing updates to the app itself (what fraction of users will get the update, and how quickly?) or by having some kind of agility/failover method (and how reliable is that method in various scenarios?).
A while back there was a post that a manufacturer of embedded remote monitoring systems had pinned the intermediate. When Let's Encrypt switched to R3 the systems were no longer able to update.
If I recall correctly the solution was to use the old certificate which had a few days left before it expired. If it had not been caught in time the only solution would be to manually update each device which would be an enormous task.
If you absolutely have to pin, pin the root. But keep in mind there could be a situation where the main root, or backup root is compromised, destroyed, or otherwise rendered unusable and have contingencies. Personally I would avoid pinning and just rely on the device or other trust store.
@ski192man It's probably much safer to use pinning in a mobile app that's distributed via an app store compared to an embedded device that may not receive software updates at all (or that may not be connected directly to the Internet frequently). But the developer should still make sure that updates can always be distributed in the future and that they will be fairly rapidly installed by almost all users!
In other words...
PINning access to the APP is OK, as long as one can access the store to receive an updated APP when that one doesn't work any longer.
If the APP and the store [holding the updated app] are both affected by the same PIN failure, then that is a "catch-22" and you won't be able to reach the update; Because you need the update to reach it - LOL
Do not pin intermediates or leaf certificates or keys.
Pinning a Let’s Encrypt root isn’t encouraged but it’s the least bad option if you insist on both pinning and using Let’s Encrypt. Make sure you have an upgrade path.