Your server is sending the wrong intermediate certificate. Earlier this year, Let's Encrypt switched from an intermediate certificate called "Let's Encrypt Authority X1" to "Let's Encrypt Authority X3". Your certificate was issued under this new intermediate certificate (X3), but your web server is still sending the old one (X1). The reason why it's only failing intermittently, and only with some browsers, is that they might cache the intermediate certificate when you visit other sites using Let's Encrypt.
IIS (or rather a Windows component that IIS uses) has a bug that causes difficulties when you swap out the intermediate certificate - it might just keep on using the old one even though you've replaced it. Take a look at this post for a solution:
You can use SSL Labs to verify the issue is fixed. The relevant error is "This server's certificate chain is incomplete. Grade capped to B." If you hit "Clear cache" on that site, and the error disappears, you've successfully resolved this issue.