This has been discussed several times before on the forum. It’s not so much for security of the Let’s Encrypt services as much as it is Let’s Encrypt not supporting this methodology. If they provided the IPs and they subsequently changed, then it would cause a large number of users who had a working configuration to start failing renewals. My understanding is that Let’s Encrypt wishes to reserve the ability to change these ad nauseum.
If you search around, other posts had sometimes creative alternative suggestions for how to handle this process in such an environment.