If you're talking about outgoing connections to Let's Encrypt's API, you're probably best off allowing the name rather than any list of IPs, as their CDN may change them over time.
If you're talking about incoming connections from Let's Encrypt to validate that you control your requested domain names, then they intentionally check from many places around the world, which can regularly change, so that they can validate that you actually control the name as seen throughout the entire Internet. You may want to refer to this FAQ for more details: