Let's Encrypt does not publish a list of IP addresses used by its validation servers (currently 5 different centers world-wide with rotating IP).
You only need to allow URI of format /.well-known/acme-challenge/(token) on port 80. Any other inbound requests can be blocked or, perhaps better, redirected to 443.
You don't need to open anything different inbound on port 443 to satisfy an HTTP challenge unless you redirect the original HTTP request to port 443. That is under your control at your server.
Alternatively, there is a DNS Challenge which does not require any HTTP request inbound but has its own security concerns given you use a security token to update DNS records (adding and deleting TXT records).
This explains the multi-perspective validation in detail: Multi-Perspective Validation & Geoblocking FAQ