Hello @hschroff, welcome to the Let's Encrypt community. 
What IP addresses does Let’s Encrypt use to validate my web server?
Let’s Encrypt does not publish a list of IP addresses we use to validate,
and these IP addresses may change at any time.
Let's Encrypt uses Multi-Perspective Validation Improves Domain Validation Security - Let's Encrypt
Edit
You can also change to using the DNS-01 challenge of the Challenge Types - Let's Encrypt and then only the DNS Name Servers' firewall are a potential issue (but commonly not an issue).