Try check that none of these are blocked or throttled (assuming you only have IPv4 records for your domains):
18.196.17.13
18.216.110.187
18.219.177.57
3.120.126.223
54.202.29.69
66.133.109.36
34.222.229.130
52.15.254.228
64.78.149.164
I suspect though that building a comprehensive list of source IPs is going to be a bit difficult due to how Let's Encrypt are doing multi-VA.
If you haven't already read ACME v1/v2: Validating challenges from multiple network vantage points , it might help you.