Let’s Encrypt’s position is that source IP addresses should not be whitelisted because Let’s Encrypt reserves the right to perform validations from undisclosed or completely unpredictable IP addresses (and even hopes to do so in the future) in order to reduce the effectiveness of attacks that involve tampering with network routing between particular portions of the Internet.
So, I would suggest whitelisting by the destination URL path /.well-known/acme-challenge/. It’s very unlikely that bots will gain anything from trying to scrape it because the only thing normally posted there is short-lived random files with unguessable paths (whose content is not very sensitive).