No. 
Well. There are thousands of websites and other services that know your IP addresses. And there are only 4 billion IPv4 addresses, so anyone can guess yours plus or minus 2 billion. 
It's up to you whether associating your device's public IP address with your certificates is considered acceptable. It's unlikely to be an issue for most people, especially if your IP changes regularly anyway, but it is a consideration if you're a high-value target of some sort.
At the moment, while Let's Encrypt asks for permission to share your IP address, they do not yet do it.