I don't really disagree, and I can see the appeal. But the industry I think is moving more toward cryptographic identity, so if you have a DNSSEC-signed CAA record with the account identifier (which is tied to a public key), then you've got a pretty good assurance that it's the intended entity requesting the certificate. I mean, to some extent the whole reason we use TLS certificates in the first place is that IPs can be spoofed and BGP hijacked and so forth, so IP addresses in and of themselves may not be reliable.
So Certificate Transparency does tell you what certificates have been issued, and there are several tools to help one look through it. And if you're sure a certificate was issued by Let's Encrypt that shouldn't have been, you could revoke it (see the "Using a different authorized account" section).