Also, it's best to use certbot delete to remove certificates.
If you're trying to clean up from a partial manual deletion, you'll want to delete all of the associated files and directories in /etc/letsencrypt/renewal, /etc/letsencrypt/live, and /etc/letsencrypt/archive, like /etc/letsencrypt/renewal/example.com-0001.conf, /etc/letsencrypt/live/example.com-0001, and /etc/letsencrypt/archive/example.com-0001. Hopefully there are no cross-references between certificates with different names (which could somehow happen with older versions of Certbot when things were incompletely manually deleted, but which I don't think can happen nowadays).
As @rg305 mentioned, the -0001 certs happen when you have partially overlapping certificates but don't explicitly request one or more names from the old certificate, and also don't specify replacing the old certificate with --cert-name. Without --cert-name, Certbot is never willing to replace a certificate with a newer version that contains even one name fewer, so it instead creates the -0001 certificate in order to allow the old certificate to continue to exist.