I am not exactly sure why I have multiple certs. I am just guessing that I first submitted for individual subdomains then I tried the expand option like
I assumed that going with the expand option would clear up the previous ones, I guess not
So my question is to, how do I get out of this and have only one cert for the whole domain space? I am even ok with individual certs per subdomain. I just do not know how to get myself out of this mess.
If you mean the real domain name is missing from my post, I did not wnt to reveal my domain name hence my user etc. Or did you mean another thing with missing domain name?
As far as the delete command, if I understand right, I need to revoke them first?
In anycase can I just delete them all and start from scratch for real? I do not wnt to debug and spend more time on this one by one.
your web server configuration is not currently referring to any of them
your situation won't run into the issuance rate limits (https://letsencrypt.org/docs/rate-limits/) (these are only related to historical issuance volume, and not related to whether the certificates are unrevoked or in use)
Nope, these are separate. Revocation is only necessary if you have reason to believe that a certificate is no longer accurate or that someone has improperly accessed the private key. It's not generally necessary merely because an accurate certificate is no longer used.
Certbot's behavior about this can be confusing. Certbot requires you to specify all names that a certificate should cover; if new names are added, all of the old names must be listed as well. The --expand option does not cause Certbot to try to find a related certificate, but simply avoids asking a yes/no question about whether to replace an existing certificate if the new list of names is a strict superset of an old certificate's list of names. (If even a single name from the old certificate is omitted from the list, Certbot will always create a separately named and tracked certificate, unless you force it to replace the existing certificate by specifying --cert-name.)
Basically, Certbot is much less proactive about checking for possibly-related existing certificates than many users would expect it to be, and also more willing to create partially-overlapping certificates without warning the user in many circumstances.
You could confirm this with a command like grep -r /etc/letsencrypt/live /etc/apache2 to see whether any of these files are referred to by your Apache configuration.
I realized from re-reading your original message that this wasn't the exact form of Certbot's behavior that was confusing to you. Instead, what happened is that when you listed all of the names, Certbot picked one of the existing overlapping certificates, essentially at random, and reissued that one with all of the names. There is currently no "merge" or "split" feature in Certbot. It seems like you expected that Certbot would notice that the newly-requested cert would have coverage of a strict superset of all pre-existing certificates and would therefore decide to merge them into a single certificate. Although I think we have other users who actively don't want that behavior, it would still be clearer if we made Certbot print a warning about related-but-unaffected or overlapping-but-unaffected certificates (maybe like "You have 2 existing certificates that now overlap your new certificate" or something).
Thank you so much for the amazing and in depth answer. Literally my understanding of LE process just stepped up multiple folds. Thank you!
Certbot picked one of the existing overlapping certificates, essentially at random, and reissued that one with all of the names
Yup that seems to be exactly what happened because I was also confused about the certificate names, since it picked up one of the subdomains for the certificate name which you can see in my first post.