hi @kyluke
Review the syntax for removing a certificate for a given domain
Certbot has it's own way of keeping track of domains and the fact you have removed a server block from your web server is not picked up.
List Known Certificates
Certbot certificates
Delete Certificate
Certbot --delete (name of certificate as above)
Andrei