You can do this with certbot and http challenge, assuming all servers run http servers.
You write a script that first mounts the webroots of the servers locally. In my case I uses sshfs
sshfs root@host1:/var/www/html/ /mnt/host1 -o sshfs_sync,IdentityFile=id_ed25519
sshfs root@host2:/var/www/html/ /mnt/host2 -o sshfs_sync,IdentityFile=id_ed25519
Now we can run certbot and have it do the http challenge, since it has write access to all webroots.
certbot --text --webroot --webroot-path /mnt/host1 -d host1.com --webroot-path /mnt/host2 -d host2.com
In this command there are multiple --webroot-paths, one before each hostname. certbot will write one token-file to each location.
and finally unmount, in my case
fusermount -u /mnt/host1
fusermount -u /mnt/host2
Now just distribute the certificate to the servers using scp or mounts, and restart the daemons.