I have mounted in my lab, several servers with different applications configured with port 80, in front of these servers I have a reverse proxy (nginx) that performs the redirections to the different servers. Being several different sites I have configured in the sites-avalable and sites-enabled a configuration file for each of the servers. Everything works fine if I have all the servers with the certificate configured, but I have noticed that if there is one that I have not configured with HTTPS if I force the connection by typing in the browser bar instead of telling me that no certificate is found, it tells me that the certificate does not correspond to the domain I want to access and redirects me to the https that I have configured in another file, in the next in alphabetical order.
I think you mean it proxies to the different servers.
If so, then both HTTP and HTTPS are proxied.
Thus, when you try to reach a site that doesn't have a certificate, the proxy will match it to the default HTTPS vhost.
As stated, this is an unavoidable nginx [proxy] configuration issue.