I discovered the issue after I tried the webroot method, which worked because it uses http. However when I configured apache to use the new certificate, it continued to serve http. Setting the virtualhost to _default_ made https work, but I knew that setting wasn’t necessary on the working server. I looked into reverse dns lookups for my host and noticed that some dns servers didn’t know the reverse ip.
That wasn’t the issue but it gave me the idea of looking in the /etc/hosts file. The entry for my.domain was incorrect. It turns out that when creating the virtual machine I had used the wrong ip address. I later went in and changed it manually in /etc/network/interfaces and /etc/xen/my.domain.cfg, but I forgot to change /etc/hosts.
Once I corrected /etc/hosts, the normal TLS-SNI-01 method worked.
That’s a really interesting issue. Would you be willing to post an issue at https://github.com/certbot/certbot/issues/new with instructions to reproduce? It would be great if we could detect this issue and fix it. It’s also a little surprising to me that the /etc/hosts issue should result in serving HTTP on port 443, so there might be some deeper issue we need to detect and fix.
In light of @sudoman’s update I just wanted to pass along that I don’t have any /etc/hosts mismatch like that. Though if it matters for some reason, I don’t have the domain name I was requesting the cert for in /etc/hosts at all (I only have localhost and the server’s primary name in /etc/hosts). Also, the name I am requesting the cert for does not reverse resolve to itself (though it does reverse resolve).
@sudoman Not sure what you’re trying to get at. This is a machine with a single IP address hosting multiple domains. It is therefore literally impossible for every domain to resolve-reverse-resolve to itself. One of the domains resolve-reverse-resolves to itself and all the others resolve-reverse-resolve to that one. (Anyhow, that’s getting off the topic of the overall discussion.)
