It produced this output: Unable to find a virtual host listening on port 80 .......
My web server is (include version): Apache 2.4.37
The operating system my web server runs on is (include version): Rocky Linux 8.6
My hosting provider, if applicable, is: (self-hosted)
I can login to a root shell on my machine : Yes
I'm using a control panel to manage my site : No
The version of my client is : certbot 1.22.0
I have a few things I'd like to deal with in this one help topic , before I get a new SSL Cert. I have gone through the documentation and read a few forum threads but gathering all the following info together is proving difficult:
My web server runs HTTP on a non-standard port so that the firewall can NAT that port to one of several Apache servers on the same subnet behind the firewall.
The httpd config does not use VirtualHost statements since it is serving only one domain name.
When running HTTPS, the web server will also be using a non-standard port.
The public IP address is dynamic, and is handled at dyn.com with an aliased domain name. The web server IP resolves using a CNAME record that points to the dyn.com alias.
The web site works fine using HTTP on a non-standard port.
What is the best way to handle these requirements so that certbot doesn't choke when trying to set up the initial and subsequent certificate requests?
HTTP based challenge validation will only ever come in on port 80 due to rules that Let's Encrypt must abide by as a publicly trusted certificate authority. If your firewall/NAT rules can't get that traffic to your web server (regardless of the port it's listening on internally), you'll have to use a different challenge type.
If the problem is that other web servers are already responsible for answering on the internet-facing port 80, a common solution is to add a reverse proxy in front of all the web servers that can route traffic to the correct back end based on host header info.
That explains what you are doing, but it doesn't actually explain why inbound HTTP is not being used.
What happens to ordinary inbound HTTP requests (on TCP port 80)?
Many thanks for your responses, much appreciated ... I added the Squid reverse proxy package to my pfSense firewall but have not figured out all the details of how to configure it.
To make things a bit clearer ... the Internet arrives at a pfSense firewall machine running on dedicated hardware. Its WAN has a dynamic public IP address assigned which gets aliased to other FQDNs by dyn.com using CNAME records in my DNS provider's zone file. (my cable modem is running in bridge mode so it doesn't get in the way) That's quite simple and works well.
Currently, port 80 is NAT'd to one of the servers behind the firewall, and this server has a couple of VirtualHost configurations in Apache with separate unrelated FQDNs. Both of those web sites do not use HTTPS (yet).
I want to bring up a web application on a different server using another unrelated FQDN, and obtain a LetsEncrypt SSL cert for that domain. That web server currently listens for HTTP connections on a non-standard port, and this is where Certbot complains that it can't find that server on port 80. Certbot refuses to accept the FQDN with :port# appended.
If I use Squid as a reverse proxy running on the pfSense firewall machine, I should be able to have it examine the FQDN of the incoming request and redirect it to the appropriate server while redirecting the other two domains to their server, everything on port 80.
Unfortunately I'm not familiar with Squid (yet) so I need to learn the intricacies of getting that configured. pfSense tries to make to easy but there are a lot of config fields to populate that are not intuitively obvious.
One question that I think I can answer ... do I need to remove the normal NAT rules in the firewall and allow Squid to do it instead? Of course I still need to allow port 80 for both of the servers in the firewall rules.
It's a new complication for me, but also a good learning experience. It is my first attempt at using LetsEncrypt.
Oh !!! You mean that Apache running on the existing server can be configured to toss port 80 requests (and port 443 as well) for the new FQDN over to a different machine on the subnet? I must look into that. Does that also mean that replies will get back to the requester via Apache or do those packets go directly back thru the firewall from the responding machine?
rmbolger and rg305 ... you two have been very helpful, with my thanks! I love learning new stuff
Got it!
Fortunately the original Apache machine has tons of spare horsepower and RAM to handle just about anything, so it shouldn't care about any additional workload. I assume Certbot won't even know or care this is going on. I will research how to use mod_proxy and let you know how it turns out ... perhaps in a couple of days.
With my gratitude, the Apache reverse proxy method works!! Certbot had no problem installing a cert, although I suspect I need to install it on the proxy-ing machine since that's where I configure SSL, in with the proxy statements. Certbot may continue to require port 80, but it's no big deal. I'm not sure if forcing HTTPS with Rewrite statements would interfere with Certbot's renewal process. Just give it a try
As an aside, when http validation get complicated there is also the alternative of using DNS validation - if your DNS is with a provider that supports automation this is sometimes (but definitely not always!) the easiest method to use.
It's interesting that on the proxy-ing machine Certbot added a second reverse proxy conf file for HTTPS, and it also added Rewrite statements to the original port 80 proxy config file to force HTTPS over to the target machine. That seems odd.
After installing the certs on the proxy-ing server, in which Certbot automatically found the reverse proxy and added its FQDN to the list of available targets, I requested certs for both the proxy'd machine and the proxy-ing machine while I was at it. That went without a hitch.
The reverse-proxied traffic from one machine to the other is all behind the firewall so there is no need to use HTTPS between them. I removed the Rewrite statements in the reverse-proxy config so no cert is needed on the target machine.
That's expected when using certbot with --apache and enabling redirection.
In order to only obtain a cert (and then manually use it as desired), one can use the certonly option.
Glad to hear that you are up and running securely now.
Cheers
@chuckm2000 Please do also consider how you will handle certificate renewals. I didn't understand your setup completely, but if there's a manual step involving copying files, you may want to think about automating that further with scripts. Remember that Let's Encrypt certificates are only valid for 90 days!
My current settings for reverse proxy are working well, and I removed the HTTPS rewrite statements in the SSL version of the proxy config so that traffic from one host to the other is not encrypted. Both machines are behind the same firewall so HTTPS between them is not needed. This means I don't need to manually copy a SSL cert to the target machine.
I will watch for the auto-renewal process of Certbot adding those rewrite statements back into the reverse proxy config. If that is the case, I could just have a simple shell script that uses 'sed' to remove them. I would use the post-install hooks to trigger the script.
There are a few other variations of this trick too :
Route all traffic to /.well-known/acme-challenge to a specific backend.
Route all traffic to /.well-known/acme-challenge to a specific port, run certbot on that port in "webserver" mode. Certbot can be run on any port on your machine/network, but LetsEncrypt can only reach out to port 80.
Use a rewrite rule to consolidate all traffic to /.well-known/acme-challenge to a specific (sub)domain. LetsEncrypt will follow that.
