Virtual Hosts Breaking Apache

Using Centos 7
Server version: Apache/2.4.6 (CentOS)

My website got taken down and apache can't start. I've had issues reading the logs due to permissions stuff that seems to have solved itself after reinstall (apache not being able to read its own log files) but, after reinstalling and reinstating the conf files (leaving lets encrypt and it certificates) I have this issue:

This website worked fine for a couple weeks. When trying to run certbot again after apache giving issues, or just running "apachectl configtest" this is what I see:

Invalid command '<VirtuallHost', perhaps misspelled or defined by a module not included in the server configuration

This appears to have been generated by letsencrypt.

<VirtualHost *:80>
    ServerName egletech.net
    DocumentRoot /var/www/html
    ServerAlias www.egletech.net
    ErrorLog /var/www/error.log
    CustomLog /var/www/requests.log combined
RewriteEngine on
RewriteCond %{SERVER_NAME} =egletech.net [OR]
RewriteCond %{SERVER_NAME} =www.egletech.net
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]

</VirtualHost>
Include /etc/httpd/conf/httpd-le-ssl.conf

The included conf:

<IfModule mod_ssl.c>
<VirtualHost *:443>
    ServerName egletech.net
    DocumentRoot /var/www/html
    ServerAlias www.egletech.net
    ErrorLog /var/www/error.log
    CustomLog /var/www/requests.log combined
SSLCertificateFile /etc/letsencrypt/live/www.egletech.net/cert.pem
SSLCertificateKeyFile /etc/letsencrypt/live/www.egletech.net/privkey.pem
Include /etc/letsencrypt/options-ssl-apache.conf
SSLCertificateChainFile /etc/letsencrypt/live/www.egletech.net/chain.pem
</VirtualHost>
</IfModule>

My domain is egletech.net,

Removing both virtualhosts seems to be the only way to get apache to start, so there seems to be an issue with it. It's interesting the site was live and had connectivity with the same config (i never changed anything) for the last two weeks.

The version of my client is certbot 1.11.0

For what it's worth, I used lets encrypt a couple years ago for a wordpress site and don't recall apache ever breaking or having to mess with all this. That site no longer is online though, nor is the server, to look at.

Thanks for the help!

1 Like

Please read the error message properly. It contains two "l"s (lowercase L) in the directive. This typo is NOT present in the configuration files you've shown.

Please check your Apache configuration for this misspelled directive and fix it.

Why removing perfectly fine parts of the configuration would fix this error originating from a typo elsewhere is a mistery to me.

4 Likes

I will add, that part of the configuration for lets encrypt is the only part of my conf files with virtual host stuff

Now after fixing the typos to be as it initially was (tried typing by hand, lol) I get

Syntax OK

Yet in both certbot and just restarting httpd from shell, apache still fails to start but starts successfully without the virtual host lines

1 Like

Please provide any error message or, preferably, logs from Apache when it fails.

4 Likes

Now HTTPD is running, 443 and 80 are unblocked on the os and on the hosting provider, and apache is listening on the two ports, and the website still won't load, lol

Did you add an A/AAAA record in your authoritative DNS system?

% curl -IL egletech.net
curl: (6) Could not resolve host: egletech.net
% dig a egletech.net

; <<>> DiG 9.18.0-2-Debian <<>> a egletech.net
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 58626
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;egletech.net.                  IN      A

;; AUTHORITY SECTION:
egletech.net.           3569    IN      SOA     dns1.registrar-servers.com. hostmaster.registrar-servers.com. 1606157503 43200 3600 604800 3601

;; Query time: 60 msec
;; SERVER: 172.25.128.1#53(172.25.128.1) (UDP)
;; WHEN: Sun Mar 13 13:41:25 CET 2022
;; MSG SIZE  rcvd: 114

Also:

% curl -IL www.egletech.net
HTTP/1.1 301 Moved Permanently
Date: Sun, 13 Mar 2022 12:43:10 GMT
Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.2k-fips
Location: https://www.egletech.net/
Content-Type: text/html; charset=iso-8859-1

curl: (7) Failed to connect to www.egletech.net port 443 after 250 ms: No route to host
3 Likes

I’ll have to look at the dns. Curl was giving me no route to host as well. If it was the dns though, why was it working perfectly fine until now?

1 Like

Well, I have no idea.

There is no A record on egletech.net

But there is one on www.egletech.net

And it looks like apache is listening on port 80 but there's some firewall on 443.

4 Likes

What are you concluding is a firewall?
HTTPS was working perfectly fine until a couple days ago. It says Apache is listening, port is open in firewalld, even with Selinux disabled it won’t load

It’s also enabled in the vps control panel

Because of this:

% curl -IL www.egletech.net
HTTP/1.1 301 Moved Permanently
Date: Sun, 13 Mar 2022 14:32:01 GMT
Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.2k-fips
Location: https://www.egletech.net/
Content-Type: text/html; charset=iso-8859-1

curl: (7) Failed to connect to www.egletech.net port 443 after 249 ms: No route to host

I can connect on port 80 but not on 443, and it's not a timeout. This usually means a firewall is actually telling me that I'm not supposed to connect, it's not acting to make me believe there is no server there.

3 Likes

What other firewall things are there? I’ve not had this issue before. 443 is open on iptables, disabled selinux for testing, Apache is listening on it, netstat confirms it’s open, and it’s enabled in the network rules on the VPS panel, and it worked fine until the Apache issues began.

1 Like

I just don't see it. Maybe your VPS provider has some other firewall somewhere?

% nmap www.egletech.net
Starting Nmap 7.92 ( https://nmap.org ) at 2022-03-13 15:49 CET
Nmap scan report for www.egletech.net (152.67.224.3)
Host is up (0.18s latency).
Not shown: 996 filtered tcp ports (no-response), 1 filtered tcp ports (host-unreach)
PORT    STATE  SERVICE
22/tcp  open   ssh
80/tcp  open   http
587/tcp closed submission

Nmap done: 1 IP address (1 host up) scanned in 35.60 seconds

NB: 1 filtered tcp ports (host-unreach)

3 Likes

If there was, I don’t know about it. It wouldn’t have randomly changed so as to disable connectivity either

When I get home I’ll send the results of the commands I ran that make me think the port is open, if that helps. If Apache and all says it’s listening, and it’s open in IPtables, and in the VPS, where else could it be messing up?

It’s interesting that nmap doesn’t show it open though

1 Like

I think 9peppe and osiris have pointed to clear problems. If you need more convincing here are two test sites we often use that cannot see your site with https either

https://decoder.link/sslchecker/www.egletech.net/443
https://www.ssllabs.com/ssltest/analyze.html?d=www.egletech.net&hideResults=on

I also think it is some kind of firewall but I am more bothered by why your Apache conf failed with a mis-spelled <VirtuallHost line which you could never locate. And somehow removing valid VirtualHost definitions allowed Apache to run. I know you said netstat confirmed it was listening but what does this say:

sudo apachectl -S
4 Likes

Is there any NAT going on?

3 Likes

The firewall explanation is the most logical, but I can't think of anything else, especially as to why this randomly happened.
So I turned off SELinux for testing and that gets apache to start, so I think something about SELinux doesn't agree with the VirtualHosts.

As for getting HTTPS to load, apparently reloading IPTables solved it, but I used --permanent both this time and when I initially set it up to add the rules to allow the ports in, so shouldn't that have made it stay after the server restarted and things of that nature? Further, trying to add the firewall rules again (before reloading) gave me the message that the rules were already added.

Thanks for your all's help! I'm not exactly the most skilled person in the world with apache & webservers, clearly :slight_smile:

2 Likes

Now it wants to show a test page even though the vhosts and apachectl -S shows the documentroot is /var/www/html. And the index.php is the proper one for the site

1 Like

Check this directive: mod_dir - Apache HTTP Server Version 2.4

Also: iptables doesn't save the rules by itself, it needs some other software to enable persistence. (And you probably have it already)

2 Likes