Certbot fails with "Can't find virtual host" error

pjoc · April 21, 2023, 9:34pm

# netstat -pant | grep -i 'listen'

tcp        0      0 0.0.0.0:27036           0.0.0.0:*               LISTEN      4542/steam          
tcp        0      0 127.0.0.54:53           0.0.0.0:*               LISTEN      900/systemd-resolve 
tcp        0      0 127.0.0.1:38527         0.0.0.0:*               LISTEN      1774/Plex Plug-in [ 
tcp        0      0 0.0.0.0:631             0.0.0.0:*               LISTEN      1058/cupsd          
tcp        0      0 0.0.0.0:8200            0.0.0.0:*               LISTEN      1688/minidlnad      
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      1064/sshd: /usr/sbi 
tcp        0      0 127.0.0.1:32401         0.0.0.0:*               LISTEN      1704/Plex Media Ser 
tcp        0      0 127.0.0.1:57343         0.0.0.0:*               LISTEN      4542/steam          
tcp        0      0 127.0.0.1:32600         0.0.0.0:*               LISTEN      1819/Plex Tuner Ser 
tcp        0      0 0.0.0.0:1528            0.0.0.0:*               LISTEN      1817/Plex DLNA Serv 
tcp        0      0 127.0.0.1:8986          0.0.0.0:*               LISTEN      3446/qbittorrent    
tcp        0      0 192.168.122.1:53        0.0.0.0:*               LISTEN      1575/dnsmasq        
tcp        0      0 192.168.178.69:8986     0.0.0.0:*               LISTEN      3446/qbittorrent    
tcp        0      0 0.0.0.0:32469           0.0.0.0:*               LISTEN      1817/Plex DLNA Serv 
tcp        0      0 0.0.0.0:32400           0.0.0.0:*               LISTEN      1704/Plex Media Ser 
tcp        0      0 127.0.0.1:27060         0.0.0.0:*               LISTEN      4542/steam          
tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN      900/systemd-resolve 
tcp        0      0 0.0.0.0:5355            0.0.0.0:*               LISTEN      900/systemd-resolve 
tcp6       0      0 fe80::3482:d1d5:64:8986 :::*                    LISTEN      3446/qbittorrent    
tcp6       0      0 :::631                  :::*                    LISTEN      1058/cupsd          
tcp6       0      0 :::443                  :::*                    LISTEN      70618/httpd         
tcp6       0      0 :::80                   :::*                    LISTEN      70618/httpd         
tcp6       0      0 :::22                   :::*                    LISTEN      1064/sshd: /usr/sbi 
tcp6       0      0 :::1716                 :::*                    LISTEN      2543/kdeconnectd    
tcp6       0      0 ::1:8986                :::*                    LISTEN      3446/qbittorrent    
tcp6       0      0 :::5355                 :::*                    LISTEN      900/systemd-resolve

Also:

# systemctl status httpd

● httpd.service - The Apache HTTP Server
     Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; preset: disabled)
    Drop-In: /usr/lib/systemd/system/service.d
             └─10-timeout-abort.conf
             /usr/lib/systemd/system/httpd.service.d
             └─php-fpm.conf
     Active: active (running) since Fri 2023-04-21 22:15:17 BST; 19min ago
       Docs: man:httpd.service(8)
   Main PID: 70618 (httpd)
     Status: "Total requests: 22; Idle/Busy workers 100/0;Requests/sec: 0.0194; Bytes served/sec: 105 B/sec"
      Tasks: 231 (limit: 19019)
     Memory: 31.1M
        CPU: 2.079s
     CGroup: /system.slice/httpd.service
             ├─70618 /usr/sbin/httpd -DFOREGROUND
             ├─71650 /usr/sbin/httpd -DFOREGROUND
             ├─71651 /usr/sbin/httpd -DFOREGROUND
             ├─71652 /usr/sbin/httpd -DFOREGROUND
             ├─71653 /usr/sbin/httpd -DFOREGROUND
             ├─71672 /usr/sbin/httpd -DFOREGROUND
             └─71877 /usr/sbin/httpd -DFOREGROUND

Apr 21 22:15:17 Bree systemd[1]: Starting httpd.service - The Apache HTTP Server...
Apr 21 22:15:17 Bree httpd[70618]: Server configured, listening on: port 443, port 80
Apr 21 22:15:17 Bree systemd[1]: Started httpd.service - The Apache HTTP Server.
Apr 21 22:30:07 Bree systemd[1]: httpd.service: Sent signal SIGUSR1 to main process 70618 (httpd) on client request.
Apr 21 22:30:07 Bree httpd[70618]: Server configured, listening on: port 443, port 80  <-----****

rg305 · April 21, 2023, 9:59pm

Yes, it is listening on ports 80 & 443.

pjoc · April 21, 2023, 10:18pm

rg305:

I know. That's my point. If it's listening, why does Certbot fail? I'm slightly concerned about netstat showing it listening on tcp6 and not regular tcp, but that may be my lack of familiarity with netstat output.

rg305 · April 22, 2023, 2:12am

Showing tcp6 is "normal" / expected.
That only means it is listening on both IPv4 and IPv6.
OR better said that is can serve on both.

But if nothing has been configured to use port 80, then nothing will be served from it.

MikeMcQ · April 22, 2023, 2:22am

True. But, I think they are falling into a default Apache server which is why you saw results from HTTP earlier.

The problem is there is no explicit VirtualHost for this domain and port 80. @Osiris also noted this in post #16

We saw a VirtualHost for port 443 and this domain in post #8 but not for port 80.

pjoc · April 22, 2023, 10:14am

I get that. What I don't really understand is how Certbot can tell that a host is not virtual, or why it cares. If it can access port 80 with the URL, why does this matter?

And of course, how do I fix it? I followed the instructions referenced earlier and have double-checked them. I'm new to configuring Apache so a hint on where to look would be appreciated.

pjoc · April 22, 2023, 12:02pm

On reading the Apache docs, I see that there is a missing ServerName field in my VirtualHost config (i.e. the guide I referenced is wrong). However correcting this and restarting Apache makes no difference. This is the corrected config:

# cat bree.org.uk

<VirtualHost *:80>
    ServerName bree.org.uk:80
    ServerAdmin pocallaghan@gmail.com
    DocumentRoot /var/www/bree.org.uk/html
    ErrorLog /var/www/bree.org.uk/error.log
    CustomLog /var/www/bree.org.uk/log/access.log combined
</VirtualHost>

MikeMcQ · April 22, 2023, 12:50pm

Can you show the result of this now?

httpd -t -D DUMP_VHOSTS

As for this:

Certbot is the ACME Client and when using the --apache plug-in (like you are) it makes a temp change to the VirtualHost config. It then requests the cert and the Let's Encrypt ACME Server sends the HTTP Challenge request to your domain. If successful, Certbot retrieves the cert and makes any needed permanent changes to your Apache config VirtualHost(s).

pjoc · April 22, 2023, 12:53pm

MikeMcQ:

Can you show the result of this now?
httpd -t -D DUMP_VHOSTS
httpd -t -D DUMP_VHOSTS

VirtualHost configuration:
*:443 bree.org.uk (/etc/httpd/conf.d/ssl.conf:56)

As for this:

pjoc:

What I don't really understand is how Certbot can tell that a host is not virtual, or why it cares. If it can access port 80 with the URL, why does this matter?

Certbot is the ACME Client and when using the --apache plug-in (like you are) it makes a temp change to the VirtualHost config. It then requests the cert and the Let's Encrypt ACME Server sends the HTTP Challenge request to your domain. If successful, Certbot retrieves the cert and makes any needed permanent changes to your Apache config VirtualHost(s).

Thanks

MikeMcQ · April 22, 2023, 12:55pm

Why doesn't the VirtualHost you showed appear in the DUMP_VHOSTS output?

Is it in its own file? Is it in the /etc/httpd/conf.d folder? It is odd that it doesn't appear

pjoc · April 22, 2023, 1:10pm

cat /etc/httpd/conf.d/bree

<VirtualHost *:80>
    ServerName bree.org.uk:80
    ServerAdmin pocallaghan@gmail.com
    DocumentRoot /var/www/bree.org.uk/html
    ErrorLog /var/www/bree.org.uk/error.log
    CustomLog /var/www/bree.org.uk/log/access.log combined
</VirtualHost>

MikeMcQ · April 22, 2023, 1:20pm

Hmmm. That should be fine. One thing ... use 3 backticks before and after the Apache config output otherwise some tags may be lost. Like this:
```
apache virtualhost stuff
```

Can you add this to that port 80 VirtualHost as a test.
Restart apache and let us know when done

RewriteEngine on
RewriteRule ^/test.html$ /test2.html [R,L]

rg305 · April 22, 2023, 3:10pm

If the include statement is anything like:
include /etc/httpd/conf.d/*.conf
It won't pick that file up.

pjoc · April 22, 2023, 4:45pm

When I rename the file from 'bree' to 'bree.conf' I get:

apachectl restart

Job for httpd.service failed because the control process exited with error code.
See "systemctl status httpd.service" and "journalctl -xeu httpd.service" for details.

The journal doesn't give any more detail.

pjoc · April 22, 2023, 4:51pm

MikeMcQ:

Hmmm. That should be fine. One thing ... use 3 backticks before and after the Apache config output otherwise some tags may be lost. Like this:
apache virtualhost stuff
OK:
<VirtualHost *:80>
    ServerName bree.org.uk:80
    ServerAdmin [redacted]
    DocumentRoot /var/www/bree.org.uk/html
    ErrorLog /var/www/bree.org.uk/error.log
    CustomLog /var/www/bree.org.uk/log/access.log combined
</VirtualHost>
Can you add this to that port 80 VirtualHost as a test.
Restart apache and let us know when done
RewriteEngine on
RewriteRule ^/test.html$ /test2.html [R,L]

Not sure I understand this last bit. I don't have a 'test.html'. Which file is this referring to?

rg305 · April 22, 2023, 5:07pm

Change that line to:
ServerName bree.org.uk

MikeMcQ · April 22, 2023, 5:08pm

It's okay that you don't have a test.html file this is a technique we are going to use to make sure this virtual host is actually working

rg305 · April 22, 2023, 5:09pm

What shows?:
ls -l /var/www/bree.org.uk/

pjoc · April 22, 2023, 5:34pm

Sure. It was that way before today so I'll change it back.

pjoc · April 22, 2023, 5:36pm

# ls -l /var/www/bree.org.uk/
total 0
-rw-r--r--. 1 root   root    0 Apr 21 22:28 access.log
-rw-r--r--. 1 root   root    0 Apr 21 22:28 error.log
drwxr-xr-x. 1 apache apache 20 Apr 16 22:24 html
drwxr-xr-x. 1 root   root    0 Apr 16 22:21 log