New server, used certbot plugin for wildcard, but site times out

donottaptheglass · February 9, 2021, 6:30am

Domain: wpmu.work
ip: 128.199.8.245

This is a fresh install of Ubuntu 20.04 with Apache installed. I used the instructions to obtain a wildcard from here: Certbot - Ubuntufocal Apache

The certbot plugin I used is digitalocean.

Here is the command I ran:
sudo certbot -a dns-digitalocean --dns-digitalocean -i apache --dns-digitalocean-credentials ~/.secrets/certbot/digitalocean.ini -d *.wpmu.work -d wpmu.work

Output:
Created an SSL vhost at /etc/apache2/sites-available/wpmu-le-ssl.conf
Enabled Apache socache_shmcb module
Enabled Apache ssl module
Deploying Certificate to VirtualHost /etc/apache2/sites-available/wpmu-le-ssl.conf
Deploying Certificate to VirtualHost /etc/apache2/sites-available/wpmu-le-ssl.conf
Redirecting vhost in /etc/apache2/sites-enabled/wpmu.conf to ssl vhost in /etc/apache2/sites-available/wpmu-le-ssl.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Congratulations! You have successfully enabled https://*.wpmu.work and
https://wpmu.work
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Seems successful... but when I visit https://wpmu.work or http://wpmu.work, it times out.

wpmu.conf

 <VirtualHost *:80>
    ServerAdmin webmaster@localhost
    ServerName wpmu.work
    ServerAlias www.wpmu.work
    DocumentRoot /var/www/wpmu
    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined

    RewriteEngine on
    RewriteCond %{SERVER_NAME} =wpmu.work [OR]
    RewriteCond %{SERVER_NAME} =www.wpmu.work
    RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>

wpmu-le-ssl-.conf

<IfModule mod_ssl.c>
<VirtualHost *:443>
    ServerAdmin webmaster@localhost
    ServerName wpmu.work
    ServerAlias www.wpmu.work
    DocumentRoot /var/www/wpmu
    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined

    Include /etc/letsencrypt/options-ssl-apache.conf
    SSLCertificateFile /etc/letsencrypt/live/wpmu.work/fullchain.pem
    SSLCertificateKeyFile /etc/letsencrypt/live/wpmu.work/privkey.pem
</VirtualHost>
</IfModule>

Can anyone spot what it is I am doing wrong?

Not sure if this is helpful, but there is this as well...

root@wpmu-work:/var/log/apache2# tail -n 5 error.log
[Tue Feb 09 05:56:40.955107 2021] [core:notice] [pid 16014:tid 140521637821504] AH00094: Command line: '/usr/sbin/apache2'
[Tue Feb 09 05:56:49.380984 2021] [mpm_event:notice] [pid 16014:tid 140521637821504] AH00493: SIGUSR1 received.  Doing graceful restart
AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1. Set the 'ServerName' directive globally to suppress this message
[Tue Feb 09 05:56:49.405766 2021] [mpm_event:notice] [pid 16014:tid 140521637821504] AH00489: Apache/2.4.41 (Ubuntu) OpenSSL/1.1.1f configured -- resuming normal operations
[Tue Feb 09 05:56:49.405786 2021] [core:notice] [pid 16014:tid 140521637821504] AH00094: Command line: '/usr/sbin/apache2'

rg305 · February 9, 2021, 7:32am

I see the posted configs:

But you don't show their full path, nor if they are actually being used by Apache, nor if there are any name overlaps/conflicts with other files (not shown).

So, let's start by having a look at what names Apache is using and from which files with:
apachectl -S

_az · February 9, 2021, 10:17am

It looks very much like TCP port 443 is closed on your DigitalOcean droplet. That port is required for HTTPS.

Make sure that you have that port opened on any firewall which is on your server (ufw allow https) and also in the DigitalOcean firewall in your control panel.

donottaptheglass · February 9, 2021, 6:36pm

/var/www/
html wpmu

/etc/apache2/sites-available/
000-default.conf default-ssl.conf wpmu-le-ssl.conf wpmu.conf

/etc/apache2/sites-enabled/
wpmu-le-ssl.conf wpmu.conf

apachectl -S
Output:
AH00526: Syntax error on line 18 of /etc/apache2/sites-enabled/wpmu-le-ssl.conf: SSLCertificateFile: file '/etc/letsencrypt/live/wpmu.work/fullchain.pem' does not exist or is empty Action '-S' failed. The Apache error log may have more information.

I checked the contents of /etc/letsencrypt/live/wpmu.work/fullchain.pem, it exists and has 2 cert... keys? chains? key chains (lol)? Not sure what to call it, but it does have contents inside.

rg305 · February 9, 2021, 6:50pm

Try it with sudo:
sudo apachectl -S

And please show:
ls -l /etc/apache2/sites-enabled/

donottaptheglass · February 9, 2021, 7:06pm

Instructions I used to setup Apache: How To Install the Apache Web Server on Ubuntu 20.04 | DigitalOcean

(I have inserted dashes below in an attempt to make this more human readable)
ufw status
Status: active
To -------------------Action----------From
OpenSSH---------ALLOW--------Anywhere
Apache-------------ALLOW-------Anywhere
OpenSSH (v6)---ALLOW--------Anywhere (v6)
Apache (v6)-------ALLOW-------Anywhere (v6)

sudo systemctl status apache2
● apache2.service - The Apache HTTP Server
Loaded: loaded (/lib/systemd/system/apache2.service; enabled; vendor preset: enabled)
Active: active (running) since Tue 2021-02-09 06:39:58 UTC; 12h ago
Docs: Apache HTTP Server Version 2.4 Documentation - Apache HTTP Server Version 2.4
Main PID: 34573 (apache2)
Tasks: 55 (limit: 1137)
Memory: 8.1M
CGroup: /system.slice/apache2.service
├─34573 /usr/sbin/apache2 -k start
├─34638 /usr/sbin/apache2 -k start
└─34639 /usr/sbin/apache2 -k start

Feb 09 06:39:57 wpmu-work systemd[1]: Starting The Apache HTTP Server...
Feb 09 06:39:57 wpmu-work apachectl[34568]: AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1. Set the 'ServerName' directive globally to suppress this message
Feb 09 06:39:58 wpmu-work systemd[1]: Started The Apache HTTP Server.
Feb 09 06:40:04 wpmu-work systemd[1]: Reloading The Apache HTTP Server.
Feb 09 06:40:04 wpmu-work apachectl[34636]: AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1. Set the 'ServerName' directive globally to suppress this message
Feb 09 06:40:04 wpmu-work systemd[1]: Reloaded The Apache HTTP Server.

I did not have any firewalls setup through the control panel on DO, so I added some (but still same outcome).

DO Firewall Setup:
(I have inserted dashes below in an attempt to make this more human readable)
INBOUND
SSH------TCP--22------All IPv4 All IPv6
HTTP----TCP--80------All IPv4 All IPv6
HTTPS--TCP--443----All IPv4 All IPv6
MySQL--TCP--3306--All IPv4 All IPv6

OUTBOUND
ICMP-----ICMP------------------All IPv4 All Pv6
All TCP---TCP-----All ports---All IPv4 All IPv6
All UDP---UDP----All ports---All IPv4 All IPv6z

donottaptheglass · February 9, 2021, 7:16pm

sudo apachectl -S
AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1. Set the 'ServerName' directive globally to suppress this message
VirtualHost configuration:
*:443 wpmu.work (/etc/apache2/sites-enabled/wpmu-le-ssl.conf:2)
*:80 wpmu.work (/etc/apache2/sites-enabled/wpmu.conf:1)
ServerRoot: "/etc/apache2"
Main DocumentRoot: "/var/www/html"
Main ErrorLog: "/var/log/apache2/error.log"
Mutex watchdog-callback: using_defaults
Mutex rewrite-map: using_defaults
Mutex ssl-stapling-refresh: using_defaults
Mutex ssl-stapling: using_defaults
Mutex ssl-cache: using_defaults
Mutex default: dir="/var/run/apache2/" mechanism=default
PidFile: "/var/run/apache2/apache2.pid"
Define: DUMP_VHOSTS
Define: DUMP_RUN_CFG
User: name="www-data" id=33
Group: name="www-data" id=33

ls -l /etc/apache2/sites-enabled/
total 0
lrwxrwxrwx 1 root root 35 Feb 9 05:11 wpmu-le-ssl.conf -> ../sites-available/wpmu-le-ssl.conf
lrwxrwxrwx 1 root root 28 Feb 9 04:42 wpmu.conf -> ../sites-available/wpmu.conf

ServerName is defined as wpmu.work (and ServerAlias www.wpmu.work) in both wpmu.conf and wpmu-le-ssl.conf Any thoughts on why Apache cannot determine the domain name?

rg305 · February 9, 2021, 7:17pm

OK please show:
curl -4 ifconfig.co

You can ignore that; it just means there is no ServerName defined anywhere in the main config - only in the vhosts [which is OK].

donottaptheglass · February 9, 2021, 7:17pm

Output:
128.199.8.245

rg305 · February 9, 2021, 7:19pm

HTTP works:

curl 128.199.8.245
*sobbing in silence*

HTTPS times out

Please show:
netstat -pant | grep -i listen | grep -i apache

donottaptheglass · February 9, 2021, 7:21pm

tcp6       0      0 :::443                  :::*                    LISTEN      34573/apache2
tcp6       0      0 :::80                   :::*                    LISTEN      34573/apache2

rg305 · February 9, 2021, 7:22pm

Is there any other FW or NAT type device inline?

Please show:
ifconfig | grep -Ei 'addr|inet'

donottaptheglass · February 9, 2021, 7:26pm

Not that I know of... or at least that does not sound familiar to me, so I don't think I have made any edits to anything mentioning FW or NAT. I did make some ufw changes, but not sure if that is relative or the same.

ifconfig | grep -Ei 'addr|inet'

    inet 128.199.8.245  netmask 255.255.240.0  broadcast 128.199.15.255
    inet6 fe80::b471:eeff:fe2a:4d43  prefixlen 64  scopeid 0x20<link>
    inet 10.124.0.11  netmask 255.255.240.0  broadcast 10.124.15.255
    inet6 fe80::8499:b3ff:fe44:9b12  prefixlen 64  scopeid 0x20<link>
    inet 127.0.0.1  netmask 255.0.0.0
    inet6 ::1  prefixlen 128  scopeid 0x10<host>

rg305 · February 9, 2021, 7:31pm

Name:    wpmu.work
Address: 128.199.8.245

Well... that's a match, so the problem has to be somewhere within your server.
[very unlikely that your ISP is blocking port 443]

As there is no need for NAT, we can rule that out.
So that only leaves firewalling.
Now, I see you opened up some ports, but there might be more than one firewall at play.

Do not disable any firewall; as your system is directly connected to the Internet.
We just need to see which are running and how they are configured.

rg305 · February 9, 2021, 7:32pm

Please show:
ufw status

donottaptheglass · February 9, 2021, 7:40pm

YES!!!!!!!!!

sudo ufw status verbose
Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), disabled (routed)
New profiles: skip

To Action From
22/tcp (OpenSSH) ALLOW IN Anywhere
80/tcp (Apache) ALLOW IN Anywhere
22/tcp (OpenSSH (v6)) ALLOW IN Anywhere (v6)
80/tcp (Apache (v6)) ALLOW IN Anywhere (v6)

sudo ufw allow https
Rule added
Rule added (v6)

sudo ufw status verbose
Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), disabled (routed)
New profiles: skip

To Action From
22/tcp (OpenSSH) ALLOW IN Anywhere
80/tcp (Apache) ALLOW IN Anywhere
443/tcp ALLOW IN Anywhere
22/tcp (OpenSSH (v6)) ALLOW IN Anywhere (v6)
80/tcp (Apache (v6)) ALLOW IN Anywhere (v6)
443/tcp (v6) ALLOW IN Anywhere (v6)

I checked the site in the browser and it is secure! Thank you for walking me through this @rg305 !!!

rg305 · February 9, 2021, 9:00pm

We WIN ! ! !

Cheers from Miami

[now back to trading crypto for beer...]

griffin · February 9, 2021, 7:49pm

That's some fine work!

system · March 11, 2021, 7:50pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.