SSL Certificate Renewal Failure

Help
1

Hi all,

Having issues with SSL certificate renewal on a Unifi Controller that I inherited and don't know much about web hosting, apparently the certificate previously auto-renewed successfully but doesn't anymore although nothing has changed to my knowledge. I honestly have no idea where to start with this, any help is much appreciated.

My domain is: pippostech.com.au, website is unifi.pippostech.com.au

I ran this command: sudo certbot renew --dry-run

It produced this output: Cert is due for renewal, auto-renewing...
Plugins selected: Authenticator apache, Installer apache
Simulating renewal of an existing certificate for unifi.pippostech.com.au
Performing the following challenges:
http-01 challenge for unifi.pippostech.com.au
Waiting for verification...
Challenge failed for domain unifi.pippostech.com.au
http-01 challenge for unifi.pippostech.com.au
Cleaning up challenges
Attempting to renew cert (unifi.pippostech.com.au) from /etc/letsencrypt/renewal/unifi.pippostech.com.au.conf produced an unexpected error: Some challenges have failed.. Skipping.
All renewal attempts failed. The following certs could not be renewed:
/etc/letsencrypt/live/unifi.pippostech.com.au/fullchain.pem (failure)

** DRY RUN: simulating 'certbot renew' close to cert expiry
** (The test certificates below have not been saved.)

All renewal attempts failed. The following certs could not be renewed:
/etc/letsencrypt/live/unifi.pippostech.com.au/fullchain.pem (failure)
** DRY RUN: simulating 'certbot renew' close to cert expiry
** (The test certificates above have not been saved.)

1 renew failure(s), 0 parse failure(s)

IMPORTANT NOTES:

  • The following errors were reported by the server:

    Domain: unifi.pippostech.com.au
    Type: unauthorized
    Detail: Invalid response from
    http://unifi.pippostech.com.au/.well-known/acme-challenge/l3-73fABoVr7-jRt0qETyvGpPwk4_rm2u656DAus0Kw
    [35.189.9.26]: "<!doctype html><html lang="en">HTTP
    Status 404 – Not Found<style type="text/css">body
    {font-family:Tahoma"

    To fix these errors, please make sure that your domain name was
    entered correctly and the DNS A/AAAA record(s) for that domain
    contain(s) the right IP address.

My web server is (include version): Apache 2.4.25

The operating system my web server runs on is (include version): Debian 9.13

My hosting provider, if applicable, is: Google Cloud

I can login to a root shell on my machine (yes or no, or I don't know): yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): Certbot 1.10.1

Also, Apachectl -S outputs:
VirtualHost configuration:
*:443 unifi.pippostech.com.au (/etc/apache2/sites-enabled/000-default-le-ssl.conf:2)
*:80 unifi-controller.australia-southeast1-b.c.unifi-251300.internal (/etc/apache2/sites-enabled/000-default.conf:1)
ServerRoot: "/etc/apache2"
Main DocumentRoot: "/var/www/html"
Main ErrorLog: "/var/log/apache2/error.log"
Mutex watchdog-callback: using_defaults
Mutex rewrite-map: using_defaults
Mutex ssl-stapling-refresh: using_defaults
Mutex ssl-stapling: using_defaults
Mutex ssl-cache: using_defaults
Mutex default: dir="/var/run/apache2/" mechanism=default
PidFile: "/var/run/apache2/apache2.pid"
Define: DUMP_VHOSTS
Define: DUMP_RUN_CFG
User: name="www-data" id=33
Group: name="www-data" id=33

1 Like
2

What's listening on port 80 on this IP address?

It's for sure not Apache, because the Server response header is missing:

$ curl -i unifi.pippostech.com.au/.well-known/acme-challenge/test
HTTP/1.1 404
Content-Type: text/html;charset=utf-8
Content-Language: en
Content-Length: 431
Date: Mon, 07 Dec 2020 03:48:53 GMT

If something else is on that port, then Certbot isn't able to respond to the Let's Encrypt challenge in the right way.

Port 443 is Apache, but unfortunately that is not enough to make certbot --apache work.

You can check who is listening with:

sudo ss -tlnp | grep -E ":80"
2 Likes
3

Hi _az,

the command you posted returned the following result:

LISTEN 0 100 *:8080 : users:(("java",pid=10287,fd=153))
LISTEN 0 128 *:80 : users:(("apache2",pid=18869,fd=3),("apache2",pid=6201,fd=3),("apache2",pid=6200,fd=3))

1 Like
4

Thanks!

Did you happen to set up any port forwarding on this server to redirect port 80 to port 8080?

For example, using iptables or perhaps some tool in Google Cloud?

2 Likes
5

The iptables -S output is below, I don't think any of them are forwarding.
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-N f2b-sshd
-N f2b-unifi-controller
-A INPUT -p tcp -m multiport --dports 8443 -j f2b-unifi-controller
-A INPUT -p tcp -m multiport --dports 22 -j f2b-sshd
-A INPUT -p tcp -m tcp --dport 80 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
-A INPUT -p tcp -m tcp --dport 443 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
-A OUTPUT -p tcp -m tcp --sport 80 -m conntrack --ctstate ESTABLISHED -j ACCEPT
-A OUTPUT -p tcp -m tcp --sport 443 -m conntrack --ctstate ESTABLISHED -j ACCEPT
-A f2b-sshd -j RETURN
-A f2b-unifi-controller -j RETURN

The GCP firewall rules appear to be the same, only accepting, no redirection.

1 Like
6

That only shows the filter table, which might not catch the relevant rules. Try checking this one:

iptables-save
2 Likes
7

Absolute legend, I found the prerouting rule that forwarded 80 to 8080 and removed it.

Just ran a Certbot Dryrun and it passed, thank you so much!

2 Likes
8

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.