Problem renewing expired certificates for Apache on Google Cloud Compute

I've been trying to renew SSL certificates for my domain evelynferwalt.com that expired a few months back, and I'm having trouble with the renewal process. My web server is Apache 2.4.29 on a Google Cloud Compute f1-micro running Ubuntu 18.04.4 LTS. I use Google Domains for the domain service.

First of course I tried the easiest solution, to no avail.

sudo certbot renew

resulted in

Saving debug log to /var/log/letsencrypt/letsencrypt.log
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/evelynferwalt.com.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cert is due for renewal, auto-renewing...
Could not choose appropriate plugin: The manual plugin is not working; there may be problems with your existing configuration.
The error was: PluginError('An authentication script must be provided with --manual-auth-hook when using the manual plugin non-interactively.',)
Attempting to renew cert (evelynferwalt.com) from /etc/letsencrypt/renewal/evelynferwalt.com.conf produced an unexpected error: The manual plugin is not working; there may be problems with
 your existing configuration.
The error was: PluginError('An authentication script must be provided with --manual-auth-hook when using the manual plugin non-interactively.',). Skipping.
All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/evelynferwalt.com/fullchain.pem (failure)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/evelynferwalt.com/fullchain.pem (failure)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1 renew failure(s), 0 parse failure(s)

Next I tried

sudo certbot certonly --force-renew -d evelynferwalt.com

Although I'm not sure if this would also take care of my subdomains at *.evelynferwalt.com, I honestly don't really care about the subdomains right now. It output

How would you like to authenticate with the ACME CA?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: Apache Web Server plugin (apache)
2: Obtain certificates using a DNS TXT record (if you are using Google Cloud DNS
for DNS). (dns-google)
3: Spin up a temporary webserver (standalone)
4: Place files in webroot directory (webroot)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate number [1-4] then [enter] (press 'c' to cancel):

I selected 1, because I just sort of assumed that if I was using certbot commands, I would have the Apache Web Server Plugin. None of the other options seemed familiar or doable at first glance. It then spat out

Plugins selected: Authenticator apache, Installer None
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for evelynferwalt.com
Waiting for verification...
Cleaning up challenges
Failed authorization procedure. evelynferwalt.com (http-01): urn:ietf:params:acme:error:connection :: The server could not connect to the client to verify the domain :: Fetching http://evelynferwalt.com/.well-known/acme-challenge/-0MdCHZ6wMkeZrWE1OOM8W5-flUqkk92mQWNqvkx9Ko: Timeout during connect (likely firewall problem)

IMPORTANT NOTES: 
- The following errors were reported by the server:

Domain: evelynferwalt.com 
Type: connection 
Detail: Fetching http://evelynferwalt.com/.well-known/acme-challenge/-0MdCHZ6wMkeZrWE1OOM8W5-flUqkk92mQWNqvkx9Ko: Timeout during connect (likely firewall problem) 

To fix these errors, please make sure that your domain name was entered correctly and the DNS A/AAAA record(s) for that domain contain(s) the right IP address. Additionally, please check that your computer has a publicly routable IP address and that no firewalls are preventing the server from communicating with the client. If you're using the webroot plugin, you should also verify that you are serving files from the webroot path you provided.

I do not understand how the firewall could be so messed up on a web server specifically designed for web hosting! Do I need to fully generate a new key and delete the old one? How would I do that? If not, how do I renew an expired key of this kind?

Looking back, it was evidently this tutorial I used to create the certificate in the first place. I did it manually because I was trying to do a multi-site thing, with different subdomains, and it ended up with both a evelynferwalt.com and *.evelynferwalt.com certificate.

Unsure if this is useful, but here's the current contents of /etc/apache2/sites-enabled/evelynferwalt.com.conf, which used to work fine:

<Directory "/var/www/evelynferwalt.com">
    Options Indexes FollowSymLinks
    AllowOverride All
    Require all granted
</Directory>
<VirtualHost *:80>

        #~~~Recommended fix: use canonical name
        #UseCanonicalName on    

        # The ServerName directive sets the request scheme, hostname and port that
        # the server uses to identify itself. This is used when creating
        # redirection URLs. In the context of virtual hosts, the ServerName
        # specifies what hostname must appear in the request's Host: header to
        # match this virtual host. For the default virtual host (this file) this
        # value is not decisive as it is used as a last resort host regardless.
        # However, you must set it for any further virtual host explicitly.
        ServerName evelynferwalt.com
        ServerAlias www.evelynferwalt.com

        ServerAdmin webmaster@localhost
        DocumentRoot /var/www/evelynferwalt.com

        # Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
        # error, crit, alert, emerg.
        # It is also possible to configure the loglevel for particular
        # modules, e.g.
        #LogLevel info ssl:warn

        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined

        # For most configuration files from conf-available/, which are
        # enabled or disabled at a global level, it is possible to
        # include a line for only one particular virtual host. For example the
        # following line enables the CGI configuration for this host only
        # after it has been globally disabled with "a2disconf".
        #Include conf-available/serve-cgi-bin.conf
RewriteEngine on
RewriteCond %{HTTP_HOST} evelynferwalt\.com$
RewriteRule ^ https://%{HTTP_HOST}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>

# vim: syntax=apache ts=4 sw=4 sts=4 sr noet

And here's the final contents of /etc/apache2/sites-enabled/evelynferwalt.com-le-ssl.conf, which also used to work just fine:

<IfModule mod_ssl.c>
<VirtualHost *:443>
        # The ServerName directive sets the request scheme, hostname and port that
        # the server uses to identify itself. This is used when creating
        # redirection URLs. In the context of virtual hosts, the ServerName
        # specifies what hostname must appear in the request's Host: header to
        # match this virtual host. For the default virtual host (this file) this
        # value is not decisive as it is used as a last resort host regardless.
        # However, you must set it for any further virtual host explicitly.
        ServerName evelynferwalt.com
        ServerAlias www.evelynferwalt.com

        ServerAdmin webmaster@localhost
        DocumentRoot /var/www/evelynferwalt.com

        <Directory /var/www/www.example.com/public_html>
        Options -Indexes +FollowSymLinks
        AllowOverride All
        Order allow,deny
        allow from all
        </Directory>

        # Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
        # error, crit, alert, emerg.
        # It is also possible to configure the loglevel for particular
        # modules, e.g.
        #LogLevel info ssl:warn

        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined

        # For most configuration files from conf-available/, which are
        # enabled or disabled at a global level, it is possible to
        # include a line for only one particular virtual host. For example the
        # following line enables the CGI configuration for this host only
        # after it has been globally disabled with "a2disconf".
        #Include conf-available/serve-cgi-bin.conf
SSLCertificateFile /etc/letsencrypt/live/evelynferwalt.com/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/evelynferwalt.com/privkey.pem
Include /etc/letsencrypt/options-ssl-apache.conf
</VirtualHost>
</IfModule>

Lastly, if it helps, here's a link to a thread I started when I was first certifying this domain, which might have more context about what might be going on under the hood. Thank you!

1 Like

Hi @ASCII69,

Right now your DNS record for evelynferwalt.com and www.evelynferwalt.com are pointing at 35.222.146.150, which doesn't seem to have ports 80 or 443 accessible from the Internet.

Is there some kind of firewall policy that would prevent the public from accessing your server at all? It doesn't look like the main problem is with Certbot or Apache, but rather with something about the network connectivity to the server.

1 Like

I'm unsure what could have caused this to happen, but my Google Cloud VM external IP is now 34.72.156.19. You're right, my domain DNS was directed to something else (presumably the previous ip address) but I just put in the new external IP to the evelynferwalt.com DNS redirection.

Now the problem is doubly weird - I can't even access my Apache server directly via 34.72.156.19, going to port 80 or 443 or anything! I've triple checked that Apache2 exists and is actively running on the server. I guess now I'll hit stackoverflow or something to see if I can get the server even accessible before trying to certify it. Thanks!

2 Likes

Hi @ASCII69

I can -

https://evelynferwalt.com/

works.

Home

This website is still under construction. Check back soon for updates and new content!

Looks you have found a solution or you had an old browser cache.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.