As the incoming dev for my company, I've inherited the AWS managment of our sites, set up buy previous developers, we are using Certbot - which seems to be able to renew some domains on the EC2 instance, but not others. I'm wondering if I can get some assistance with the below issue.
sudo certbot renew --dry-run or sudo certbot renew
It produced this output:
Domain: staging.eastern-escape.co.uk
Type: connection
Detail: 3.10.33.189: Fetching
http://staging.eastern-escape.co.uk/.well-known/acme-challenge/iVwyDaWwbY9JT4_yAlzUjs0gmA_FFKGdHYr-uLcctZs:
Connection refused
All renewal attempts failed. The following certs could not be renewed:
/etc/letsencrypt/live/staging.burlington-place.co.uk/fullchain.pem (failure)
/etc/letsencrypt/live/staging.eastern-escape.co.uk/fullchain.pem (failure)
My web server is (include version):
Server version: Apache/2.4.57 (Ubuntu)
Server built: 2023-04-08T12:55:31
The operating system my web server runs on is (include version):
Distributor ID:
Ubuntu
Description:
Ubuntu 20.04.6 LTS
Release:
20.04
Codename:
focal
My hosting provider, if applicable, is:
Amazon AWS EC2
I can login to a root shell on my machine (yes or no, or I don't know):
Yes (With sudo)
I'm using a control panel to manage my site (no, or provide the name and version of the control panel):
The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): certbot 0.40.0
I cannot reproduce the "Connection Refused" from my own AWS test server or from the Let's Debug test site
My guess is you have some sort of firewall blocking the IP address(es) of the Let's Encrypt validation servers.
I see a problem with your Apache server's redirects though. The HTTP Challenge request is being redirected to HTTPS and your home page. Instead, your server needs to reply with the challenge token placed by Certbot.
#Sample HTTP Challenge redirected to index.php
curl -ILk http://staging.eastern-escape.co.uk/.well-known/acme-challenge/Test404 -A "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
HTTP/1.1 302 Found
Server: Apache/2.4.57 (Ubuntu)
Location: https://staging.eastern-escape.co.uk/index.php
# Returns index.php value rather than error 404 (for this test)
# (the -k above is needed to ignore the expired cert for this domain)
HTTP/1.1 200 OK
Date: Thu, 30 Nov 2023 14:35:58 GMT
Server: Apache/2.4.57 (Ubuntu)
It is one problem. But, do you still get "Connection Refused" on repeated attempts? And, do you see that Refused with both --dry-run and without? Because this is the first problem to solve.
As for Apache, good place to start is showing us this output
sudo apache2ctl -t -D DUMP_VHOSTS
As an aside, Certbot 0.40 from the apt install of that Ubuntu is very old. The recommended install is using the snap version. Follow the instructions below carefully. There have been numerous improvements to certbot since then. Although, none would resolve a "connection refused"
We do indeed get connection refused on eastern for both renew and renew --dry-run
Please see below for the output of $sudo certbt renew
$ sudo certbot renew
Saving debug log to /var/log/letsencrypt/letsencrypt.log
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/staging.burlington-place.co.uk.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cert is due for renewal, auto-renewing...
Plugins selected: Authenticator standalone, Installer None
Running pre-hook command: service apache2 stop
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for staging.maynard-meadows.co.uk
Waiting for verification...
Challenge failed for domain staging.maynard-meadows.co.uk
http-01 challenge for staging.maynard-meadows.co.uk
Cleaning up challenges
Attempting to renew cert (staging.burlington-place.co.uk) from /etc/letsencrypt/renewal/staging.burlington-place.co.uk.conf produced an unexpected error: Some challenges have failed.. Skipping.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/staging.eastern-escape.co.uk.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cert is due for renewal, auto-renewing...
Plugins selected: Authenticator webroot, Installer None
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for staging.eastern-escape.co.uk
Waiting for verification...
Challenge failed for domain staging.eastern-escape.co.uk
http-01 challenge for staging.eastern-escape.co.uk
Cleaning up challenges
Attempting to renew cert (staging.eastern-escape.co.uk) from /etc/letsencrypt/renewal/staging.eastern-escape.co.uk.conf produced an unexpected error: Some challenges have failed.. Skipping.
All renewal attempts failed. The following certs could not be renewed:
/etc/letsencrypt/live/staging.burlington-place.co.uk/fullchain.pem (failure)
/etc/letsencrypt/live/staging.eastern-escape.co.uk/fullchain.pem (failure)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
All renewal attempts failed. The following certs could not be renewed:
/etc/letsencrypt/live/staging.burlington-place.co.uk/fullchain.pem (failure)
/etc/letsencrypt/live/staging.eastern-escape.co.uk/fullchain.pem (failure)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Running post-hook command: service apache2 start
2 renew failure(s), 0 parse failure(s)
IMPORTANT NOTES:
- The following errors were reported by the server:
Domain: staging.eastern-escape.co.uk
Type: connection
Detail: 3.10.33.189: Fetching
http://staging.eastern-escape.co.uk/.well-known/acme-challenge/XXvl3jRHBKO12dJtj5FW7fZtFRf7qw4MwuGLKPBNou4:
Connection refused
To fix these errors, please make sure that your domain name was
entered correctly and the DNS A/AAAA record(s) for that domain
contain(s) the right IP address. Additionally, please check that
your computer has a publicly routable IP address and that no
firewalls are preventing the server from communicating with the
client. If you're using the webroot plugin, you should also verify
that you are serving files from the webroot path you provided.
- The following errors were reported by the server:
Domain: staging.maynard-meadows.co.uk
Type: dns
Detail: DNS problem: NXDOMAIN looking up A for
staging.maynard-meadows.co.uk - check that a DNS record exists for
this domain; DNS problem: NXDOMAIN looking up AAAA for
staging.maynard-meadows.co.uk - check that a DNS record exists for
this domain
I agree that the cert version is outdated, however, given the nature of these servers, I'd rather solve one issue at a time, so this will be added to my todo after I get the certificates renewed sucessfull (as I alss want to add this action to a cron job)
As you can see from this dump, we have a few domains on this service:
$sudo apache2ctl -t -D DUMP_VHOSTS
VirtualHost configuration:
*:80 staging.burlington-place.co.uk (/etc/apache2/sites-enabled/vhosts.conf:36)
*:443 is a NameVirtualHost
default server staging.eastern-escape.co.uk (/etc/apache2/sites-enabled/vhosts.conf:3)
port 443 namevhost staging.eastern-escape.co.uk (/etc/apache2/sites-enabled/vhosts.conf:3)
port 443 namevhost staging.burlington-place.co.uk (/etc/apache2/sites-enabled/vhosts.conf:54)
port 443 namevhost staging.alexandra-mansions.co.uk (/etc/apache2/sites-enabled/vhosts.conf:87)
port 443 namevhost staging.thompson-staithe.co.uk (/etc/apache2/sites-enabled/vhosts.conf:120)
port 443 namevhost staging.tollesbury-house.co.uk (/etc/apache2/sites-enabled/vhosts.conf:153)
port 443 namevhost staging.narrows-place.co.uk (/etc/apache2/sites-enabled/vhosts.conf:186)
port 443 namevhost staging.maynard-meadows.co.uk (/etc/apache2/sites-enabled/vhosts.conf:219)
port 443 namevhost staging.simpsons-place.co.uk (/etc/apache2/sites-enabled/vhosts.conf:252)
Thanks again for your help in this, whilst I am not a total novice in this kind of operation, my lack of certain bits of knowledge as well as picking up someone else's work is making me tread carefully, so I do appreciate your patience.
Your first cert uses --standalone and you stop Apache to allow that to work. Note this for later. But, it is for the domain staging.maynard-meadows.co.uk which no longer has an A or AAAA record in the public DNS. Do you still need a cert for this domain? It's a little confusing to see that domain name in a conf file named for staging.burlington-place.co.uk. Do you know how that happened?
Your second cert for staging.eastern-escape uses --webroot which requires Apache to handle the request. But, it looks like Apache is not restarted until after all the cert renewals are tried. I am going just by when the running post-hook command: service apache2 start is issued. And, this would explain why you get connection refused for the --webroot request.
You are not wrong. I've picked this up after my company not having a dev for a few months.
When I asked the question internally the initial response was 'We have a site for that?' so I'm going to go with no... and that pretty much sums up the situation I'm in right now.
Can you show the entire contents of this file? It looks large so ideally use the upload file button. You may need to copy it to a temp folder as a .txt to upload it (I am not sure).
Also, there are 2 renewal conf files in the /etc/letsencrypt/renewal folder. Please paste output of both of those here too.
That's not as simple as you might imagine. Your Apache server will fail to start if it refers to cert files that don't exist. Recovering from that is probably harder than fixing your cert profiles.
Let's start by trying to renew just your staging.eastern domain. Can you show us what this does:
certbot: error: ambiguous option: --renew could match --renew-by-default, --renew-with-new-domains, --renew-hook
I assumed --renew was renew and got the following:
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/staging.eastern-escape.co.uk.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cert is due for renewal, auto-renewing...
Plugins selected: Authenticator webroot, Installer None
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for staging.eastern-escape.co.uk
Waiting for verification...
Challenge failed for domain staging.eastern-escape.co.uk
http-01 challenge for staging.eastern-escape.co.uk
Cleaning up challenges
Attempting to renew cert (staging.eastern-escape.co.uk) from /etc/letsencrypt/renewal/staging.eastern-escape.co.uk.conf produced an unexpected error: Some challenges have failed.. Skipping.
All renewal attempts failed. The following certs could not be renewed:
/etc/letsencrypt/live/staging.eastern-escape.co.uk/fullchain.pem (failure)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
** DRY RUN: simulating 'certbot renew' close to cert expiry
** (The test certificates below have not been saved.)
All renewal attempts failed. The following certs could not be renewed:
/etc/letsencrypt/live/staging.eastern-escape.co.uk/fullchain.pem (failure)
** DRY RUN: simulating 'certbot renew' close to cert expiry
** (The test certificates above have not been saved.)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1 renew failure(s), 0 parse failure(s)
IMPORTANT NOTES:
- The following errors were reported by the server:
Domain: staging.eastern-escape.co.uk
Type: unauthorized
Detail: 3.10.33.189: Invalid response from
https://staging.eastern-escape.co.uk/index.php: "<!DOCTYPE
html>\n<html lang=\"en\" style=\"scroll-behavior: smooth;\">\n
<head>\n <meta charset=\"utf-8\">\n <meta name=\""
To fix these errors, please make sure that your domain name was
entered correctly and the DNS A/AAAA record(s) for that domain
contain(s) the right IP address.
I totally get that - I have to also look into this on our live server too so I'm just making sure we have the right procedure in place, whatever that may be. I appricate everyones patience in this.