Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. https://crt.sh/?q=example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.
I ran this command:
We have multiple sites. So, I ran the following command:
Sudo certbot renew --dryrun
I got an email notification that the certificate was renewed for most of the sites except for two, for which I know the reason. In this list of successful additions, aioradar.com was also one of them. However, I still get notifications that the site requires renewal.
The operating system my web server runs on is (include version):
Apache
My hosting provider, if applicable, is:
Own server
I can login to a root shell on my machine (yes or no, or I donāt know):
Yes
Iām using a control panel to manage my site (no, or provide the name and version of the control panel):
Usually, yes, but now, we are working on an upgrade and updated the certificates command line.
The version of my client is (e.g. output of certbot --version or certbot-auto --version if youāre using Certbot):
Certbot 0.27.0
I think that this domain is not the one having the issue. It is another. This one is on Nginx server. I followed the certbot renewal process and also ran the shell that was created last time for this purpose, but it is throwing an error.
Install additional dependencies in order to request Letās Encrypt certificate
If running with jetty serving web content, will stop Jitsi Videobridge
Configure and reload nginx or apache2, whichever is used
Configure the coturn server to use Letās Encrypt certificate and add required deploy hooks
Add command in weekly cron job to renew certificates regularly
You need to agree to the ACME serverās Subscriber Agreement (https://letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf)
by providing an email address for important account notifications
Enter your email and press [ENTER]:
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Cert not yet due for renewal
Keeping the existing certificate
Certificate not yet due for renewal; no action taken.
How would you like to authenticate with the ACME CA?
1: Nginx Web Server plugin (nginx)
2: Spin up a temporary webserver (standalone)
3: Place files in webroot directory (webroot)
Select the appropriate number [1-3] then [enter] (press ācā to cancel): 1
Plugins selected: Authenticator nginx, Installer None
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for www.aioexplorer.com
nginx: [emerg] āserver_names_hash_bucket_sizeā directive is duplicate in /etc/nginx/sites-enabled/aioexplorer.com.conf:1
Cleaning up challenges
nginx restart failed:
bāā
bāā
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for www.aioexplorer.com
Using the webroot path /usr/share/jitsi-meet for all unmatched domains.
Waiting for verificationā¦
Challenge failed for domain www.aioexplorer.com
http-01 challenge for www.aioexplorer.com
Cleaning up challenges
Some challenges have failed.
it looks like jitsi makes some customization to nginxās config that doesnāt go down to well with certbot.
I suggest:
mkdir -p /var/www/certbot
then create a /etc/nginx/sites-available/certbot file:
server {
listen 80 default_server;
listen [::]:80 default_server;
location /.well-known/acme-challenge/ {
root /var/www/certbot;
}
# If you want an http -> https redirect, put it here
}
Thanks for your suggestion. I followed the suggested instructions, but it appears to be conflicting with the existing conf file.
See the output below -
ug 11 19:01:17 jitsi systemd[1]: Starting A high performance web server and a reverse proxy serverā¦
Aug 11 19:01:17 jitsi nginx[8856]: nginx: [emerg] a duplicate default server for 0.0.0.0:80 in /etc/nginx/sites-enabled/defa>
Aug 11 19:01:17 jitsi nginx[8856]: nginx: configuration file /etc/nginx/nginx.conf test failed
Aug 11 19:01:17 jitsi systemd[1]: nginx.service: Control process exited, code=exited, status=1/FAILURE
Aug 11 19:01:17 jitsi systemd[1]: nginx.service: Failed with result āexit-codeā.
Aug 11 19:01:17 jitsi systemd[1]: Failed to start A high performance web server and a reverse proxy server.
Aug 11 19:05:57 jitsi systemd[1]: nginx.service: Unit cannot be reloaded because it is inactive.
As soon as I removed the newly created file, the server was back and fully functional.
===========================================================
This is what is inside the current conf file -
server_names_hash_bucket_size 128;
It looks like aioexplorer.com and www.aioexplorer.com use two separate webroots.
This usually means some kind of webserver misconfiguration. Not in your case: dns misconfiguration it is:
% dig +short a aioexplorer.com
178.79.176.219
162.255.119.172
% dig +short a www.aioexplorer.com
178.79.176.219
% for ip in $(dig +short a aioexplorer.com); do echo $ip; curl -IL $ip; curl -IL --resolve aioexplorer.com:443:$ip https://aioexplorer.com; done
178.79.176.219
HTTP/1.1 200 OK
Date: Tue, 11 Aug 2020 20:10:08 GMT
Server: Apache/2.4.29 (Ubuntu)
Last-Modified: Wed, 20 May 2020 16:11:18 GMT
ETag: "ebb-5a616a568bcdb"
Accept-Ranges: bytes
Content-Length: 3771
Vary: Accept-Encoding
Content-Type: text/html
HTTP/2 200
server: nginx/1.17.10 (Ubuntu)
date: Tue, 11 Aug 2020 20:10:09 GMT
content-type: text/html
vary: Accept-Encoding
strict-transport-security: max-age=31536000
162.255.119.172
HTTP/1.1 204 No Content
Server: nginx
Date: Tue, 11 Aug 2020 20:10:09 GMT
Connection: keep-alive
X-Served-By: Namecheap URL Forward
curl: (7) Failed to connect to aioexplorer.com port 443: Connessione rifiutata
% for ip in $(dig +short a www.aioexplorer.com); do echo $ip; curl -IL $ip; curl -IL --resolve www.aioexplorer.com:443:$ip https://www.aioexplorer.com; done
178.79.176.219
HTTP/1.1 200 OK
Date: Tue, 11 Aug 2020 20:10:23 GMT
Server: Apache/2.4.29 (Ubuntu)
Last-Modified: Wed, 20 May 2020 16:11:18 GMT
ETag: "ebb-5a616a568bcdb"
Accept-Ranges: bytes
Content-Length: 3771
Vary: Accept-Encoding
Content-Type: text/html
HTTP/2 200
server: nginx/1.17.10 (Ubuntu)
date: Tue, 11 Aug 2020 20:10:24 GMT
content-type: text/html
vary: Accept-Encoding
strict-transport-security: max-age=31536000
And it also looks like you have Apache on port 80 and nginx on port 443ā¦ if you didnāt do it on purpose you have a problem. If you did do it on purpose, you have to do the challenges through apache, not nginx.
Thanks for bringing this to my attention. I also confirmed the same using nslookup and identified an extra url redirect record causing it. That has been taken care of now. Would this resolve the issue?
There is a cron shell script that says the domain is not yet available for renewal, which is right as there is still 10 days or so left. In that case, if I decide to renew manually, what would be the suggested way?
It was necessary to resolve it, but I don't think it will be sufficient.
That's not right, it should renew automatically at 30 days left, not 10. It's probably referring to your non-www domain, which has a 79-day certificate. Let's Debug Toolkit
About the whole apache and nginx thing: what can you tell me about it?
You could try with a certbot renew --apache --dry-run and see if it works...
I have done this renewal process many times before and should not be this complicated. Anyways, I decided to retrace the steps that I have done until now and thought of sharing.
This is nginx server and not an apache server. I am confident of it. However, as this community does not know the complete story, I am sharing the same here. Just ran a set of basic server commands to confirm that.
########:~$ systemctl status nginx
ā nginx.service - A high performance web server and a reverse proxy server
Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2020-08-12 10:56:24 UTC; 6min ago
Docs: man:nginx(8)
Process: 216 ExecStartPre=/usr/sbin/nginx -t -q -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
Process: 258 ExecStart=/usr/sbin/nginx -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
Process: 739 ExecReload=/usr/sbin/nginx -g daemon on; master_process on; -s reload (code=exited, status=0/SUCCESS)
Main PID: 283 (nginx)
Tasks: 5 (limit: 65000)
Memory: 21.2M
CGroup: /system.slice/nginx.service
āā283 nginx: master process /usr/sbin/nginx -g daemon on; master_process on;
āā741 nginx: worker process
āā742 nginx: worker process
āā743 nginx: worker process
āā744 nginx: worker process
Warning: some journal files were not opened due to insufficient permissions.
#######:~$ systemctl status apache2.service
Unit apache2.service could not be found.
As mentioned, both the www.aioexplorer.com and aioexplorer.com ssls have been functional for some time now, even with the redirect URL record that was showing two different ip addresses.
Commands tried:
a.Ran the following script, which led to the following error:
#!/bin/bash
/usr/bin/certbot renew >> /var/log/le-renew.log
service nginx reload
error:
Challenge failed for domain www.aioexplorer.com
http-01 challenge for www.aioexplorer.com
Cleaning up challenges
Attempting to renew cert (www.aioexplorer.com) from /etc/letsencrypt/renewal/www.aioexplorer.com.conf produced an unexpected error: Some challenges have failedā¦ Skipping.
All renewal attempts failed. The following certs could not be renewed:
/etc/letsencrypt/live/www.aioexplorer.com/fullchain.pem (failure)
1 renew failure(s), 0 parse failure(s)
b. root@j######### ./install* (Cron script that is usually provided for renewing SSL scripts).
This script will:
Need a working DNS record pointing to this machine(for domain aioexplorer.com)
Install additional dependencies in order to request Letās Encrypt certificate
If running with jetty serving web content, will stop Jitsi Videobridge
Configure and reload nginx or apache2, whichever is used
Configure the coturn server to use Letās Encrypt certificate and add required deploy hooks
Add command in weekly cron job to renew certificates regularly
You need to agree to the ACME serverās Subscriber Agreement (https://letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf)
by providing an email address for important account notifications
Enter your email and press [ENTER]: ###########
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Cert not yet due for renewal
Keeping the existing certificate
Certificate not yet due for renewal; no action taken.
Configuring nginx
c. Went through the sequence from scratch given on the cerbot site and reinstalled/updated all related apt files and tried to renew the certificate.
Cert not due for renewal, but simulating renewal for dry run
Plugins selected: Authenticator webroot, Installer None
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for aioexplorer.com
Using the webroot path /usr/share/jitsi-meet for all unmatched domains.
Waiting for verificationā¦
Cleaning up challenges
Dry run: skipping deploy hook command: /etc/letsencrypt/renewal-hooks/deploy/0000-coturn-certbot-deploy.sh
Skipping deploy-hook ā/etc/letsencrypt/renewal-hooks/deploy/0000-coturn-certbot-deploy.shā as it was already run.
new certificate deployed without reload, fullchain is
/etc/letsencrypt/live/aioexplorer.com/fullchain.pem
Cert is due for renewal, auto-renewingā¦
Plugins selected: Authenticator webroot, Installer None
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for www.aioexplorer.com
http-01 challenge for aioexplorer.com
Using the webroot path /usr/share/jitsi-meet for all unmatched domains.
Waiting for verificationā¦
Challenge failed for domain www.aioexplorer.com
http-01 challenge for www.aioexplorer.com
Cleaning up challenges
Attempting to renew cert (www.aioexplorer.com) from /etc/letsencrypt/renewal/www.aioexplorer.com.conf produced an unexpected error: Some challenges have failedā¦ Skipping.
The following certs could not be renewed:
/etc/letsencrypt/live/www.aioexplorer.com/fullchain.pem (failure)
** DRY RUN: simulating ācertbot renewā close to cert expiry
** (The test certificates below have not been saved.)
The following certs were successfully renewed:
/etc/letsencrypt/live/aioexplorer.com/fullchain.pem (success)
The following certs could not be renewed:
/etc/letsencrypt/live/www.aioexplorer.com/fullchain.pem (failure)
** DRY RUN: simulating ācertbot renewā close to cert expiry
** (The test certificates above have not been saved.)
To fix these errors, please make sure that your domain name was
entered correctly and the DNS A/AAAA record(s) for that domain
contain(s) the right IP address.
============================================================
I also asked the question what system change was done, which could result in authenticator or other errors, and we did update the server to 20.0.4. So, I went through the sequence to update the python environment apt files etc to accommodate for the update.
As per a Jitsi forum, this was also brought up, which confirmed my thought process.
I also followed their instructions and modified the installation shell script accordingly. The output presented above was after the change in the script. Before the change, it was not resulting in the error, but was pointing out the unavailability of Python scripts.
Out of curiosity, did you change your configuration for the renewal certificate to be a wildcard *. where previously you certified your base domain and www. separately?
I ask because:
@schoen
Didn't you tell me something the other day about how certbot would not performed mixed (http and dns) challenges without specifying both in --preferred-challenges. I believe this mattered when using a wildcard cert where previously there was no wildcard.
@skidambi75
I believe you're getting these failures because you're trying to use http challenges with a wildcard, which requires dns challenge. Since you didn't specify --preferred-challenge, it will try to use dns challenges for BOTH domains.
I noticed that you tried to expand (overlap the www) with the wildcard here. Your webroot will be irrelevant since a dns verification is required for wildcard.
My honest suggestion is that you do a "--manual --preferred-challenges dns -d aioexplorer.com -d *.aioexplorer.com" certification, manually add the DNS TXT records, and get and install a correct certificate just to get your ducks in order. Then try updating your process accordingly. That way you don't have multiple certs interfering with the process.
I just checked the records, and we have both listed. Not sure why. It has been a while. Usually, those did not appear to impact in the past.
I tried again three variants and the good news is that this one worked - @ (or without the hostname), which appears to align with your suggestion that it could be connected to wild card expansion. Thanks @freessltools.com
Congratulations! Your certificate and chain have been saved at:
############/aioexplorer.com/fullchain.pem
Your key file has been saved at:
############/aioexplorer.com/privkey.pem
Your cert will expire on 2020-11-10. To obtain a new or tweaked
version of this certificate in the future, simply run certbot
again. To non-interactively renew all of your certificates, run
"certbot renew"
Without the wildcard, only www & @ Challenge failed for domain www.aioexplorer.com
With the wildcard, * & @
Client with the currently selected authenticator does not support any combination of challenges that will satisfy the CA. You may need to use an authenticator plugin that can do challenges over DNS.
Client with the currently selected authenticator does not support any combination of challenges that will satisfy the CA. You may need to use an authenticator plugin that can do challenges over DNS.
If you don't mind, can you be explicit about the last part in your previous message? This would help me to add the wild card expansion to this certificate.
Blockquote
*My honest suggestion is that you do a āāmanual --preferred-challenges dns -d aioexplorer.com -d .aioexplorer.comā certification, manually add the DNS TXT records, and get and install a correct certificate just to get your ducks in order. Then try updating your process accordingly. That way you donāt have multiple certs interfering with the process.
So, if I'm understanding correctly, it looks like you previously got separate certificates for aioexplorer.com and www.aioexplorer.com, which will cause you to have different renewals. Combining them (and expanding to *.aioexplorer.com) is wise as it prevents a lot of confusion (and gives you more coverage than just www.). Trouble is that you can't use file-based challenges over http to do it. I suppose you could partially for the non-www, but why bother if you must use dns for *. anyhow.
My suggestion above was just to use a manual intervention process (which ignores the renewal dates) to get a correct certificate installed so that you would have the exact domains (aioexplorer.com and *.aioexplorer.com) you want in your cert. It was just to get you a baseline.
The irony here is that all the previous renewal failures were for www.aioexplorer.com. When you use the wildcard, you replace the broken http challenge with a dns challenge that requires dns txt records (which I don't believe you have a hook to create/destroy automatically). That's why I suggested the manual route. If you don't mind, try the following and tell me what you get:
certbot certonly --manual --preferred-challenges dns -d *.aioexplorer.com -d aioexplorer.com
Yes, I was just working on it, when you had shared this message. I had to sign into the registrar to add the record. Anyways, it is all done and I have added the wild card entry. This should help for the future. Thanks again for your timely assistance. It took a while to resolve, but that certainly helped to clean up my understanding on this certification process, and also to go over the Nginx server settings after a while.
Keep in mind that certonly does not install the certificate. It only gets you the certificate. I wanted to make sure we got that far first. Now you need to (cross your fingers and) install the new certificate. The process I had you go through does not solve your renewal configuration problems, but those can be resolved after you get a functional certificate for the correct domain combination installed.
Is there a reason you're not permanently redirecting aioexplorer.com to www.aioexplorer.com (or vice versa)? It's really bad for your search rankings to have non-authoritative duplication like this. Having a single canonical address is usually preferable.
