Certbot failing with sudo certbot renew --dry-run

Help
1

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. https://crt.sh/?q=example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: apna5.com

I ran this command: sudo certbot renew --dry-run
I also ran: sudo certbot renew --dry-run --preferred-challenges http --webroot -w /var/www/html

It produced this output:
Attempting to renew cert (apna5.com) from /etc/letsencrypt/renewal/apna5.com.conf produced an unexpected error: Failed authorization procedure. www.apna5.com (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from https://www.yoursite.com [185.185.84.210]: "<html\nclass=“no-overflow-y avada-html-layout-wide” lang=en-US prefix=“og: http://ogp.me/ns# fb: http://ogp.me/ns/”. Skipping.
All renewal attempts failed. The following certs could not be renewed:
/etc/letsencrypt/live/apna5.com/fullchain.pem (failure)

/var/log/letsencrypt/letsencrypt.log
Traceback (most recent call last):
File “/usr/lib/python3/dist-packages/certbot/renewal.py”, line 452, in handle_renewal_request
main.renew_cert(lineage_config, plugins, renewal_candidate)
File “/usr/lib/python3/dist-packages/certbot/main.py”, line 1193, in renew_cert
renewed_lineage = _get_and_save_cert(le_client, config, lineage=lineage)
File “/usr/lib/python3/dist-packages/certbot/main.py”, line 116, in _get_and_save_cert
renewal.renew_cert(config, domains, le_client, lineage)
File “/usr/lib/python3/dist-packages/certbot/renewal.py”, line 310, in renew_cert
new_cert, new_chain, new_key, _ = le_client.obtain_certificate(domains, new_key)
File “/usr/lib/python3/dist-packages/certbot/client.py”, line 353, in obtain_certificate
orderr = self._get_order_and_authorizations(csr.data, self.config.allow_subset_of_names)
File “/usr/lib/python3/dist-packages/certbot/client.py”, line 389, in _get_order_and_authorizations
authzr = self.auth_handler.handle_authorizations(orderr, best_effort)
File “/usr/lib/python3/dist-packages/certbot/auth_handler.py”, line 82, in handle_authorizations
self._respond(aauthzrs, resp, best_effort)
File “/usr/lib/python3/dist-packages/certbot/auth_handler.py”, line 168, in _respond
self._poll_challenges(aauthzrs, chall_update, best_effort)
File “/usr/lib/python3/dist-packages/certbot/auth_handler.py”, line 239, in _poll_challenges
raise errors.FailedChallenges(all_failed_achalls)

My web server is (include version): nginx 1.14.2

The operating system my web server runs on is (include version): ubuntu 16.04

My hosting provider, if applicable, is: godaddy.com

I can login to a root shell on my machine (yes or no, or I don’t know): Yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel): No

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you’re using Certbot):

2

Can you show us the complete output from "sudo certbot renew --dry-run"?

The domain registration is with GoDaddy, but the website is running on a DigitalOcean droplet, right?

3

root@apna5-loadbal-prod:/# sudo certbot renew --dry-run
Saving debug log to /var/log/letsencrypt/letsencrypt.log

Processing /etc/letsencrypt/renewal/apna5.com.conf

Cert is due for renewal, auto-renewing…
Plugins selected: Authenticator webroot, Installer None
Starting new HTTPS connection (1): acme-staging-v02.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for apna5.com
http-01 challenge for www.apna5.com
Waiting for verification…
Cleaning up challenges
Attempting to renew cert (apna5.com) from /etc/letsencrypt/renewal/apna5.com.conf produced an unexpected error: Failed authorization procedure. www.apna5.com (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from https://www.yoursite.com [185.185.84.210]: "<html\nclass=“no-overflow-y avada-html-layout-wide” lang=en-US prefix=“og: http://ogp.me/ns# fb: http://ogp.me/ns/”, apna5.com (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from https://www.yoursite.com [185.185.84.210]: "<html\nclass=“no-overflow-y avada-html-layout-wide” lang=en-US prefix=“og: http://ogp.me/ns# fb: http://ogp.me/ns/”. Skipping.
All renewal attempts failed. The following certs could not be renewed:
/etc/letsencrypt/live/apna5.com/fullchain.pem (failure)

** DRY RUN: simulating ‘certbot renew’ close to cert expiry
** (The test certificates below have not been saved.)

All renewal attempts failed. The following certs could not be renewed:
/etc/letsencrypt/live/apna5.com/fullchain.pem (failure)
** DRY RUN: simulating ‘certbot renew’ close to cert expiry
** (The test certificates above have not been saved.)

1 renew failure(s), 0 parse failure(s)

IMPORTANT NOTES:

4

Yes, my nameservers are with digitalocean

5

My DNS record configuration:

I have 2 A configuration files, www.apna5.com & apna5.com & both point to 134.209.156.203

6

You’re using the webroot plugin, so your web server (Nginx) has to serve files from the configured directory at /.well-known/acme-challenge/ (or redirect somewhere that does).

http://apna5.com/.well-known/acme-challenge/test is a 301 redirect to https://apna5.com/.well-known/acme-challenge/test

https://apna5.com/.well-known/acme-challenge/test is a 307 redirect to http://acme.yourdomain.com/.well-known/acme-challenge/test?redirect=yes

http://acme.yourdomain.com/.well-known/acme-challenge/test?redirect=yes is a 302 redirect to https://www.yoursite.com

By the same token, http://www.apna5.com/.well-known/acme-challenge/test redirects to https://www.apna5.com/.well-known/acme-challenge/test, which also redirects to http://acme.yourdomain.com/.well-known/acme-challenge/test?redirect=yes

Are acme.yourdomain.com and www.yoursite.com your websites?

Can you show us the Nginx configuration for the apna5.com and www.apna5.com server blocks?

7

No. That's why I am very confused on how this is getting routed to.

8

cat nginx.conf

user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
worker_connections 768;
# multi_accept on;
}

http {

    ##
    # Basic Settings
    ##

    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 65;
    types_hash_max_size 2048;
    # server_tokens off;

    # server_names_hash_bucket_size 64;
    # server_name_in_redirect off;

    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    ##
    # SSL Settings
    ##

    ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
    ssl_prefer_server_ciphers on;

    ##
    # Logging Settings
    ##

    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;

    ##
    # Gzip Settings
    ##

    gzip on;

    # gzip_vary on;
    # gzip_proxied any;
    # gzip_comp_level 6;
    # gzip_buffers 16 8k;
    # gzip_http_version 1.1;
    # gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

    ##
    # Virtual Host Configs
    ##

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;

    client_max_body_size 50M;

}

#mail {

# See sample authentication script at:

# Using a PHP Script on an Apache Server as the IMAP Auth Backend | NGINX

# auth_http localhost/auth.php;

# pop3_capabilities "TOP" "USER";

# imap_capabilities "IMAP4rev1" "UIDPLUS";

server {

listen localhost:110;

protocol pop3;

proxy on;

}

server {

listen localhost:143;

protocol imap;

proxy on;

}

9

Hi @jerry_dev1

there are some checks of your domain, 9 and 3 hours old - https://check-your-website.server-daten.de/?q=apna5.com

If you have such a redirect list to acme.yourdomain.com, then to yoursite.com, your hoster blocks the Letsencrypt certificate http validation.

See

https://yoursite.com/ssl-certificates/

There you can buy Sectigo- or PositiveSSL- certificates.

Change to a hoster with Letsencrypt support.

PS: There are some other domains with the same redirect list.

10

Are you sure there aren’t any references to acme.yourdomain.com in your Nginx configuration?

“sudo nginx -T” can display the whole thing.

11

sudo nginx -T
nginx: [emerg] no "ssl_certificate_key" is defined for certificate "/etc/letsencrypt/live/apna5.com/fullchain.pem"
nginx: configuration file /etc/nginx/nginx.conf test failed

12

You need to fix that, and any other configuration issues. Nginx can’t reload or restart – or display the configuration – if it’s not valid.

13

The reason for failure is:
no “ssl_certificate_key” is defined for certificate “/etc/letsencrypt/live/apna5.com/fullchain.pem”

No changes were made to any config file so this is very unusual that I am getting all sorts of issues with the ssl cert getting expired

Should I just reinstall ssl?

14

I am not following you. I did take a look at the link that you posted. Could you please explain?

15

Someone has used my online tool to check your domain. First is (now) 10 hours old, so it's older then this topic. Last ( 25.08.2019 09:05:39 ) was my own check, reading this topic.

16

Either a change must have been made at some point, or the running Nginx must be using a different nginx.conf.

Without knowing exactly what's wrong, it's hard to say what would be the best way to fix it.

What do you mean by "reinstall"?

17

I have a clustered service. My loadbalancer.conf is ...
/etc/nginx/conf.d# cat loadbalancer.conf
upstream clusterwpadmin {
server 10.139..;
}
upstream clusternodes {
ip_hash;
server 10.139.. max_fails=3;
server 10.139.. max_fails=3;
}
server {
listen 80;

this block is for letsencrypt

root /var/www/html;

root /var/www/wpicluster;

location ~ /.well-known {
allow all;
try_files $uri $uri/ =404;
}
server_name _;
return 301 https://$host$request_uri;
location ~ /wp-(admin/|login.php\b|cron.php) {
proxy_pass http://clusterwpadmin;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
}
location / {
proxy_pass http://clusternodes;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
}
}
server {
listen 443 ssl;
ssl_certificate /etc/letsencrypt/live/apna5.com/cert.pem;
ssl_certificate /etc/letsencrypt/live/apna5.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/apna5.com/privkey.pem;
location ~ /wp-(admin/|login.php\b|cron.php) {
proxy_pass http://clusterwpadmin;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
}
location / {
proxy_pass http://clusternodes;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
}
}
#if a user connects to yourdomain.com:9443 they will be directed to node 1. This is where admins should connect to add plugins etc.
server {
listen 9443 ssl;
server_name _;
#ssl_certificate /etc/letsencrypt/live/yourdomain.com/cert.pem;
#ssl_certificate_key /etc/letsencrypt/live/yourdomain.com/privkey.pem;
location / {
proxy_pass http://clusterwpadmin;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
}
}

18

I tried deleting the ssl cert and now when I try to reissue it I get:
root@apna5-loadbal-prod:certbot certonly --webroot --webroot-path=/var/www/html -d www.apna5.com -d apna5.com
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Starting new HTTPS connection (1): acme-v02.api.letsencrypt.org
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for apna5.com
http-01 challenge for www.apna5.com
Using the webroot path /var/www/html for all unmatched domains.
Waiting for verification…
Cleaning up challenges
Failed authorization procedure. apna5.com (http-01): urn:ietf:params:acme:error:connection :: The server could not connect to the client to verify the domain :: Fetching http://apna5.com/.well-known/acme-challenge/zLArrRiWqoZe_YzCm-YhqgqQCCH-TbCz1SBMuix-Fso: Connection refused, www.apna5.com (http-01): urn:ietf:params:acme:error:connection :: The server could not connect to the client to verify the domain :: Fetching http://www.apna5.com/.well-known/acme-challenge/RaR9ZYCrSqhyj-sNrtNeQ0d8lG4RnzecH8GvRiyqKcc: Connection refused

IMPORTANT NOTES:

19

Oh…I might have searched. I was searching on the web for various solutions & I did bump into your website link.

20

What did you delete?

How did you delete it?

Did you stop or try to restart Nginx?

What does “sudo nginx -T” show now?

/etc/nginx/nginx.conf is Nginx’s central configuration file, right?