Creating certificates with --nginx works. Renew or certonly doesn't

Help
62

In sites-enabled I have one file with - all of them in one file:

fastcgi_cache_path /var/run/nginx-cache levels=1:2 keys_zone=WORDPRESS:100m inactive=60m;
fastcgi_cache_key “$scheme$request_method$host$request_uri”;
fastcgi_cache_use_stale error timeout invalid_header http_500;
fastcgi_ignore_headers Cache-Control Expires Set-Cookie;
server {
listen 80;
listen [::]:80;
listen 443 default ssl http2;
listen [::]:443 default ssl http2 ipv6only=on;
server_name .aamjanata.com wide-aware.com;
root /var/www/aj;
include global/wordpress-fastcgi.conf;
}
server {
listen 80;
listen [::]:80;
listen 443 ssl http2;
listen [::]:443 ssl http2;
server_name .digitalindia.watch;
root /var/www/diwdru;
include global/diw1.conf;
}
server {
listen 80;
listen [::]:80;
listen 443 ssl http2;
listen [::]:443 ssl http2;
server_name .fekle.in;
root /var/www/fl;
include global/wordpress-fastcgi.conf;
}
server {
listen 80;
listen [::]:80;
listen 443 ssl http2;
listen [::]:443 ssl http2;
server_name .nisarga.info;
root /var/www/ni;
include global/wordpress-fastcgi.conf;
}
server {
listen 80;
listen [::]:80;
listen 443 ssl http2;
listen [::]:443 ssl http2;
server_name .vidyut.net;
root /var/www/vn;
include global/wordpress-fastcgi.conf;
}
server {
listen 80;
listen [::]:80;
listen 443 ssl http2;
listen [::]:443 ssl http2;
server_name .vidyut.info;
root /var/www/vin;
include global/wordpress-fastcgi.conf;
}
server {
listen 80;
listen [::]:80;
listen 443 ssl http2;
listen [::]:443 ssl http2;
server_name .homeschoolingindia.in;
root /var/www/hs;
include global/wordpress-fastcgi.conf;
}
server {
listen 80;
listen [::]:80;
listen 443 ssl http2;
listen [::]:443 ssl http2;
server_name india.aamjanata.com;
ssl_certificate /etc/letsencrypt/live/vidyut.net-0001/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/vidyut.net-0001/privkey.pem;
root /var/www/map;
add_header Strict-Transport-Security “max-age=31536000”;
add_header Alternate-Protocol 443:npn-spdy/3;
ssl_protocols SSLv3 TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_ciphers !aNULL:!LOW:!MD5:!EXP:RC4:AES256:3DES:AES128:SEED:CAMELLIA;
resolver 8.8.8.8 8.8.4.4 208.67.222.222 208.67.220.220 valid=300s;
resolver_timeout 10s;
ssl_stapling on;
ssl_stapling_verify on;
ssl_trusted_certificate /etc/nginx/ssl/stapling.crt;
charset utf-8;
override_charset on;

location /platform {
	fastcgi_param   QUERY_STRING            $query_string;
	fastcgi_param   REQUEST_METHOD          $request_method;
	fastcgi_param   CONTENT_TYPE            $content_type;
	fastcgi_param   CONTENT_LENGTH          $content_length;

	fastcgi_split_path_info					^(/platform/)(.*)$;

	fastcgi_param   SCRIPT_FILENAME         $document_root/platform/httpdocs/index.php;
	fastcgi_param   SCRIPT_NAME             $fastcgi_script_name;
	fastcgi_param   PATH_INFO               $fastcgi_path_info;
	fastcgi_param   PATH_TRANSLATED         $document_root$fastcgi_path_info;
	fastcgi_param   REQUEST_URI             $request_uri;
	fastcgi_param   DOCUMENT_URI            $document_uri;
	fastcgi_param   DOCUMENT_ROOT           $document_root;
	fastcgi_param   SERVER_PROTOCOL         $server_protocol;

	fastcgi_param   GATEWAY_INTERFACE       CGI/1.1;
	fastcgi_param   SERVER_SOFTWARE         nginx/$nginx_version;

	fastcgi_param   REMOTE_ADDR             $remote_addr;
	fastcgi_param   REMOTE_PORT             $remote_port;
	fastcgi_param   SERVER_ADDR             $server_addr;
	fastcgi_param   SERVER_PORT             $server_port;
	fastcgi_param   SERVER_NAME             $server_name;

	fastcgi_param   HTTPS                   $https;

	fastcgi_pass    php;
}

location / {
	try_files $uri /index.html;
}

}
server {
listen 80;
listen [::]:80;
listen 443 ssl http2;
listen [::]:443 ssl http2;
server_name .aadhaar.fail;
include global/wordpress-fastcgi.conf;
root /var/www/af;
}

server {
listen 80;
listen [::]:80;
listen 443 ssl http2;
listen [::]:443 ssl http2;
server_name .aamjanata.in;
root /var/www/ajin;
include global/wordpress-fastcgi.conf;
}

63

Included file contains

ssl_certificate /etc/letsencrypt/live/vidyut.net-0001/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/vidyut.net-0001/privkey.pem;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
ssl_ciphers ‘ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA’;
ssl_prefer_server_ciphers on;
ssl_dhparam /etc/nginx/ssl/dhparams.pem;
resolver 8.8.8.8 8.8.4.4 208.67.222.222 208.67.220.220 valid=300s;
ssl_ecdh_curve X25519:P-256:P-384:P-521;
resolver_timeout 10s;

ssl_stapling on;
ssl_stapling_verify on;
ssl_trusted_certificate /etc/nginx/ssl/stapling.crt;
set $skip_cache 0;

	if ($request_method = POST) {
	set $skip_cache 1;
}   
if ($query_string != "") {
	set $skip_cache 1;
}   

	if ($request_uri ~* "/wp-admin/|/xmlrpc.php|wp-.*.php|/feed/|index.php|sitemap(_index)?.xml") {
	set $skip_cache 1;
}   

	if ($http_cookie ~* "comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_no_cache|wordpress_logged_in") {
	set $skip_cache 1;
}

rewrite ^/wp-content/uploads/knewsimages/(.*).(jpg|jpeg|gif|png)$ /wp-content/plugins/knews/direct/track.php?img=$1 break;
location / {
try_files $uri $uri/ /index.php?$args;
}
location ~ /.well-known/acme-challenge {
allow all;
default_type text/plain;
}

    location ~ \.php$ {
            try_files $uri /index.php;
            include fastcgi_params;
            fastcgi_pass php;

fastcgi_buffers 8 128k;
fastcgi_buffer_size 256k;
fastcgi_cache_bypass $skip_cache;
fastcgi_no_cache $skip_cache;
fastcgi_read_timeout 300;
fastcgi_cache WORDPRESS;
fastcgi_cache_valid 60m;
add_header X-Cache $upstream_cache_status;
}

location ~ /purge(/.*) {
    	}	
location ~* ^.+\.(ogg|ogv|svg|svgz|eot|otf|woff|mp4|ttf|rss|atom|gif|ico|zip|tgz|gz|rar|bz2|doc|xls|exe|ppt|tar|mid|midi|wav|bmp|rtf|css|js)$ {
	access_log off;	log_not_found off; expires max;

fastcgi_hide_header “Set-Cookie”;
}
location ~* ^.+.(png|jpe?g)$ {
add_header Vary Accept;
access_log off; error_log off; expires max;
try_files $uri$webp_suffix $uri =404;
}

64

Are those names all actually prepended with "." ?

65

yes. to serve domain and subdomains. Is that a problem?

66

I speak to dozens of people - please don't make me look for that info...
What does it say?

67

sorry. default_type text/plain; - adding it to that post also.

68

OK I find nothing obviously wrong with the setup…
Looking closer at the error message(s)…
They all contain “Timeout during connect (likely firewall problem)”
Perhaps we are now facing a problem outside your system - Like: Firewall/IPS/Geo-Location blocking

69

It is a digitalocean droplet. No idea if they block any traffic, but I certainly haven’t blocked any. No firewall, no IPs, no geo-location, etc

If DO were blocking, we’d see a lot more of these problems by now.

70

what about these names?:

71

I could try switching “.example.com” type server names to the longer, but same “example.com and *.example.com” if you think it matters. Nginx accepts this configuration - letsencrypt also has so far. Sites are served and functioning perfectly well. http://nginx.org/en/docs/http/server_names.html (see “wildcard” section)

This has been the configuration for years and certificates have been updating smoothly so far.

72

Not with “*” - match the names in the cert.
like:
server_name digitalindia.watch www.digitalindia.watch;
or separately like:
server_name digitalindia.watch;
server_name www.digitalindia.watch;

73

What I don’t get is why only 10 fetches fail (out of 21 names on the cert):

Detail: Fetching http://vidyut.net/.well-known/acme-challenge/mtAlSi9QZUHAwxc5R16Io_nAecrv_7ISEfoDjA515tk: Timeout during connect (likely firewall problem)
Detail: Fetching http://www.vidyut.info/.well-known/acme-challenge/wfYJ4d9tFgk3D0r40fm9XDyUzLp4mahjtzLIx7k-HCs: Timeout during connect (likely firewall problem)
Detail: Fetching http://www.digitalindia.watch/.well-known/acme-challenge/whEiWqXVvwLPvoTXs5zb-IdEL9csQp0CtrLyWwSzQgU: Timeout during connect (likely firewall problem)
Detail: Fetching http://www.homeschoolingindia.in/.well-known/acme-challenge/KNvSiSkCIDM6Qes-wTnusT6g8X-q1VdNQBew0mftIOM: Timeout during connect (likely firewall problem)
Detail: Fetching http://www.wide-aware.com/.well-known/acme-challenge/tAbsTbJ9KPYaDjjdEebsXrc6IhaEVVduO7xasAXwQiQ: Timeout during connect (likely firewall problem)
Detail: Fetching http://nisarga.info/.well-known/acme-challenge/F-9Dq4_G1vSdk91DFNKEN7F_oIKLZxaq2_N2_Cx7cFM: Timeout during connect (likely firewall problem)
Detail: Fetching http://www.aamjanata.in/.well-known/acme-challenge/wypFslfVw8dBZTtYuI4kW1esWkikEDTknBX8L1ZCKjA: Timeout during connect (likely firewall problem)
Detail: Fetching http://digitalindia.watch/.well-known/acme-challenge/zSQHYe7p9kjcl5O572-X9TVJZUmjBm4Qu2cEC07NbRo: Timeout during connect (likely firewall problem)
Detail: Fetching http://homeschoolingindia.in/.well-known/acme-challenge/K2uZfkwTvry2U1b7jEeQodIwEGFOATIEmprZGBgX8rU: Timeout during connect (likely firewall problem)
Detail: Fetching http://www.aadhaar.fail/.well-known/acme-challenge/s-LUeRrJpNR3PVPjuohuv7_GFpBVql30clShAtwCPUY: Timeout during connect (likely firewall problem)

            DNS:aadhaar.fail
            DNS:aamjanata.com
            DNS:aamjanata.in
            DNS:digitalindia.watch
            DNS:fekle.in
            DNS:homeschoolingindia.in
            DNS:india.aamjanata.com
            DNS:nisarga.info
            DNS:vidyut.info
            DNS:vidyut.net
            DNS:wide-aware.com
            DNS:www.aadhaar.fail
            DNS:www.aamjanata.com
            DNS:www.aamjanata.in
            DNS:www.digitalindia.watch
            DNS:www.fekle.in
            DNS:www.homeschoolingindia.in
            DNS:www.nisarga.info
            DNS:www.vidyut.info
            DNS:www.vidyut.net
            DNS:www.wide-aware.com
74

Same result. Timed out.

75

First thing I'd done was checked error logs for if the server was under stress or anything. Server normal. No timeouts (other than some php plugin related ones - irrelevant to this and timings don't match) or ANY errors related to .well-known

I have to go out for two hours right now, but do you think it is worth activating access log for .well-known when I return? To see if the requests are being made at all?

I don't think this is a DNS issue. Different files fail to fetch - not same ones. Plus sites are and have been working fine for years.

76

Timeout it network.
Why? I don’t know.
You can reach a test file.
I can reach a test file.
LE (staging) can’t reach their test file.

Today production is only using HTTP-01 (like staging is with --dry-run).
We can try to force a renewal and that will be done via HTTP-01.

Try this only once:
certbot renew --installer null --force-renewal -vv

[and show the output]

77

Looooooong output (as usual) but summary is:

Congratulations, all renewals succeeded. The following certs have been renewed:
/etc/letsencrypt/live/vidyut.info-0001/fullchain.pem (success)

no renewal failures

Still need log for debugging? Can post.

78

As long as it shows HTTP-01 (which it should) that part is OK.

I don’t understand why the production systems can reach your server and yet the staging systems can’t…

79

The only thing to check (which has now been reset to 60 days from today) is that it can renew automatically on its’ own.

80

Happy to help debug in any way, but this means that when the time comes, actual renewal will work and I can breathe easy?

Have to head out right now, but will return in 2 hours.

81

Yes, we can continue in 61 days (if it fails to renew automatically).