Getting Connection reset by peer, i have been struggling for two days with no success

I have been using letsencrypt certificate for 3 months. but i can not renew, i reinstalled certbot and tried to get new certificate with no success. i don't see any error in nginx error log. and A and AAAA records are created and both port 443 and port 80 are open.
my nginx config looks like this:

server {
listen 80;
listen [::]:80;

server_name estudent.kmu.edu.et;
root /home/user/workspace/myapp/public;

passenger_enabled on;
passenger_app_env production;
}

My domain is:
estudent.kmu.edu.et

I ran this command:
sudo certbot --nginx -d estudent.kmu.edu.et

It produced this output:
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator nginx, Installer nginx
Enter email address (used for urgent renewal and security notices) (Enter 'c' to
cancel): estudent.system@gmail.com


Please read the Terms of Service at
https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf. You must
agree in order to register with the ACME server at
https://acme-v02.api.letsencrypt.org/directory


(A)gree/(C)ancel: A


Would you be willing to share your email address with the Electronic Frontier
Foundation, a founding partner of the Let's Encrypt project and the non-profit
organization that develops Certbot? We'd like to send you email about our work
encrypting the web, EFF news, campaigns, and ways to support digital freedom.


(Y)es/(N)o: Y
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for estudent.kmu.edu.et
Waiting for verification...
Cleaning up challenges
Failed authorization procedure. estudent.kmu.edu.et (http-01): urn:ietf:params:acme:error:connection :: The server could not connect to the client to verify the domain :: Fetching http://estudent.kmu.edu.et/.well-known/acme-challenge/641Di3Co-c_fFVHgeTorJvfICt0vY0Ayg5TfyUk5QlU: Connection reset by peer

IMPORTANT NOTES:

  • The following errors were reported by the server:

    Domain: estudent.kmu.edu.et
    Type: connection
    Detail: Fetching
    http://estudent.kmu.edu.et/.well-known/acme-challenge/641Di3Co-c_fFVHgeTorJvfICt0vY0Ayg5TfyUk5QlU:
    Connection reset by peer

    To fix these errors, please make sure that your domain name was
    entered correctly and the DNS A/AAAA record(s) for that domain
    contain(s) the right IP address. Additionally, please check that
    your computer has a publicly routable IP address and that no
    firewalls are preventing the server from communicating with the
    client. If you're using the webroot plugin, you should also verify
    that you are serving files from the webroot path you provided.

  • Your account credentials have been saved in your Certbot
    configuration directory at /etc/letsencrypt. You should make a
    secure backup of this folder now. This configuration directory will
    also contain certificates and private keys obtained by Certbot so
    making regular backups of this folder is ideal.

My web server is (include version):
nginx/1.14.0 (Ubuntu)

The operating system my web server runs on is (include version):
Ubuntu 18.04.5 LTS (GNU/Linux 4.15.0-117-generic x86_64)

My hosting provider, if applicable, is:
Ethio Telecom(Ethiopia)

I can login to a root shell on my machine (yes or no, or I don't know):
Yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):
No

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):
certbot 0.31.0

1 Like

Hi @yohannes

see your error: Your port 80 doesn't work. A working port 80 is required if you want to use http validation.

Is there a firewall?

What says

nginx -T
2 Likes

Hello @JuergenAuer Thank Your for quick response,
nginx -T results in

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
# configuration file /etc/nginx/nginx.conf:
user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
	worker_connections 768;
	# multi_accept on;
}

http {

	##
	# Basic Settings
	##

	sendfile on;
	tcp_nopush on;
	tcp_nodelay on;
	keepalive_timeout 65;
	types_hash_max_size 2048;
	# server_tokens off;

	server_names_hash_bucket_size 64;
	# server_name_in_redirect off;

	include /etc/nginx/mime.types;
	default_type application/octet-stream;

	##
	# SSL Settings
	##

	ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
	ssl_prefer_server_ciphers on;

	##
	# Logging Settings
	##

	access_log /var/log/nginx/access.log;
	error_log /var/log/nginx/error.log;

	##
	# Gzip Settings
	##

	gzip on;

	# gzip_vary on;
	# gzip_proxied any;
	# gzip_comp_level 6;
	# gzip_buffers 16 8k;
	# gzip_http_version 1.1;
	# gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

	##
	# Virtual Host Configs
	##

	include /etc/nginx/conf.d/*.conf;
	include /etc/nginx/sites-enabled/*;
}


#mail {
#	# See sample authentication script at:
#	# http://wiki.nginx.org/ImapAuthenticateWithApachePhpScript
# 
#	# auth_http localhost/auth.php;
#	# pop3_capabilities "TOP" "USER";
#	# imap_capabilities "IMAP4rev1" "UIDPLUS";
# 
#	server {
#		listen     localhost:110;
#		protocol   pop3;
#		proxy      on;
#	}
# 
#	server {
#		listen     localhost:143;
#		protocol   imap;
#		proxy      on;
#	}
#}

# configuration file /etc/nginx/modules-enabled/10-mod-http-ndk.conf:
load_module modules/ndk_http_module.so;

# configuration file /etc/nginx/modules-enabled/50-mod-http-auth-pam.conf:
load_module modules/ngx_http_auth_pam_module.so;

# configuration file /etc/nginx/modules-enabled/50-mod-http-cache-purge.conf:
load_module modules/ngx_http_cache_purge_module.so;

# configuration file /etc/nginx/modules-enabled/50-mod-http-dav-ext.conf:
load_module modules/ngx_http_dav_ext_module.so;

# configuration file /etc/nginx/modules-enabled/50-mod-http-echo.conf:
load_module modules/ngx_http_echo_module.so;

# configuration file /etc/nginx/modules-enabled/50-mod-http-fancyindex.conf:
load_module modules/ngx_http_fancyindex_module.so;

# configuration file /etc/nginx/modules-enabled/50-mod-http-geoip.conf:
load_module modules/ngx_http_geoip_module.so;

# configuration file /etc/nginx/modules-enabled/50-mod-http-headers-more-filter.conf:
load_module modules/ngx_http_headers_more_filter_module.so;

# configuration file /etc/nginx/modules-enabled/50-mod-http-image-filter.conf:
load_module modules/ngx_http_image_filter_module.so;

# configuration file /etc/nginx/modules-enabled/50-mod-http-lua.conf:
load_module modules/ngx_http_lua_module.so;

# configuration file /etc/nginx/modules-enabled/50-mod-http-passenger.conf:
load_module /usr/lib/nginx/modules/ngx_http_passenger_module.so;

# configuration file /etc/nginx/modules-enabled/50-mod-http-perl.conf:
load_module modules/ngx_http_perl_module.so;

# configuration file /etc/nginx/modules-enabled/50-mod-http-subs-filter.conf:
load_module modules/ngx_http_subs_filter_module.so;

# configuration file /etc/nginx/modules-enabled/50-mod-http-uploadprogress.conf:
load_module modules/ngx_http_uploadprogress_module.so;

# configuration file /etc/nginx/modules-enabled/50-mod-http-upstream-fair.conf:
load_module modules/ngx_http_upstream_fair_module.so;

# configuration file /etc/nginx/modules-enabled/50-mod-http-xslt-filter.conf:
load_module modules/ngx_http_xslt_filter_module.so;

# configuration file /etc/nginx/modules-enabled/50-mod-mail.conf:
load_module modules/ngx_mail_module.so;

# configuration file /etc/nginx/modules-enabled/50-mod-nchan.conf:
load_module modules/ngx_nchan_module.so;

# configuration file /etc/nginx/modules-enabled/50-mod-stream.conf:
load_module modules/ngx_stream_module.so;

# configuration file /etc/nginx/mime.types:

types {
    text/html                             html htm shtml;
    text/css                              css;
    text/xml                              xml;
    image/gif                             gif;
    image/jpeg                            jpeg jpg;
    application/javascript                js;
    application/atom+xml                  atom;
    application/rss+xml                   rss;

    text/mathml                           mml;
    text/plain                            txt;
    text/vnd.sun.j2me.app-descriptor      jad;
    text/vnd.wap.wml                      wml;
    text/x-component                      htc;

    image/png                             png;
    image/tiff                            tif tiff;
    image/vnd.wap.wbmp                    wbmp;
    image/x-icon                          ico;
    image/x-jng                           jng;
    image/x-ms-bmp                        bmp;
    image/svg+xml                         svg svgz;
    image/webp                            webp;

    application/font-woff                 woff;
    application/java-archive              jar war ear;
    application/json                      json;
    application/mac-binhex40              hqx;
    application/msword                    doc;
    application/pdf                       pdf;
    application/postscript                ps eps ai;
    application/rtf                       rtf;
    application/vnd.apple.mpegurl         m3u8;
    application/vnd.ms-excel              xls;
    application/vnd.ms-fontobject         eot;
    application/vnd.ms-powerpoint         ppt;
    application/vnd.wap.wmlc              wmlc;
    application/vnd.google-earth.kml+xml  kml;
    application/vnd.google-earth.kmz      kmz;
    application/x-7z-compressed           7z;
    application/x-cocoa                   cco;
    application/x-java-archive-diff       jardiff;
    application/x-java-jnlp-file          jnlp;
    application/x-makeself                run;
    application/x-perl                    pl pm;
    application/x-pilot                   prc pdb;
    application/x-rar-compressed          rar;
    application/x-redhat-package-manager  rpm;
    application/x-sea                     sea;
    application/x-shockwave-flash         swf;
    application/x-stuffit                 sit;
    application/x-tcl                     tcl tk;
    application/x-x509-ca-cert            der pem crt;
    application/x-xpinstall               xpi;
    application/xhtml+xml                 xhtml;
    application/xspf+xml                  xspf;
    application/zip                       zip;

    application/octet-stream              bin exe dll;
    application/octet-stream              deb;
    application/octet-stream              dmg;
    application/octet-stream              iso img;
    application/octet-stream              msi msp msm;

    application/vnd.openxmlformats-officedocument.wordprocessingml.document    docx;
    application/vnd.openxmlformats-officedocument.spreadsheetml.sheet          xlsx;
    application/vnd.openxmlformats-officedocument.presentationml.presentation  pptx;

    audio/midi                            mid midi kar;
    audio/mpeg                            mp3;
    audio/ogg                             ogg;
    audio/x-m4a                           m4a;
    audio/x-realaudio                     ra;

    video/3gpp                            3gpp 3gp;
    video/mp2t                            ts;
    video/mp4                             mp4;
    video/mpeg                            mpeg mpg;
    video/quicktime                       mov;
    video/webm                            webm;
    video/x-flv                           flv;
    video/x-m4v                           m4v;
    video/x-mng                           mng;
    video/x-ms-asf                        asx asf;
    video/x-ms-wmv                        wmv;
    video/x-msvideo                       avi;
}

# configuration file /etc/nginx/conf.d/mod-http-passenger.conf:
### Begin automatically installed Phusion Passenger config snippet ###
passenger_root /usr/lib/ruby/vendor_ruby/phusion_passenger/locations.ini;
passenger_ruby /home/winner/.rbenv/shims/ruby;
### End automatically installed Phusion Passenger config snippet ###

# configuration file /etc/nginx/sites-enabled/estudent.conf:
server {
  listen 80;
  listen [::]:80;

  server_name 10.1.5.4;
  root /home/winner/workspace/erp/public;

  passenger_enabled on;
  passenger_app_env production;

  location /cable {
    passenger_app_group_name estudent_websocket;
    passenger_force_max_concurrent_requests_per_process 0;
  }

  # Allow uploads up to 15MB in size
  client_max_body_size 15M;

  location ~ ^/(assets|packs) {
    expires max;
   gzip_static on;
  }
}

server {
  listen 80;
  listen [::]:80;

  server_name estudent.kmu.edu.et 197.156.85.251;
  root /home/winner/workspace/erp/public;

  passenger_enabled on;
  passenger_app_env production;

  location /cable {
    passenger_app_group_name estudent_websocket;
    passenger_force_max_concurrent_requests_per_process 0;
  }

  # Allow uploads up to 15MB in size
  client_max_body_size 15M;

  location ~ ^/(assets|packs) {
    expires max;
   gzip_static on;
  }

}
1 Like

Mhm. An ip address as server_name?

Works http internal, from that machine?

curl http://estudent.kmu.edu.et/

If yes, it's a firewall / routing problem.

2 Likes

Wireshark dump of curl -LIv estudent.kmu.edu.et:

39	11.240774406	192.168.178.26	197.156.85.251	TCP	74	41996 → 80 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1 TSval=2151445194 TSecr=0 WS=128
40	11.415093858	197.156.85.251	192.168.178.26	TCP	74	80 → 41996 [SYN, ACK] Seq=0 Ack=1 Win=65160 Len=0 MSS=1452 SACK_PERM=1 TSval=17299256 TSecr=2151445194 WS=12841	11.415140697	192.168.178.26	197.156.85.251	TCP	66	41996 → 80 [ACK] Seq=1 Ack=1 Win=64256 Len=0 TSval=2151445368 TSecr=17299256
42	11.415308068	192.168.178.26	197.156.85.251	HTTP	150	HEAD / HTTP/1.1 
43	11.553449903	197.156.85.251	192.168.178.26	TCP	54	80 → 41996 [RST, NS, Reserved] Seq=1 Win=64256 Len=0

After the HTTP request by the client, your server immediately closes the connection with a reset. This is probably due to a misconfiguration of your nginx, or even more probably, that backend I've never heared about, Phusion Passenger.

If you're 200 % sure your nginx logs don't record any error, you should look at the Phusion Passenger logs.

3 Likes

@JuergenAuer yes it works internally with local ip and publicly through domain name. and i want to add ssl to the public domain only.

1 Like

Then it's a firewall- or something else problem.

May be that Phusion Passenger requires some additional configurations to allow external access.

2 Likes

@Osiris, @JuergenAuer Thank you i will check the passenger logs and i will let you know. Thank You!

3 Likes

Hello @JuergenAuer, @Osiris
I tried to look at the Phusion Passenger loges and for some reason certbot tries to restart passenger. is that normal behavior? if not how could i prevent it from happening? these is the log i get.

What i did here:

  • restart nginx with 'sudo service nginx restart'

  • clear nginx error log(passenger also writes to nginx log file)

  • sudo certbot --nginx -d estudent.kmu.edu.et

and this is what i get from the log file.

2020/11/13 08:10:01 [info] 7116#7116: Using 32768KiB of shared memory for nchan in /etc/nginx/nginx.conf:63
2020/11/13 08:10:03 [notice] 7118#7118: signal process started

[ N 2020-11-13 08:10:03.8016 6822/T8 age/Cor/CoreMain.cpp:671 ]: Signal received. Gracefully shutting down... (send signal 2 more time(s) to force shutdown)
[ N 2020-11-13 08:10:03.8017 6822/T1 age/Cor/CoreMain.cpp:1246 ]: Received command to shutdown gracefully. Waiting until all clients have disconnected...
[ N 2020-11-13 08:10:03.8017 6822/T1 age/Cor/CoreMain.cpp:1147 ]: Checking whether to disconnect long-running connections for process 6954, application /home/winner/workspace/erp (production)
[ N 2020-11-13 08:10:03.8018 6822/T1 age/Cor/CoreMain.cpp:1147 ]: Checking whether to disconnect long-running connections for process 6972, application /home/winner/workspace/erp (production)
[ N 2020-11-13 08:10:03.8019 6822/T1 age/Cor/CoreMain.cpp:1147 ]: Checking whether to disconnect long-running connections for process 6990, application /home/winner/workspace/erp (production)
[ N 2020-11-13 08:10:03.8020 6822/T1 age/Cor/CoreMain.cpp:1147 ]: Checking whether to disconnect long-running connections for process 7008, application /home/winner/workspace/erp (production)
[ N 2020-11-13 08:10:03.8020 6822/T1 age/Cor/CoreMain.cpp:1147 ]: Checking whether to disconnect long-running connections for process 7026, application /home/winner/workspace/erp (production)
[ N 2020-11-13 08:10:03.8021 6822/T1 age/Cor/CoreMain.cpp:1147 ]: Checking whether to disconnect long-running connections for process 7045, application /home/winner/workspace/erp (production)
[ N 2020-11-13 08:10:03.8022 6822/T1 age/Cor/CoreMain.cpp:1147 ]: Checking whether to disconnect long-running connections for process 7063, application /home/winner/workspace/erp (production)
[ N 2020-11-13 08:10:03.8022 6822/T1 age/Cor/CoreMain.cpp:1147 ]: Checking whether to disconnect long-running connections for process 7082, application /home/winner/workspace/erp (production)
[ N 2020-11-13 08:10:03.8023 6822/Te Ser/Server.h:902 ]: [ServerThr.4] Freed 0 spare client objects
[ N 2020-11-13 08:10:03.8023 6822/Tc Ser/Server.h:902 ]: [ServerThr.3] Freed 0 spare client objects
[ N 2020-11-13 08:10:03.8023 6822/T8 Ser/Server.h:902 ]: [ServerThr.1] Freed 0 spare client objects
[ N 2020-11-13 08:10:03.8023 6822/Te Ser/Server.h:558 ]: [ServerThr.4] Shutdown finished
[ N 2020-11-13 08:10:03.8023 6822/Ta Ser/Server.h:902 ]: [ServerThr.2] Freed 0 spare client objects
[ N 2020-11-13 08:10:03.8024 6822/Tj Ser/Server.h:902 ]: [ServerThr.6] Freed 0 spare client objects
[ N 2020-11-13 08:10:03.8024 6822/Tc Ser/Server.h:558 ]: [ServerThr.3] Shutdown finished
[ N 2020-11-13 08:10:03.8024 6822/T8 Ser/Server.h:558 ]: [ServerThr.1] Shutdown finished
[ N 2020-11-13 08:10:03.8024 6822/Tj Ser/Server.h:558 ]: [ServerThr.6] Shutdown finished
[ N 2020-11-13 08:10:03.8024 6822/Ta Ser/Server.h:558 ]: [ServerThr.2] Shutdown finished
[ N 2020-11-13 08:10:03.8024 6822/Tl Ser/Server.h:902 ]: [ServerThr.7] Freed 0 spare client objects
[ N 2020-11-13 08:10:03.8024 6822/Tm Ser/Server.h:902 ]: [ServerThr.8] Freed 0 spare client objects
[ N 2020-11-13 08:10:03.8024 6822/Tl Ser/Server.h:558 ]: [ServerThr.7] Shutdown finished
[ N 2020-11-13 08:10:03.8023 6822/Tg Ser/Server.h:902 ]: [ServerThr.5] Freed 0 spare client objects
[ N 2020-11-13 08:10:03.8024 6822/Tm Ser/Server.h:558 ]: [ServerThr.8] Shutdown finished
[ N 2020-11-13 08:10:03.8024 6822/Tg Ser/Server.h:558 ]: [ServerThr.5] Shutdown finished
[ N 2020-11-13 08:10:03.8025 6822/To Ser/Server.h:902 ]: [ApiServer] Freed 0 spare client objects
[ N 2020-11-13 08:10:03.8025 6822/To Ser/Server.h:558 ]: [ApiServer] Shutdown finished
[ N 2020-11-13 08:10:03.8027 6822/T1 age/Cor/CoreMain.cpp:1147 ]: Checking whether to disconnect long-running connections for process 6954, application /home/winner/workspace/erp (production)
[ N 2020-11-13 08:10:03.8027 6822/T1 age/Cor/CoreMain.cpp:1147 ]: Checking whether to disconnect long-running connections for process 6972, application /home/winner/workspace/erp (production)
[ N 2020-11-13 08:10:03.8028 6822/T1 age/Cor/CoreMain.cpp:1147 ]: Checking whether to disconnect long-running connections for process 6990, application /home/winner/workspace/erp (production)
[ N 2020-11-13 08:10:03.8029 6822/T1 age/Cor/CoreMain.cpp:1147 ]: Checking whether to disconnect long-running connections for process 7008, application /home/winner/workspace/erp (production)
[ N 2020-11-13 08:10:03.8029 6822/T1 age/Cor/CoreMain.cpp:1147 ]: Checking whether to disconnect long-running connections for process 7026, application /home/winner/workspace/erp (production)
[ N 2020-11-13 08:10:03.8030 6822/T1 age/Cor/CoreMain.cpp:1147 ]: Checking whether to disconnect long-running connections for process 7045, application /home/winner/workspace/erp (production)
[ N 2020-11-13 08:10:03.8030 6822/T1 age/Cor/CoreMain.cpp:1147 ]: Checking whether to disconnect long-running connections for process 7063, application /home/winner/workspace/erp (production)
[ N 2020-11-13 08:10:03.8031 6822/T1 age/Cor/CoreMain.cpp:1147 ]: Checking whether to disconnect long-running connections for process 7082, application /home/winner/workspace/erp (production)
[ N 2020-11-13 08:10:03.8480 7119/T1 age/Wat/WatchdogMain.cpp:1373 ]: Starting Passenger watchdog...
[ N 2020-11-13 08:10:03.9060 7125/T1 age/Cor/CoreMain.cpp:1340 ]: Starting Passenger core...
[ N 2020-11-13 08:10:03.9063 7125/T1 age/Cor/CoreMain.cpp:256 ]: Passenger core running in multi-application mode.
[ N 2020-11-13 08:10:04.0197 7125/T1 age/Cor/CoreMain.cpp:1015 ]: Passenger core online, PID 7125
[ N 2020-11-13 08:10:04.9645 6822/T1 age/Cor/CoreMain.cpp:1325 ]: Passenger core shutdown finished
[ N 2020-11-13 08:10:06.7262 7125/T5 age/Cor/SecurityUpdateChecker.h:519 ]: Security update check: no update found (next check in 24 hours)
2020/11/13 08:10:09 [notice] 7169#7169: signal process started

[ N 2020-11-13 08:10:09.3306 7125/T7 age/Cor/CoreMain.cpp:671 ]: Signal received. Gracefully shutting down... (send signal 2 more time(s) to force shutdown)
[ N 2020-11-13 08:10:09.3307 7125/T1 age/Cor/CoreMain.cpp:1246 ]: Received command to shutdown gracefully. Waiting until all clients have disconnected...
[ N 2020-11-13 08:10:09.3308 7125/Tf Ser/Server.h:902 ]: [ServerThr.4] Freed 0 spare client objects
[ N 2020-11-13 08:10:09.3308 7125/Th Ser/Server.h:902 ]: [ServerThr.5] Freed 0 spare client objects
[ N 2020-11-13 08:10:09.3308 7125/Tc Ser/Server.h:902 ]: [ServerThr.3] Freed 0 spare client objects
[ N 2020-11-13 08:10:09.3308 7125/Th Ser/Server.h:558 ]: [ServerThr.5] Shutdown finished
[ N 2020-11-13 08:10:09.3308 7125/Tf Ser/Server.h:558 ]: [ServerThr.4] Shutdown finished
[ N 2020-11-13 08:10:09.3308 7125/Tl Ser/Server.h:902 ]: [ServerThr.7] Freed 0 spare client objects
[ N 2020-11-13 08:10:09.3308 7125/Tc Ser/Server.h:558 ]: [ServerThr.3] Shutdown finished
[ N 2020-11-13 08:10:09.3308 7125/Tn Ser/Server.h:902 ]: [ServerThr.8] Freed 0 spare client objects
[ N 2020-11-13 08:10:09.3308 7125/Tj Ser/Server.h:902 ]: [ServerThr.6] Freed 0 spare client objects
[ N 2020-11-13 08:10:09.3308 7125/Tl Ser/Server.h:558 ]: [ServerThr.7] Shutdown finished
[ N 2020-11-13 08:10:09.3308 7125/Tn Ser/Server.h:558 ]: [ServerThr.8] Shutdown finished
[ N 2020-11-13 08:10:09.3308 7125/T7 Ser/Server.h:902 ]: [ServerThr.1] Freed 0 spare client objects
[ N 2020-11-13 08:10:09.3308 7125/Tj Ser/Server.h:558 ]: [ServerThr.6] Shutdown finished
[ N 2020-11-13 08:10:09.3309 7125/T7 Ser/Server.h:558 ]: [ServerThr.1] Shutdown finished
[ N 2020-11-13 08:10:09.3309 7125/Ta Ser/Server.h:902 ]: [ServerThr.2] Freed 0 spare client objects
[ N 2020-11-13 08:10:09.3310 7125/To Ser/Server.h:902 ]: [ApiServer] Freed 0 spare client objects
[ N 2020-11-13 08:10:09.3310 7125/Ta Ser/Server.h:558 ]: [ServerThr.2] Shutdown finished
[ N 2020-11-13 08:10:09.3310 7125/To Ser/Server.h:558 ]: [ApiServer] Shutdown finished
[ N 2020-11-13 08:10:09.3780 7173/T1 age/Wat/WatchdogMain.cpp:1373 ]: Starting Passenger watchdog...
[ N 2020-11-13 08:10:09.4032 7176/T1 age/Cor/CoreMain.cpp:1340 ]: Starting Passenger core...
[ N 2020-11-13 08:10:09.4034 7176/T1 age/Cor/CoreMain.cpp:256 ]: Passenger core running in multi-application mode.
[ N 2020-11-13 08:10:09.4428 7176/T1 age/Cor/CoreMain.cpp:1015 ]: Passenger core online, PID 7176
[ N 2020-11-13 08:10:10.0810 7125/T1 age/Cor/CoreMain.cpp:1325 ]: Passenger core shutdown finished
[ N 2020-11-13 08:10:12.2669 7176/T5 age/Cor/SecurityUpdateChecker.h:519 ]: Security update check: no update found (next check in 24 hours)
2020/11/13 08:10:16 [crit] 6851#6851: *2 connect() to unix:/tmp/passenger.9Yg8XQo/agents.s/core failed (2: No such file or directory) while connecting to upstream, client: 196.191.53.229, server: estudent.kmu.edu.et, request: "GET /estudent/ HTTP/1.1", upstream: "passenger:unix:/tmp/passenger.9Yg8XQo/agents.s/core:", host: "197.156.85.251", referrer: "http://197.156.85.251/estudent"

1 Like

Seeing this log will be useful:

Also seeing the renewal config file may be useful.
Maybe:
/etc/letsencrypt/renewal/estudent.kmu.edu.et.conf

And also seeing if there are any hooks being called:
ls -Rl /etc/letsencrypt/renewal-hooks/

2 Likes

Hello @rg305
letsencrypt.log shows this error

2020-11-13 08:49:07,643:DEBUG:acme.client:Sending POST request to https://acme-v02.api.letsencrypt.org/acme/authz-v3/8568133943:
{
"protected": "eyJhbGciOiAiUlMyNTYiLCAia2lkIjogImh0dHBzOi8vYWNtZS12MDIuYXBpLmxldHNlbmNyeXB0Lm9yZy9hY21lL2FjY3QvMTAyMTMzNjcxIiwgIm5vbmNlIjogIjAwMDRTQlkxbGQwVXczeFJzN29jOXR3aVZrSzFWSmJpdjVMbUJLVVQ5MmV5RjkwIiwgInVybCI6ICJodHRwczovL2FjbWUtdjAyLmFwaS5sZXRzZW5jcnlwdC5vcmcvYWNtZS9hdXRoei12My84NTY4MTMzOTQzIn0",
"signature": "mfkteesdbySRVYDrBM91IXX_DSzAmIeIvvdPL3516DpgDUTkQQ9saVWdVLG87X9qzYZN44GtaPM0l1RQowL30zT2nHFdWskQzvp8aqlDpBQXd9QrLsfIwLmnSsqZVavjLxulCDHQGrrpstXvbT3GgWcGl9M0ZWllaF1FmO0vyIMyWEXCUaM4bO4K9l0rfd7lBFCfmfhE5wXj6mICMgVuCNOyPmOfvACpVYKvRCEYzoNcWujZ5Bglcmao-P1M0JPLaQu5ox87oxNaVUYtL4TVCFO5qC3TCZWPvqWdvcmChORwBTUtwXruQQ_bS0ASM83Kj7INuO0d-jhMI58dADj8Vw",
"payload": ""
}
2020-11-13 08:49:07,915:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 "POST /acme/authz-v3/8568133943 HTTP/1.1" 200 992
2020-11-13 08:49:07,915:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Fri, 13 Nov 2020 05:49:07 GMT
Content-Type: application/json
Content-Length: 992
Connection: keep-alive
Boulder-Requester: 102133671
Cache-Control: public, max-age=0, no-cache
Link: https://acme-v02.api.letsencrypt.org/directory;rel="index"
Replay-Nonce: 0003FtlRZw-sOLkqevK4iI7Upfn-13S5FKSZXfFeTukRbKc
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

{
"identifier": {
"type": "dns",
"value": "estudent.kmu.edu.et"
},
"status": "invalid",
"expires": "2020-11-20T05:49:02Z",
"challenges": [
{
"type": "http-01",
"status": "invalid",
"error": {
"type": "urn:ietf:params:acme:error:connection",
"detail": "Fetching http://estudent.kmu.edu.et/.well-known/acme-challenge/jeJMmx_Cu_3WlEB73Z3Plu_hiTumPUG48Vr4DcY29ho: Connection reset by peer",
"status": 400
},
"url": "https://acme-v02.api.letsencrypt.org/acme/chall-v3/8568133943/QtGYmQ",
"token": "jeJMmx_Cu_3WlEB73Z3Plu_hiTumPUG48Vr4DcY29ho",
"validationRecord": [
{
"url": "http://estudent.kmu.edu.et/.well-known/acme-challenge/jeJMmx_Cu_3WlEB73Z3Plu_hiTumPUG48Vr4DcY29ho",
"hostname": "estudent.kmu.edu.et",
"port": "80",
"addressesResolved": [
"197.156.85.251"
],
"addressUsed": "197.156.85.251"
}
]
}
]
}
2020-11-13 08:49:07,915:DEBUG:acme.client:Storing nonce: 0003FtlRZw-sOLkqevK4iI7Upfn-13S5FKSZXfFeTukRbKc
2020-11-13 08:49:07,916:DEBUG:certbot.reporter:Reporting to user: The following errors were reported by the server:

Domain: estudent.kmu.edu.et
Type: connection
Detail: Fetching http://estudent.kmu.edu.et/.well-known/acme-challenge/jeJMmx_Cu_3WlEB73Z3Plu_hiTumPUG48Vr4DcY29ho: Connection reset by peer

sudo cat /etc/letsencrypt/renewal/estudent.kmu.edu.et.conf
cat: /etc/letsencrypt/renewal/estudent.kmu.edu.et.conf: No such file or directory
i reinstalled certbot and i am not trying to renew!

sudo ls -Rl /etc/letsencrypt/renewal-hooks/
/etc/letsencrypt/renewal-hooks/:
total 12
drwxr-xr-x 2 root root 4096 Nov 12 20:57 deploy
drwxr-xr-x 2 root root 4096 Nov 12 20:57 post
drwxr-xr-x 2 root root 4096 Nov 12 20:57 pre

/etc/letsencrypt/renewal-hooks/deploy:
total 0

/etc/letsencrypt/renewal-hooks/post:
total 0

/etc/letsencrypt/renewal-hooks/pre:
total 0

1 Like

So this would be the first cert for certbot?

2 Likes

since i couldn't renew the certs. i uninstalled and reinstalled certbot and i am trying to get a new cert. and the certs were expired!

1 Like

Please show:
certbot certificates

And also try:

certbot certonly --webroot -w /home/winner/workspace/erp/public -d estudent.kmu.edu.et
2 Likes

Thank You for your quick response.

but i get similar error

Saving debug log to /var/log/letsencrypt/letsencrypt.log


No certs found.


and

$ sudo certbot certonly --webroot -w /home/winner/workspace/erp/public -d estudent.kmu.edu.et
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for estudent.kmu.edu.et
Using the webroot path /home/winner/workspace/erp/public for all unmatched domains.
Waiting for verification...
Cleaning up challenges
Failed authorization procedure. estudent.kmu.edu.et (http-01): urn:ietf:params:acme:error:connection :: The server could not connect to the client to verify the domain :: Fetching http://estudent.kmu.edu.et/.well-known/acme-challenge/TEQgSWmq4YcipY9JF9pNRN2LNutR7c-kzd9yMEtcGnQ: Connection reset by peer

IMPORTANT NOTES:

  • The following errors were reported by the server:

    Domain: estudent.kmu.edu.et
    Type: connection
    Detail: Fetching
    http://estudent.kmu.edu.et/.well-known/acme-challenge/TEQgSWmq4YcipY9JF9pNRN2LNutR7c-kzd9yMEtcGnQ:
    Connection reset by peer

    To fix these errors, please make sure that your domain name was
    entered correctly and the DNS A/AAAA record(s) for that domain
    contain(s) the right IP address. Additionally, please check that
    your computer has a publicly routable IP address and that no
    firewalls are preventing the server from communicating with the
    client. If you're using the webroot plugin, you should also verify
    that you are serving files from the webroot path you provided.

1 Like

I'm seeing the exact same thing:

curl http://estudent.kmu.edu.et/.well-known/acme-challenge/TEQgSWmq4YcipY9JF9pNRN2LNutR7c-kzd9yMEtcGnQ
curl: (56) Recv failure: Connection reset by peer

curl -v http://estudent.kmu.edu.et/
*   Trying 197.156.85.251...
* TCP_NODELAY set
* Connected to estudent.kmu.edu.et (197.156.85.251) port 80 (#0)
> GET / HTTP/1.1
> Host: estudent.kmu.edu.et
> User-Agent: curl/7.58.0
> Accept: */*
>
* Recv failure: Connection reset by peer
* stopped the pause stream!
* Closing connection 0
curl: (56) Recv failure: Connection reset by peer

There is something about your web config that isn't right.

2 Likes

@rg305 that must be it!

but it is ok on myside.

curl -v http://estudent.kmu.edu.et/

  • Trying 197.156.85.251:80...
  • TCP_NODELAY set
  • Connected to estudent.kmu.edu.et (197.156.85.251) port 80 (#0)

GET / HTTP/1.1
Host: estudent.kmu.edu.et
User-Agent: curl/7.68.0
Accept: /

  • Mark bundle as not supporting multiuse
    < HTTP/1.1 301 Moved Permanently
    < Content-Type: text/html
    < Content-Length: 103
    < Connection: keep-alive
    < Status: 301 Moved Permanently
    < Cache-Control: no-cache
    < X-Request-Id: b6cf88b1-ca25-49f3-9592-1bb2ddaa2173
    < Location: http://estudent.kmu.edu.et/auth/login
    < X-Runtime: 0.000950
    < Date: Fri, 13 Nov 2020 06:19:19 GMT
    < X-Powered-By: Phusion Passenger 6.0.6
    < Server: nginx/1.14.0 + Phusion Passenger 6.0.6
    <
  • Connection #0 to host estudent.kmu.edu.et left intact
You are being redirected.
1 Like

Is there a firewall or proxy between your server and the Internet?

2 Likes

@rg305 Yes there is firewall. but both port 80 and 443 are open. what could it be? i don't have access to the firewall

1 Like

Firewall policy prohibiting
GeoLocation blocking
IPS rule
Inline proxy
Honeypot
Fail2BAN

I could go on.
There are many reasons why things can fail.
Before going to network/firewall teams for help, can you review the nginx logs for incoming http connections?
Do you see requests to /.well-known/acme-challenge/... in either log?

	access_log /var/log/nginx/access.log;
	error_log /var/log/nginx/error.log;
2 Likes