404 Not found with nginx - Ubuntu 18.04

gertvanoss · November 20, 2018, 4:32pm

hi,
I’ve experimented a lot. Seems all is working. ningx shows the standard page without the certbot-setup. When trying to set this up with Let’s Encrypt all I see is “404 Not Found”.

Note: I’ve tried 2 more servers earlier today. Do not know if that causes the problem.
Thanks,

My domain is:

https://poetreekids.com/

I ran this command:

nginx -T

It produced this output:

root@ubuntu-s-1vcpu-1gb-ams3-01:~# nginx -T
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
# configuration file /etc/nginx/nginx.conf:
user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
	worker_connections 768;
	# multi_accept on;
}

http {

	##
	# Basic Settings
	##

	sendfile on;
	tcp_nopush on;
	tcp_nodelay on;
	keepalive_timeout 65;
	types_hash_max_size 2048;
	# server_tokens off;

	# server_names_hash_bucket_size 64;
	# server_name_in_redirect off;

	include /etc/nginx/mime.types;
	default_type application/octet-stream;

	##
	# SSL Settings
	##

	ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
	ssl_prefer_server_ciphers on;

	##
	# Logging Settings
	##

	access_log /var/log/nginx/access.log;
	error_log /var/log/nginx/error.log;

	##
	# Gzip Settings
	##

	gzip on;

	# gzip_vary on;
	# gzip_proxied any;
	# gzip_comp_level 6;
	# gzip_buffers 16 8k;
	# gzip_http_version 1.1;
	# gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

	##
	# Virtual Host Configs
	##

	include /etc/nginx/conf.d/*.conf;
	include /etc/nginx/sites-enabled/*;
}


#mail {
#	# See sample authentication script at:
#	# http://wiki.nginx.org/ImapAuthenticateWithApachePhpScript
#
#	# auth_http localhost/auth.php;
#	# pop3_capabilities "TOP" "USER";
#	# imap_capabilities "IMAP4rev1" "UIDPLUS";
#
#	server {
#		listen     localhost:110;
#		protocol   pop3;
#		proxy      on;
#	}
#
#	server {
#		listen     localhost:143;
#		protocol   imap;
#		proxy      on;
#	}
#}

# configuration file /etc/nginx/modules-enabled/50-mod-http-geoip.conf:
load_module modules/ngx_http_geoip_module.so;

# configuration file /etc/nginx/modules-enabled/50-mod-http-image-filter.conf:
load_module modules/ngx_http_image_filter_module.so;

# configuration file /etc/nginx/modules-enabled/50-mod-http-xslt-filter.conf:
load_module modules/ngx_http_xslt_filter_module.so;

# configuration file /etc/nginx/modules-enabled/50-mod-mail.conf:
load_module modules/ngx_mail_module.so;

# configuration file /etc/nginx/modules-enabled/50-mod-stream.conf:
load_module modules/ngx_stream_module.so;

# configuration file /etc/nginx/mime.types:

types {
    text/html                             html htm shtml;
    text/css                              css;
    text/xml                              xml;
    image/gif                             gif;
    image/jpeg                            jpeg jpg;
    application/javascript                js;
    application/atom+xml                  atom;
    application/rss+xml                   rss;

    text/mathml                           mml;
    text/plain                            txt;
    text/vnd.sun.j2me.app-descriptor      jad;
    text/vnd.wap.wml                      wml;
    text/x-component                      htc;

    image/png                             png;
    image/tiff                            tif tiff;
    image/vnd.wap.wbmp                    wbmp;
    image/x-icon                          ico;
    image/x-jng                           jng;
    image/x-ms-bmp                        bmp;
    image/svg+xml                         svg svgz;
    image/webp                            webp;

    application/font-woff                 woff;
    application/java-archive              jar war ear;
    application/json                      json;
    application/mac-binhex40              hqx;
    application/msword                    doc;
    application/pdf                       pdf;
    application/postscript                ps eps ai;
    application/rtf                       rtf;
    application/vnd.apple.mpegurl         m3u8;
    application/vnd.ms-excel              xls;
    application/vnd.ms-fontobject         eot;
    application/vnd.ms-powerpoint         ppt;
    application/vnd.wap.wmlc              wmlc;
    application/vnd.google-earth.kml+xml  kml;
    application/vnd.google-earth.kmz      kmz;
    application/x-7z-compressed           7z;
    application/x-cocoa                   cco;
    application/x-java-archive-diff       jardiff;
    application/x-java-jnlp-file          jnlp;
    application/x-makeself                run;
    application/x-perl                    pl pm;
    application/x-pilot                   prc pdb;
    application/x-rar-compressed          rar;
    application/x-redhat-package-manager  rpm;
    application/x-sea                     sea;
    application/x-shockwave-flash         swf;
    application/x-stuffit                 sit;
    application/x-tcl                     tcl tk;
    application/x-x509-ca-cert            der pem crt;
    application/x-xpinstall               xpi;
    application/xhtml+xml                 xhtml;
    application/xspf+xml                  xspf;
    application/zip                       zip;

    application/octet-stream              bin exe dll;
    application/octet-stream              deb;
    application/octet-stream              dmg;
    application/octet-stream              iso img;
    application/octet-stream              msi msp msm;

    application/vnd.openxmlformats-officedocument.wordprocessingml.document    docx;
    application/vnd.openxmlformats-officedocument.spreadsheetml.sheet          xlsx;
    application/vnd.openxmlformats-officedocument.presentationml.presentation  pptx;

    audio/midi                            mid midi kar;
    audio/mpeg                            mp3;
    audio/ogg                             ogg;
    audio/x-m4a                           m4a;
    audio/x-realaudio                     ra;

    video/3gpp                            3gpp 3gp;
    video/mp2t                            ts;
    video/mp4                             mp4;
    video/mpeg                            mpeg mpg;
    video/quicktime                       mov;
    video/webm                            webm;
    video/x-flv                           flv;
    video/x-m4v                           m4v;
    video/x-mng                           mng;
    video/x-ms-asf                        asx asf;
    video/x-ms-wmv                        wmv;
    video/x-msvideo                       avi;
}

# configuration file /etc/nginx/sites-enabled/my_ult_nginx.conf:
server {

server_name www.poetreekids.com poetreekids.com www.poetree.nl poetree.nl www.poetreekids.nl poetreekids.nl;
    return 404; # managed by Certbot
    # server_name www.poetreekids.com poetreekids.com www.poetree.nl poetree.nl www.poetreekids.nl poetreekids.nl;

	location / {
             proxy_set_header Host $http_host;
             proxy_set_header X-Forwarded-Host $http_host;
             proxy_set_header X-Real-IP $remote_addr;
             proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
	     proxy_set_header X-FORWARDED-PROTO https;
             proxy_pass https://localhost:5000;
	}


    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/www.poetreekids.nl/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/www.poetreekids.nl/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot






    #}
    #server {
    if ($host = www.poetreekids.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


    if ($host = www.poetreekids.nl) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


    if ($host = poetreekids.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


    if ($host = poetreekids.nl) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


    if ($host = www.poetree.nl) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


    if ($host = poetree.nl) {
        return 301 https://$host$request_uri;
    } # managed by Certbot







}
server {
    if ($host = www.poetreekids.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


    if ($host = www.poetreekids.nl) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


    if ($host = poetreekids.nl) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


    if ($host = poetreekids.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


    if ($host = www.poetree.nl) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


    if ($host = poetree.nl) {
        return 301 https://$host$request_uri;
    } # managed by Certbot



server_name www.poetreekids.com poetreekids.com www.poetree.nl poetree.nl www.poetreekids.nl poetreekids.nl;
    listen 80;
    return 404; # managed by Certbot












}

# configuration file /etc/letsencrypt/options-ssl-nginx.conf:
# This file contains important security parameters. If you modify this file
# manually, Certbot will be unable to automatically provide future security
# updates. Instead, Certbot will print and log an error message with a path to
# the up-to-date file that you will need to refer to when manually updating
# this file.

ssl_session_cache shared:le_nginx_SSL:1m;
ssl_session_timeout 1440m;

ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;

ssl_ciphers "ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA:ECDHE-ECDSA-DES-CBC3-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA:!DSS";

root@ubuntu-s-1vcpu-1gb-ams3-01:~#
My web server is (include version):
Ubuntu 18.04

The operating system my web server runs on is (include version):
perl-5.28.0

My hosting provider, if applicable, is:

digital ocean

I can login to a root shell on my machine (yes or no, or I don’t know):

yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel):

no

rg305 · November 20, 2018, 6:13pm

Not sure how those 404s got in there, but they definitely need to be removed.
The http (port 80) vhost configs need to be separated from the https (port 443) vhost configs.
ServerNames should be grouped to match the corresponding cert names.

ssl_protocols & ssl_prefer_server_ciphers are defined in multiple places:
in the HTTP section and in file /etc/letsencrypt/options-ssl-nginx.conf

gertvanoss · November 20, 2018, 9:49pm

I’ve cleaned up the config file and got it to work, thanks.

rg305 · November 20, 2018, 9:51pm

So, all good now?
Or do you still have any problem?

mnordhoff · November 20, 2018, 10:35pm

Certbot intentionally sets a catch-all "return 404;" in HTTP virtual hosts that are only used to redirect to HTTPS.

It certainly shouldn't be in your HTTPS virtual host, though.

gertvanoss · November 21, 2018, 9:04am

Not solved entirely.

I started with one domain in my config: domain-1.com
I added www.domain-1.com as server_name in the config and ran certbot --expand -d domain-1.com,www.domain-1.com

this worked

Next domain-2.com and www.domain-2.com were added in the config (in seperate group) and ran certbot --expand -d domain-1.com,www.domain-1.com,domain-2.com,www.domain-2.com

all worked as expected

Finally added domain-3.com and www.domain-3.com to the config. When I then ran the command:

certbot --expand -d domain-1.com,www.domain-1.com,domain-2.com,www.domain-2.com,domain-3.com,www.domain-3.com

the message was:

too many certificates already issued for exact set of domains ....

Asking for certbot certificates doesn’t show domain-3.com?

I tried to many times?

mnordhoff · November 21, 2018, 9:17am

It means you successfully issued 5 certificates for that exact list of names recently.

Searching the Certificate Transparency logs based on the domain name in your first post, I guess they are these 5 certificates, issued yesterday:

There are also a few certificates for 1 or 2 of your domains.

Do you know how they were created or what happened to them?

As the documentation explains, you can bypass the duplicate certificate rate limit by issuing certificates with different combinations of names – for example, two of your domains in one certificate, and your third domain in a second one.

But it would be best to use one of your existing certificates if possible.

gertvanoss · November 21, 2018, 10:30am

thanks for your detailed response.

I’m not fully there yet. Yesterday I indeed did request for some certificates via sudo certbot --nginx. The server via which I did this request is destroyed. Y’day evening I created a new server and also this morning I did the expand commands. That worked. The last attempt for poetree.nl and www.poetree.nl (domain-3) gave no success. Now I’m not sure via what command I should get these two domains to work.

JuergenAuer · November 21, 2018, 12:21pm

Hi @gertvanoss

if you want to test such things, you should really use the test system, not the productive system. The test system has own limits and they are higher.

system · December 21, 2018, 12:21pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
404 not found on certificate Help	12	2308	December 17, 2018
404 on new cert after switching to certbot Help	4	1119	March 2, 2019
Redirect with LetsEncrypt Certbot certificates not working Server	3	2935	October 10, 2018
NGINX server in Docker Help	4	426	April 9, 2024
Let's Encrypt on Ubuntu 16.04 + Nginx Help	9	3028	March 21, 2018

404 Not found with nginx - Ubuntu 18.04

Related topics