Unauthorized - 404 Not Found (Jail, Jenkins, Nginx)

Help
1

My domain is:

https://jenkins.balliu.tech

I ran this command:

certbot --standalone certonly --cert-name balliu.tech -d balliu.tech -d git.balliu.tech -d jenkins.balliu.tech

It produced this output:

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator standalone, Installer None

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
You are updating certificate balliu.tech to include new domain(s):
+ jenkins.balliu.tech

You are also removing previously included domain(s):
(None)

Did you intend to make this change?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(U)pdate cert/(C)ancel: U
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for balliu.tech
http-01 challenge for git.balliu.tech
http-01 challenge for jenkins.balliu.tech
Waiting for verification...
Cleaning up challenges
Failed authorization procedure. jenkins.balliu.tech (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://jenkins.balliu.tech/.well-known/acme-challenge/EpI9UC4ZKYJH8HITvFcYuoMO3AF49ePe-lkgO0pZnG8: "<html>\r\n<head><title>404 Not Found</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>404 Not Found</h1></center>\r\n<hr><center>"

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: jenkins.balliu.tech
   Type:   unauthorized
   Detail: Invalid response from
   http://jenkins.balliu.tech/.well-known/acme-challenge/EpI9UC4ZKYJH8HITvFcYuoMO3AF49ePe-lkgO0pZnG8:
   "<html>\r\n<head><title>404 Not Found</title></head>\r\n<body
   bgcolor=\"white\">\r\n<center><h1>404 Not
   Found</h1></center>\r\n<hr><center>"

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address.

My web server is (include version):

nginx version: nginx/1.14.0

The operating system my web server runs on is (include version):

FreeNAS 11.2 jail (FreeBSD 11.2)

My hosting provider, if applicable, is:

Home server

I can login to a root shell on my machine (yes or no, or I don’t know):

Yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel):

No

Nginx Config:

#user  nobody;
worker_processes  1;

# This default error log path is compiled-in to make sure configuration parsing
# errors are logged somewhere, especially during unattended boot when stderr
# isn't normally logged anywhere. This path will be touched on every nginx
# start regardless of error log location configured here. See
# https://trac.nginx.org/nginx/ticket/147 for more info.
#
error_log  /var/log/nginx/error.log;
#

#pid        logs/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  logs/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    #keepalive_timeout  0;
    keepalive_timeout  65;

	#gzip  on;

	server {
		listen 80 default_server;
		server_name _;
                return 301 https://$host$request_uri;
        }

        server {
                listen 443 ssl;
                server_name git.balliu.tech;
                include ssl_common.conf;
                location / {
                        proxy_pass http://192.168.1.199:3000;
			include basic_proxy_directives.conf;
		}
	}

#	server {
#		listen 443 ssl;
#		server_name jenkins.balliu.tech;
#		include ssl_common.conf;
#		location / {
#			proxy_pass http://192.168.1.205:8180;
#			include basic_proxy_directives.conf;
#			proxy_redirect http://192.168.1.205:8180 https://jenkins.balliu.tech;
#
#		}
#	}

	server {
 		listen 443 ssl;
		server_name jenkins.balliu.tech;
		include ssl_common.conf;
		location / {
			proxy_set_header        Host $host:$server_port;
			proxy_set_header        X-Real-IP $remote_addr;
			proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
			proxy_set_header        X-Forwarded-Proto $scheme;
			proxy_redirect http://192.168.1.205:8180 https://jenkins.balliu.tech;
			proxy_pass              http://192.168.1.205:8180;
			# Required for new HTTP-based CLI
			proxy_http_version 1.1;
			proxy_request_buffering off;
			proxy_buffering off; # Required for HTTP-based CLI to work over SSL
			# workaround for https://issues.jenkins-ci.org/browse/JENKINS-45651
		    	add_header 'X-SSH-Endpoint' 'jenkins.balliu.tech:50022' always;
		}
	}
}

ssl_common.conf (referenced in nginx.conf):

# Thanks to https://cipherli.st/ for providing a great reference! Please check out their site
# to make sure your SSL Configuration is up to date with current standards! Be aware that in this
# example we use a slightly liberal cipherlist to allow for older browsers on older devices, Eg.
# IE8, android 2.4, etc
# Enable Perfect Forward Secrecy (PFS)
ssl_prefer_server_ciphers on;
ssl_certificate /usr/local/etc/letsencrypt/live/balliu.tech/fullchain.pem;
ssl_certificate_key /usr/local/etc/letsencrypt/live/balliu.tech/privkey.pem;
# Disable SSLv2 and SSLv3 (BEAST and POODLE attacks)
ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
# Enable our strong DH Key
ssl_dhparam /usr/local/etc/ssl/dhparams.pem;
# Cipher-list for PFS.
ssl_ciphers
"EECDH+AESGCM:EDH+AESGCM:ECDHE-RSA-AES128-GCM-SHA256:AES256+EECDH:DHE-RSA-AES128-GCM-SHA256:AES256+EDH:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4";
ssl_ecdh_curve secp384r1;
# Requires nginx >= 1.1.0
ssl_session_cache shared:SSL:10m;
ssl_session_tickets off;
# Requires nginx >= 1.5.9
ssl_stapling on;
# Requires nginx >= 1.3.7
ssl_stapling_verify on;
# Requires nginx => 1.3.7
resolver 8.8.8.8 4.4.4.4 valid=300s;
resolver_timeout 5s;
# HSTS Support
add_header Strict-Transport-Security "max-age=63072000;includeSubdomains; preload";
# These headers can break applications, be careful!
add_header X-Frame-Options DENY;
add_header X-Content-Type-Options nosniff;

I have followed the guide to get ssl on Jenkins inside a jail in Jenkins official wiki:
https://wiki.jenkins.io/display/JENKINS/Running+Jenkins+behind+Nginx

For Nginx:

I have tried everything and cannot get it to work. I feel like I am missing something.

I set up Gitea on https://git.balliu.tech super fast, however Jenkins is being troublesome.

Something I suspect to be the issue is the webroot not being in the same jail and Nginx not having permissions to it. But Gitea is in a different jail too and that worked

If someone could guide me I would greatly appreciate it. Even ideas are fine. At this point I just need another pair of eyes on this stuff.

Thank you!

2

I think you have to handle the authentication requests locally (at wherever certbot intersects with http://jenkins.balliu.tech/) and not proxy them to another system.

3

You mean in the same jail where Jenkins is installed have an instance of nginx running? If that’s the case then that defeats the point of a singular reverse proxy. If this is common then I’d have to do this for every other app out there leading to several nginx instances I have to deal with.

Otherwise I am not sure I follow.

I have tried to make a shared folder which mounts on both jails.
Like this:
Jenkins jail (webroot folder) -> shared folder <- Nginx jail (/jenkins-webroot)

Then I would set root to /jenkins-webroot in nginx.conf for ./well-known route. That has led to some FreeNAS related issues that I want to circumvent for now.

The above solution seems unnecessary due to the fact that I have an almost exact situation that’s already working like I mentioned (Gitea jail). I want to understand where the problem lies in Jenkins’ case.

4

No, I don't mean to move the nginx.
If certbot runs where nginx already lives, then I mean to configure the nginx to handle the authentication requests there - not to forward them.
If certbot and nginx are not in the same place then I need to better understand the big picture.

5

You are correct, nginx and certbot are in the same jail.

I see what you mean… So have a location directive matching .well-known and its root being somewhere in the nginx jail? That way it doesn’t have to deal with Jetty on the other jail (Jenkins’ servlet engine/http server, which I have considered to be the problem)

Ill check it out when I get home.

6

Exactly. That should do the "trick" nicely.

7

Well… After playing around a little bit I got it working. This whole time I had forgotten to stop my nginx when using --standalone. Once I stopped my nginx and let certbot create its own webserver this command just worked:

certbot --standalone certonly --cert-name balliu.tech -d balliu.tech -d git.balliu.tech -d jenkins.balliu.tech

Now I feel like an idiot.

Thanks for the help!

8

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.