Load balancing with Nginx

philipfwilson · April 30, 2017, 3:13pm

I have a nginx image load balanced setup. Lets encrypt works great on my basic site but having issues with the load balanced part. Any help would be great.

I ran the steps below. Then I copied my pem files to the server 1 and server 2. Does not seem to be working

Please fill out the fields below so we can help you better.

My domain is: pics.example.com

nginx load balancer
nginx server 1
nginx server 2
…etc

================ start ==========================

This is my load balance round robin server

philipfwilson@picslb:/etc/nginx/conf.d$ cat ld-balancer.com

upstream picsinfo {
server 192.168.00.123:1234;
server 192.168.00.124:1234;
server 192.168.00.115:1234;
}

server {
listen 443 ssl;
server_name pics.example.com;

    proxy_ssl_certificate            /etc/letsencrypt/live/pics.example.com/fullchain.pem;
    proxy_ssl_certificate_key        /etc/letsencrypt/live/pics.example.com/privkey.pem;
    proxy_ssl_trusted_certificate    /etc/letsencrypt/live/pics.example.com/chain.pem;


    ssl_certificate            /etc/letsencrypt/live/pics.example.com/fullchain.pem;
    ssl_certificate_key        /etc/letsencrypt/live/pics.example.com/privkey.pem;
    ssl_trusted_certificate    /etc/letsencrypt/live/pics.example.com/chain.pem;

    proxy_ssl_verify        on;
    proxy_ssl_verify_depth  2;
    proxy_ssl_session_reuse on;


   location / {
      proxy_pass http://picsinfo;
    }

}
================ end lb round robin setup ==========================

================ start server setup ==========================
server {
listen 443 ssl;
server_name default_server;

    proxy_ssl_certificate            /etc/letsencrypt/live/pics.example.com/fullchain.pem;
    proxy_ssl_certificate_key        /etc/letsencrypt/live/pics.example.com/privkey.pem;
    proxy_ssl_trusted_certificate    /etc/letsencrypt/live/pics.example.com/chain.pem;

    ssl_certificate            /etc/letsencrypt/live/pics.example.com/fullchain.pem;
    ssl_certificate_key        /etc/letsencrypt/live/pics.example.com/privkey.pem;
    ssl_trusted_certificate    /etc/letsencrypt/live/pics.example.com/chain.pem;

    proxy_ssl_verify        on;
    proxy_ssl_verify_depth  2;
    proxy_ssl_session_reuse on;

    root /var/www/pics_images;

			# Add index.php to the list if you are using PHP
      index index.html index.htm index.nginx-debian.html;

      server_name _;

    location /pics_images/ {
    }

			location / {
				# First attempt to serve request as file, then
				# as directory, then fall back to displaying a 404.
				try_files $uri $uri/ =404;
			}

}

================ end server setup ==========================

==========================
I ran this command:
$ sudo add-apt-repository ppa:certbot/certbot
$ sudo apt-get update
$ sudo apt-get install certbot
sudo certbot certonly --standalone -d pics.example.com

It produced this output:

phil@picsloader:/etc/nginx$ sudo certbot certonly --standalone -d pics.example.com

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Enter email address (used for urgent renewal and security notices) (Enter ‘c’ to
cancel):philipfwilson@emailaddress.com
Starting new HTTPS connection (1): acme-v01.api.letsencrypt.org

Please read the Terms of Service at
https://letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf. You must agree
in order to register with the ACME server at
https://acme-v01.api.letsencrypt.org/directory

(A)gree/©ancel: A

Would you be willing to share your email address with the Electronic Frontier
Foundation, a founding partner of the Let’s Encrypt project and the non-profit
organization that develops Certbot? We’d like to send you email about EFF and
our work to encrypt the web, protect its users and defend digital rights.

(Y)es/(N)o: N
Obtaining a new certificate
Performing the following challenges:
tls-sni-01 challenge for pics.example.com
Waiting for verification…
Cleaning up challenges
Generating key (2048 bits): /etc/letsencrypt/keys/1234_key-certbot.pem
Creating CSR: /etc/letsencrypt/csr/0000_csr-certbot.pem

IMPORTANT NOTES:

Congratulations! Your certificate and chain have been saved at
/etc/letsencrypt/live/pics.example.com/fullchain.pem. Your cert
will expire on 2017-07-27. To obtain a new or tweaked version of
this certificate in the future, simply run certbot again. To
non-interactively renew all of your certificates, run “certbot
renew”
If you lose your account credentials, you can recover through
e-mails sent to philipfwilson@yahoo.com.
Your account credentials have been saved in your Certbot
configuration directory at /etc/letsencrypt. You should make a
secure backup of this folder now. This configuration directory will
also contain certificates and private keys obtained by Certbot so
making regular backups of this folder is ideal.
If you like Certbot, please consider supporting our work by:

Donating to ISRG / Let’s Encrypt: https://letsencrypt.org/donate
Donating to EFF: https://eff.org/donate-le

My operating system is (include version):

ubuntu 16.04

My web server is (include version):

nginx

My hosting provider, if applicable, is:

I can login to a root shell on my machine (yes or no, or I don’t know):

yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel):

no

ahaw021 · April 30, 2017, 11:43pm

I have a nginx image load balanced setup. Lets encrypt works great on my basic site but having issues with the load balanced part. Any help would be great.

what exactly are you having trouble with

looks like you obtained a certificate correctly?

Also it would be good to provide the real domain name so people can test etc.

Andrei

philipfwilson · May 1, 2017, 12:10am

==================

My question is how to setup this up correctly with lets encrypt and Nginx load balance servers. I think they have to share the same pem files.

I have tried to add A records for my servers behind Nginx. Since they are sharing the same IP it does not work.

Also, I tried to install lets encrypt on each node server behind Nginx and that does not work either. Lets encrypt says this is not a valid ip since it already has one for the LB server.

How do I set this up correct?

Thanks Phil

========

ahaw021 · May 1, 2017, 12:52am

thanks

I think the challenge you are running in to is that IP adresses cannot be used as SAN entries in public CA issued certificates

There are several ways of fixing this

A) I believe you should be able to use self signed certs for backed connections
B) My preferred method -
Create entries for each of your backend servers in your DNS that are FQDN
E.g. node1.pics.example.com node2.pics.example.com
Create a back-end cert with those hostnames in the SAN
Install this cert on your backend servers
Update your DNS records (internal) to resolve the DNS entries above
Update your NGINX config to do the same

Andrei

philipfwilson · May 1, 2017, 3:25am

ok. I did a simple test on one of the node image servers.

phil@pic1:/etc/nginx/sites-enabled$ curl Https://192.168.1.234/3/testImage.jpg
curl: (51) SSL: certificate subject name (pics.example.com) does not match target host name '192.168.1.234'

philipfwilson · May 1, 2017, 3:27am

This is example of what I have on one of the image node servers.

error from ssl cert here?? what is going on?

phil@pic1:/etc/nginx/sites-enabled$

curl Https://192.168.1.234/3/testImage.jpg

curl: (51) SSL: certificate subject name (pics.example.com) does not match target host name ‘192.168.1.234’

server {
        listen 80;
        server_name pics.example.com ;
        rewrite  ^  https://$server_name$request_uri? permanent;
}



server {
    listen 443 ssl ;
    server_name pics.example.com ;

    ssl_certificate            /etc/letsencrypt/live/pics.example.com/fullchain.pem;
    ssl_certificate_key        /etc/letsencrypt/live/pics.example.com/privkey.pem;
    ssl_trusted_certificate    /etc/letsencrypt/live/pics.example.com/chain.pem;

    proxy_ssl_verify        on;
    proxy_ssl_verify_depth  2;
    proxy_ssl_session_reuse on;

    root /var/www/example_images;

    location /example_images/ {
    }

  location / {
  # First attempt to serve request as file, then
  # as directory, then fall back to displaying a 404.
  try_files $uri $uri/ =404;
  }
 }

The_Decryptor · May 1, 2017, 5:55am

You’re connecting to “192.168.1.234”, but the server is presenting a certificate for “pics.example.com” which doesn’t match. You can override the hostname curl uses with the -H "Host: pics.example.com" setting, but that won’t help with nginx.

I’m not sure nginx has a setting like that, so you’ll need a proper hostname in your internal DNS for these servers (Which you can then add to your public TLS certificate, even if the systems aren’t visible from the internet), or use a self-signed certificate just for these systems (I think that’s what the proxy_ssl_... settings are for), or just not use TLS for these internal connections.

Edit: Or set proxy_ssl_verify to off, not sure how I missed that.

ahaw021 · May 2, 2017, 9:41am

just be careful that the backend servers are listening of 443 and have a certificate (even if a self signed one) when using proxy_ssl_verify to off switch

Andrei

system · June 1, 2017, 9:51am

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.

Load balancing with Nginx

Please read the Terms of Service at https://letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf. You must agree in order to register with the ACME server at https://acme-v01.api.letsencrypt.org/directory

Please read the Terms of Service at
https://letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf. You must agree
in order to register with the ACME server at
https://acme-v01.api.letsencrypt.org/directory