Let's Encrypt on Nginx SSL gateway

[lapsio]

I’m using Nginx acting as hostname based router to VMs hosted by virtualization server. System architecture looks like this:

gateway [single public ip]
v
lots of firewalls, NATs, etc
v
nginx host [multiple domain names]
v
N x user VM [single domain name, private IP]

According to my understanding the best way for me to obtain certificates in such configuration would be to set up certbot on nginx host and use:

sudo certbot certonly --standalone -d user1.domain.com -d user2.domain.com -d ...

and provide appropriate nginx config. Am I right? Because I assume certbot nginx plugin is meant for nginx as web host, not proxy right? My nginx.conf looks like this:

server {
    listen       44080;
    server_name  ogrygorenko.ddns.net;
    server_name_in_redirect off;
    port_in_redirect off;
    proxy_set_header Host $host:80;
    location / {proxy_pass http://10.0.3.2:44380;}
    error_page   500 502 503 504  /50x.html;
    location = /50x.html {root   /usr/share/nginx/html;}
}


server {
    listen       44080;
    server_name  exalex.bestpony.ml;
    server_name_in_redirect off;
    port_in_redirect off;
    proxy_set_header Host $host:80;
    location / {proxy_pass http://10.0.3.2:44380;}
    error_page   500 502 503 504  /50x.html;
    location = /50x.html {root   /usr/share/nginx/html;}
}

server {
    listen       44080;
    server_name  milosz.bestpony.ml;
    server_name_in_redirect off;
    port_in_redirect off;
    proxy_set_header Host $host:80;
    location / {proxy_pass http://10.0.2.2:44280;}
    error_page   500 502 503 504  /50x.html;
    location = /50x.html {root   /usr/share/nginx/html;}
}

...

nginx server is can listen on range 44000-44099 (ports forwarded). Network nodes before nginx translate port 80 and 443 to nginx host port 44080 and 44043.

[Patches]

--standalone is meant for sites that are not running any kind of webserver. It will not work if you have a webserver running on the standard ports.

I believe the easiest method of verification for you is tls-sni-01. Certbot will create a fake hostname with a special certificate in your nginx config, connect to the server at the domain you want the certificate for, request the fake hostname via Server Name Indication, and approve the validation request if it gets back the right certificate. Certbot generates it’s own separate server blocks in your nginx config to do this, so it won’t interfere with your existing proxied hosts.

certbot run --nginx --preferred-challenges tls-sni-01 -d user1.domain.com -d user2.domain.com -d ...

If you don’t want certbot to automatically adjust your nginx configs, you will need to use webroot authentication with certbot certonly and add location blocks to all your proxied hosts so they return files in /.well-known/acme-challenge from a local directory instead. You can point them all to the same directory and pass that to certbot as the webroot for all domains if you want; there is no need for them to be separate.

certbot certonly --webroot -w /var/www/acme-webroot -d user1.domain.com -d user2.domain.com -d ...

[lapsio]

I tried to use tls-sni-01 method but it crashed with following error:

https://pastebin.com/xvFizWWv

I also tried webroot but it crashed as well

I tried connecting to lapsio.bestpony.ml:443 via netcat after starting netcat server on nginx gateway and it works I can connect over 443 to netcat server listening on 44043 on nginx gateway.

With nginx reason seems to be quite obvious: nginx gateway does NOT listen on 443 port. Nor 80 and that’s what cerbot is adding to nginx.conf (they’re blocked on firewall before gateway) it’s listening on 44080 and 44043. Only ports 44000-44099 are available to nginx gateway. Other ports are forwarded to different servers in network.

Additionally public 443 is NATed to nginx machine 44043 and public 80 is NATed to nginx machine 44080.

I’ll try:
certbot certonly --standalone --tls-sni-01-port 44043 --http-01-port 44080 -d ...

but for now I got blocked from too many failed requests…

[Patches]

If you cannot modify the configuration for either the servers listening on port 80 or 443, then you will be unable to use http-01/webroot or tls-sni-01 authentication with Let’s Encrypt. For the security of your domain, the CA/Browser Forum requires CAs to perform validation on well-known ports such as those.

Your only choice if that’s the case is to use dns-01 authentication. This requires adding a TXT record to your DNS either manually or via an API. While certbot has some support for this, the dehydrated and acme.sh clients have support for a larger number of DNS providers.

[lapsio]

No okay command i gave in the end worked

(certbot certonly --standalone --tls-sni-01-port 44043 --http-01-port 44080 -d ...).

I had to shut down nginx for a while though. Probably i didn’t eplain it in clear way sorry my bad:

Nothing listens directly on 80 or 443. Router NATs 80 and 443 to 44080 and 44043 to nginx.IP. So full address translation chain looks like this:

*.bestpony.ml:443 -> (router) -> 192.168.2.4:44043 -> (virtualization host) -> 10.0.0.2:44043 -> (nginx ssl proxy) -> 10.0.X.2:44X80 -> (VMs http server)

Where X is VM number corresponding to certain domain name. eg. lapsio.bestpony.ml is listening on 10.0.1.2:44180 as http and milosz.bestpony.ml is listening on 10.0.2.2:44280. 10.0.0.0/16 is virtualized network containing all VMs. Technically nginx host is just one of VMs that’s why it listens on 44043/44080 ports to fall under standard VMs port forward policy (ports 44000-46999 are forwarded from WAN directly to VMs and these are the only ports VMs can use for outside world communication)

Is it possible to alter port the same way for nginx certbot plugin? It’s not like reloading shutting down nginx for few seconds once a month is a big deal but welp. It’d be “nice to have”

[Patches]

You can file a feature request:

certbot/certbot

Be sure and explain the port forwarding use case clearly so you don’t get the same initial response I gave you.

You should be able to achieve http-01 authentication with this configuration, though apparently​ more manual configuration is needed.