Chicken or the Egg initial config for nginx+certbot docker as non root

Metabaron · November 11, 2020, 5:20am

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. https://crt.sh/?q=example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: mydomain.com

I ran this command: certbot run --non-interactive --nginx --agree-tos --no-redirect -m mymail@mydomain.com -d mydomain.com --http-01-port 8000 --https-port 8443

It produced this output:

My web server is (include version): nginx/1.19.4

The operating system my web server runs on is (include version): Alpine Linux v3.12

My hosting provider, if applicable, is:

I can login to a root shell on my machine (yes or no, or I don't know): yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): certbot 1.4.0

Hi all
I'm running Certbot+Nginx in a docker container as non root (port 8000/8443). External ports 80/443 are correctly forwarded and I'm abble to get a Let's encrypt certificate, no issue at this level.

My problem is to configure Nginx and mainly https section, when starting container, before the first certificate is generated: A certificate is required to enable https and to start Nginx, and Nginx is required to perform the Let’s Encrypt validation, and I would like Nginx to be automatically configured by Certbot, WITH ssl_dhparam and the include options-ssl-nginx.conf lines.

There are 2 sections iin my Nginx default.conf, one for port http port 8000, 1 for https 8443. I'll only detail https section, as there is no problem with http section.

What I tried:

If I start with no ssl enable on port 8443:

server {
    server_name mydomain.com;
    listen 8443;
...
 }

=> certbot error : a duplicate listen 0.0.0.0:8443
from log, it tries to add a second "listen 8443 ssl; # managed by Certbot" line

If I start with a dummy initial certificate:

server {
    server_name mydomain.com;
    listen 8443 ssl;
    ssl_certificate /etc/ssl/ssl-cert-snakeoil.pem;
    ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key;
...
 }

=> The dummy certificates lines are replaced by the Let'encrypt certificate, but without the ssl_dhparam and include options-ssl-nginx.conf lines:

server {
    server_name mydomain.com;
    listen 8443 ssl; 
    ssl_certificate /etc/letsencrypt/live/mydomain.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/mydomain.com/privkey.pem; # managed by Certbot
...
}

The solution I found is to start with a dummy listen 8444 without ssl:

server {
    server_name mydomain.com;
    listen 8444 ;
...
}

=> Config is OK, WITH the ssl_dhparam and include options-ssl-nginx.conf lines , but I have to remove the dummy 8444 port and reload Nginx after Certbot reconfiguration:

server {
    server_name mydomain.com;
    listen 8444 ;
    listen 8443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/mydomain.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/mydomain.com/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
...
}

My questions are:

Is it a normal behavior at 1) that Cerbot tries to add a second "listen 8443 ssl; # managed by Certbot" line, instead of just adding "ssl" to the already existing "listen 8443;" line?
Do you have a better solution than using a dummy "listen 8444", to force Nginx reconfiguration WITH ssl_dhparam/include options-ssl-nginx.conf lines ?

Thanks for help.

_az · November 11, 2020, 6:09am

Indeed, the nginx installer won't set these protocol-related options if the SSL virtualhost for the domain already exists.

It will only do so when creating a new SSL virtualhost or upgrading a non-SSL virtualhost to also listen on SSL.

I can't think of an elegant solution for your situation off the top of my head, but I have a question: is it important for you to preconfigure the SSL virtualhost?

If your starting configuration was just:

server {
    server_name mydomain.com;
    listen 8000;
    #  ...
}

then Certbot would provision the port 8443 SSL options fully.

Metabaron · November 11, 2020, 11:24am

Indeed I have to preconfigure the 8443 section because it's not empty, it's used as reverse proxy and forwarded to a private application.
And as 8000 port is opened to the internet for Certbot, I won't forward it to 8443 port, so it's a different configuration.

Metabaron · November 11, 2020, 1:19pm

And what about 1): Should I open a bug report on github , as cerbot duplicates listen directive instead of just adding 'ssl' to the existing one:

listen 8443 ;
listen 8443 ssl; # managed by Certbot

Or is it a normal behaviour?
Maybe "It's not a bug, it's a feature"

_az · November 11, 2020, 8:05pm

I think this might be a case for certonly rather than using the installer, which may not be flexible enough to adapt to this case.

Yes, it shouldn't try do that.

You're starting with nginx configured for HTTP on port 8443, but you're telling Certbot (via CLI argument) that 8443 is the HTTPS port.

I don't think that the nginx installer supports the idea of "upgrading" a port from HTTP to HTTPS. Instead, it assumes (believes you) that 8443 is free to be used for HTTPS.

To improve this, Certbot could first check whether the change would cause a conflict, and refuse to install the certificate.

That said, the installation fails and Certbot reverts the changes anyway, right? The net difference would be in a friendlier error message.

Metabaron · November 11, 2020, 8:52pm

Yes, changes are refused by Nginx and are reverted by Cerbot, it's OK.

I have checked Cerbot code, and it removes previous config before writing new ones. But I don't undertand why it does not also remove the "listen 8443;", as it should. If I request a port reconfiguration, Cerbot can safely assume I'm OK that previous config would be removed first... But I'm not a python dev, so I dont undertand everything

Ok, I'll try a bug report then, it would be a more elegant solution than mine if it's fixed. Tanks for help.

rg305 · November 11, 2020, 10:34pm

Well...
I see two different things there:
"listen 8443 ;" <> "listen 8443;"

Metabaron · November 11, 2020, 11:00pm

As far I know, Nginx is not sensitive to space, neither Cerbot... But I just tried both, without any difference.

rg305 · November 11, 2020, 11:13pm

Then what it is trying to add ("listen 8443 ssl;") is significantly different to what is there ("listen 8443;").
And it won't replace A with B. when A and B can, and often do, coexist.
Read it this way (not technical syntax - only demonstrative):
listen 8443 HTTP;
and also
listen 8443 HTTPS;

Metabaron · November 11, 2020, 11:41pm

Tell me if I'm wrong ,but you can't in any way, set both http AND https on the same IP:port. Either IP ether port must be different, then you can't have 2 listen on the same port, with or without ssl , in the same virtual host.

rg305 · November 11, 2020, 11:43pm

You would think so.
And human logic seems to follow that train of thought.
But these are machines, they don't apply logic.
They do what they are told (they apply instructions).
And if you tell it to do both, some web servers will do both.
It is no more magic then SNI.

rg305 · November 11, 2020, 11:49pm

This:

server {
    listen 8000;
    servername one;
    #do HTTP stuff
}
server {
    listen 8000 ssl;
    servername one;
    #do HTTPS stuff
    #use a cert to do it with
}

Is no different than:

server {
    listen 8000;
    listen 8000 ssl;
    servername one;
    if protocol HTTP {
        #do HTTP stuff
    }
    if protocol HTTPS {
        #do HTTPS stuff
        #use a cert to do it with
    }
}

Edited to better fit your example.
[not actual code - for illustrative purposes only - don't try this at home - no user serviceable parts inside]

rg305 · November 11, 2020, 11:59pm

And to prove this point:
What happens to a server section when the "ssl" is left out?
[and "listen 443 ssl;" is written as "listen 443;"]

Does the server halt, fail, throw a temper tantrum?
Or does it just do HTTP for that servername (over port 443)?
[until you "fix" it]

If so, then the same system is using the same port for HTTP and HTTPS.
[Albeit unintentionally.]

The magic is in the illusion that server blocks are huge solid walls that separate them from each other.
But they are not, they are only there for our benefit (humans).
There is no wall... there is no spoon.

And with that... I need another
Cheers from Miami!
And the answer to the universe is....: 00101010
[as seen by a computer]

Metabaron · November 12, 2020, 12:08am

The problem is that your examples are not syntactically correct. I'm not very familiar with Nginx, but in Apache you would have to specify IP, then there is no way to mix server host declaration, then there no way to mix listen directive when i request a port reconfiguration when I also specify a domain.
The in your example, you can't have 2 server "one", then you can't merge both declaration and certbot can safely remove the listen declaration I want to be reconfigured.

rg305 · November 12, 2020, 12:10am

My examples purposely excluded syntax.
The logic applies to any web server (just add the syntactical correctness, if you want to test it out).
Can you not read "the idea" (in plain English)?

No you don't.
<virtualhost *:80 *:443>
Works in Apache.

My examples did not cover certbot.
It only showed how the server could be configured to do both HTTP and HTTPS in the same block.
If you want a working Apache config for that give me two minutes...

Metabaron · November 12, 2020, 12:33am

On you Apache example, your are juste enabling ssl (and ssl only) on both 10.1.2.3:80 10.1.2.3:443, you are not enabling http AND https on 10.1.2.3:80 10.1.2.3:443

I just tried your nginx example where you try to enable http AND https on the same port:

and Nginx fail:
nginx -c /etc/nginx/nginx.conf -t
2020/11/12 01:20:51 [emerg] 369#369: no "ssl_certificate" is defined for the "listen ... ssl" directive in /etc/nginx/conf.d/default.conf:2
nginx: [emerg] no "ssl_certificate" is defined for the "listen ... ssl" directive in /etc/nginx/conf.d/default.conf:2
nginx: configuration file /etc/nginx/nginx.conf test failed

rg305 · November 12, 2020, 12:45am

I gave no NGINX example.

Metabaron · November 12, 2020, 12:47am

An in you apache config, even if you can access http, the port is not configured as non ssl, it is configured as ssl and there is a forward via rewrite, it's not the same thing.

rg305 · November 12, 2020, 12:49am

In order to rewrite it to HTTPS, it has to hear it as HTTP.
Proof that is works.

rg305 · November 12, 2020, 12:53am

For NGINX, it doesn't like the same port for the exact same name!

But with different names it passes the test:

    server { 
        listen 10.1.2.3:443 ssl http2;
        server_name www.d.com;
        ssl_certificate      /root/.acme.sh/d.com_ecc/fullchain.cer;
        ssl_certificate_key  /root/.acme.sh/d.com_ecc/d.com.key;
        include /etc/nginx/global.settings.HIGH.NO.HSTS;
        error_log  logs/D.ssl.errors.log;
        location / {
            access_log logs/D.ssl.access.log combined_ssl;
            proxy_pass https://d.com/;
        }#location
    }#server

    server {
        listen 10.1.2.3:443;
        server_name d.com;
        include /etc/nginx/global.settings.HIGH.NO.HSTS;
        error_log  logs/D.ssl.errors.log;
        location / {
            access_log logs/D.ssl.access.log combined_ssl;
            proxy_pass https://d.com/;
        }#location
    }#server

nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful