Ubuntu 16.04: certbot + nginx config

[lucianp]

Hi,

I want to be able to generate letsencrypt certificates for multiple server/domains. Certbot for Nginx needs a vhost file in /etc/nginx/sites-available/ for each domain to be able to generate it, otherwise it will show the below message:

“Cannot find a VirtualHost matching domain subdomain.domain.com. In order for Certbot to correctly perform the challenge please add a corresponding server_name directive to your nginx configuration: https://nginx.org/en/docs/http/server_names.html”

Is there a way to use Certbot and not create individual files in /etc/nginx/sites-available/ for each domain? I tried to edit the /etc/nginx/sites-available/default with the below schema and got the same issue:

server {
listen 80 default_server;
listen [::]:80 default_server;
server_name example01.com;

}

server {
listen 80 default_server;
listen [::]:80 default_server;
server_name example02.com;

}

server {
listen 80 default_server;
listen [::]:80 default_server;
server_name example03.com;

}

Since I will most likely need to use this config for a lot of domains, I would like to have them stored in the same file. Is that possible?

Thanks!

[rg305]

I think you need to (also) work in the sites-enabled folder.

[schoen]

@lucianp

Since I will most likely need to use this config for a lot of domains, I would like to have them stored in the same file. Is that possible?

Certbot does not directly support this option with --nginx (or -i nginx).

If you're willing to do some of your own scripting, you could use certbot certonly (including certbot certonly -a nginx). This will obtain the certificate (if you use -a nginx, it will make temporary nginx configuration changes in order to do so), but not "install" it by editing your nginx configuration. The certificate will be available at a location in /etc/letsencrypt/live based on the first specified domain name (or alternatively based on a certificate name that you choose with the --cert-name option). Then you can make your own nginx configuration(s) that point to this, instead of using some managed by Certbot.

certbot renew should still work in this case to renew the certificates, although you'll then want to use something like --deploy-hook to reload nginx after the renewal happens.

[lucianp]

Hi guys,

To continue the discussion, I will explain exactly what I’m trying to achive: run 2 docker containers, one nginx and another certbot.
i have a shared volume between the 2 containers that exposes the /etc/letsencrypt/ from certbot and /home/www/letsencrypt from nginx to the same folder on my host.

i want to use a single nginx.conf in the nginx container for an entire list of domains but for that as you said above, i need a bit of scripting. so as far as i understand i need a “clean/standard” nginx server block text for each new added domain to the nginx.conf file, something like the below for running in parallel the certbot container and to generate the ssl certs:

server {
listen 80;
listen [::]:80;
server_name domain.com;
location ^~ /.well-known/acme-challenge/ {
default_type “text/plain”;
root /home/www/letsencrypt;
allow all;
location / {
try_files $uri $uri/ =404;
}

after using the above, i need to include in the nginx.conf a snippet or the lines that are usually generated by cerbot:

listen 443 ssl;
ssl_certificate /home/www/letsencrypt/live/domain.com/fullchain.pem;
ssl_certificate_key /home/www/letsencrypt/live/domain.com/privkey.pem;

and then reload the nginx service.

my docker-compose.yml file looks like this:

version: “3.2”

services:
certbot:
image: certbot/certbot
container_name: certbot
volumes:
- /home/certbot:/etc/letsencrypt
command: certonly --webroot -n -w /etc/letsencrypt -d domain.com --text --agree-tos --email blabla@testing.com --server https://acme-staging.api.letsencrypt.org/directory --rsa-key-size 4096 --verbose
nginx:
image: nginx
container_name: nginx
ports:
- “80:80”
- "443:443"
volumes:
- /home/certbot:/home/www/letsencrypt
- ./nginx/:/etc/nginx/conf.d/

am i missing something? because nginx should be able to serve the files from the shared volume so certbot can pass the challange but i am still getting the below error:

certbot | Connection refused
certbot |
certbot | To fix these errors, please make sure that your domain name was
certbot | entered correctly and the DNS A/AAAA record(s) for that domain
certbot | contain(s) the right IP address. Additionally, please check that
certbot | your computer has a publicly routable IP address and that no
certbot | firewalls are preventing the server from communicating with the
certbot | client. If you’re using the webroot plugin, you should also verify
certbot | that you are serving files from the webroot path you provided.

LE: the issue was that i was trying to generate the ssl certs for a domain but the server i was running the challage on, was hosted on a different domain so it obviously didn’t work…

anyway, i would gladly receive a review for the above solution from your part.

thanks!

[schoen]

Can you connect to the web server from outside of your own network while that container is running? This “Connection refused” error is from the TCP level so it could be caused by a firewall, or by a container configuration problem that results in not actually forwarding the port as you expect.

[jvanasco]

FWIW, I use another technique.

i run certbot as standalone, then proxy all requests to /acme-challenge into it

the following is defined as a macro (include file) and used in the default domain and also included on every site config:

location  /.well-known/acme-challenge  {
    proxy_set_header  X-Real-IP  $remote_addr;
    proxy_set_header  X-Forwarded-For  $proxy_add_x_forwarded_for;
    proxy_set_header  Host  $host;
    proxy_pass  http://127.0.0.1:81;
}

then it’s invoked with
./certbot-auto renew --standalone --preferred-challenge=http-01 --http-01-port=81 --force-renewal

you can run another port other than 81, it was just convenient to me.

when you configure things with a macro/include-file like this, you can quickly swap out the proxy for a static serve and test everything with a ‘hello.txt’ file placed in the directory.

[lucianp]

do you use for each site config a separate file in /etc/nignx/sites-enabled/ or you have a single file /etc/nginx/nginx.conf?

[jvanasco]

Email didn’t post… if this is a duplicate, sorry.

1. Each site has it’s own file in /etc/nginx/sites-available that is symlinked to sites-enabled.

2. There is another directory of “macro” files /etc/nginx/macros. One macro is just that config block above

3. Each site file configuration then does this in one or more server blocks:

   include /etc/nginx/macros/well-known.conf;

nginx evaluates includes like that on startup (like a traditional macro), which ends up defining the same proxy for each site. Boulder can still authenticate to certbot in standalone mode, because the domain is part of the server environment as standard proxy headers. (that may be set elsewhere in the nginx config, I can’t recall)

If we need to change auth methods with letsencrypt (ports, proxy info, validation options etc) I just edit that single include file and restart nginx.

I prefer this to webroot, because I don’t need to make/maintain/test a writable and serve-able directory for each domain. Everything can be done with a single test directory or proxy.

We also deploy with a few other things in that file to disable/enable the Certbot routes as needed, but the above is the core logic. [A rewrite rule triggers requests away from the proxy into a custom error code if a file isn’t on the machine. we also have a few custom openresty hooks to route requests to the right Letsencrypt client as we run 3 on that box]

To get things started, I used a directory to serve the /well-known path, and when that was done I moved to a quick one line python server that just returns that host name for anything in /.well-known/acme-challenge/. A quick test script iterates each domain and ensures we serve it correctly. Every change to nginx is followed by running a custom test suite that includes that script against a host-name proxy It’s only about 15minutes extra work up front, but has prevented so many last-minute emergencies.

[jvanasco]

Each site has it’s own file.

There is another directory of “macro” files. One macro is just that config above,

Each site file then does:

include /etc/nginx/macros/well-known.conf;

That ends up defining the same proxy for each site. Certbot/boulder can still authenticate because the domain is part of the server environment as standard proxy headers.

If LE needs to change auth methods (ports, proxy info, validation options etc) I just edit that single include file and restart nginx.

I prefer this to webroot, because I don’t need to make/maintain/test a writable and sereable directory for each domain. Everything can be done with a single test directory or proxy.

We also deploy with a few other things in that file to disable/enable the Certbot routes as needed.

To get things started, I used a directory to serve the /well-known path, and when that was done I moved to a quick one line python server that just returns that host name for anything in /.well-known/acme-challenge/. Another quick test script iterates each domain and ensures we serve it correctly. Every change to nginx requires running that script.

It’s about 15minutes extra work, but has prevented so many last-minute emergencies.