How can I run certbot behind nginx proxy [within Docker]

Help
1

I have the same issue. As @MikeMcQ stated port 80 must be opened so Let's Encrypt may reach that but where does this port have to be redirected to?
Currently there is a nginx listening on port 80 that is implemented as a reverse proxy. Nginx is running in a docker container.
Now I try to start a certbot using this command:

root@server-alterlaptop:~/nginx/conf# docker run --name temp_certbot -v /data/certbot/letsencrypt:/etc/letsencrypt -v /data/certbot/www:/tmp/letsencrypt -p 80:80 -p 443:443 -v /data/servers-data/certbot/log:/var/log certbot/certbot:latest  certonly --webroot --agree-tos --renew-by-default --preferred-challenges http-01 --server https://acme-v02.api.letsencrypt.org/directory --text --email mymail@gmail.com -w /tmp/letsencrypt -d schuldenuhren.net

As there is a nginx running on port 80 the container obviously could not start but when I take the nginx down letsdebug shows me, that a request to port 80 does not work.

I tried several tutorials but none of them helped me get this thing running.

This is my error message:

requests.exceptions.ConnectionError: HTTPSConnectionPool(host='acme-staging-v02.api.letsencrypt.org', port=443): Max retries exceeded with url: /directory (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f952b2ca760>: Failed to establish a new connection: [Errno -3] Try again'))

And this is my docker-compose.yml:

version: '3'
services:
  nginx:
    image: nginx:latest
    container_name: nginx
    restart: unless-stopped
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - /root/nginx/nginx.conf:/etc/nginx/nginx.conf:ro
      - /root/nginx/conf:/etc/nginx/conf.d:ro
      - /root/nginx/logs:/var/log/nginx
      - /root/certbot/www:/var/www/certbot/:ro
  certbot:
    image: certbot/certbot:latest
    container_name: certbot
    volumes:
      - /root/certbot/www/:/var/www/certbot/:rw

Edit

I have added the certbot log file:

2022-09-30 10:57:56,059:DEBUG:certbot._internal.main:certbot version: 1.30.0
2022-09-30 10:57:56,060:DEBUG:certbot._internal.main:Location of certbot entry point: /usr/local/bin/certbot
2022-09-30 10:57:56,060:DEBUG:certbot._internal.main:Arguments: ['--keep', '--webroot', '--webroot-path', '/var/www/certbot', '--email', 'kolloch.tim@web.de', '--agree-tos', '-d', 'schuldenuhren.net']
2022-09-30 10:57:56,060:DEBUG:certbot._internal.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#manual,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
2022-09-30 10:57:56,077:DEBUG:certbot._internal.log:Root logging level set at 30
2022-09-30 10:57:56,078:DEBUG:certbot._internal.plugins.selection:Requested authenticator webroot and installer None
2022-09-30 10:57:56,081:DEBUG:certbot._internal.plugins.selection:Single candidate plugin: * webroot
Description: Place files in webroot directory
Interfaces: Authenticator, Plugin
Entry point: webroot = certbot._internal.plugins.webroot:Authenticator
Initialized: <certbot._internal.plugins.webroot.Authenticator object at 0x7f37b7947970>
Prep: True
2022-09-30 10:57:56,082:DEBUG:certbot._internal.plugins.selection:Selected authenticator <certbot._internal.plugins.webroot.Authenticator object at 0x7f37b7947970> and installer None
2022-09-30 10:57:56,082:INFO:certbot._internal.plugins.selection:Plugins selected: Authenticator webroot, Installer None
2022-09-30 10:57:56,489:DEBUG:acme.client:Sending GET request to https://acme-v02.api.letsencrypt.org/directory.
2022-09-30 10:57:56,491:DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): acme-v02.api.letsencrypt.org:443
2022-09-30 10:58:01,498:DEBUG:certbot._internal.log:Exiting abnormally:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 174, in _new_conn
    conn = connection.create_connection(
  File "/usr/local/lib/python3.8/site-packages/urllib3/util/connection.py", line 72, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
  File "/usr/local/lib/python3.8/socket.py", line 918, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -3] Try again

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 386, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 1042, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 358, in connect
    self.sock = conn = self._new_conn()
  File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 186, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f37b78806d0>: Failed to establish a new connection: [Errno -3] Try again

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 489, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 787, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='acme-v02.api.letsencrypt.org', port=443): Max retries exceeded with url: /directory (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f37b78806d0>: Failed to establish a new connection: [Errno -3] Try again'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/certbot", line 33, in <module>
    sys.exit(load_entry_point('certbot', 'console_scripts', 'certbot')())
  File "/opt/certbot/src/certbot/certbot/main.py", line 19, in main
    return internal_main.main(cli_args)
  File "/opt/certbot/src/certbot/certbot/_internal/main.py", line 1744, in main
    return config.func(config, plugins)
  File "/opt/certbot/src/certbot/certbot/_internal/main.py", line 1573, in certonly
    le_client = _init_le_client(config, auth, installer)
  File "/opt/certbot/src/certbot/certbot/_internal/main.py", line 827, in _init_le_client
    acc, acme = _determine_account(config)
  File "/opt/certbot/src/certbot/certbot/_internal/main.py", line 735, in _determine_account
    acc, acme = client.register(
  File "/opt/certbot/src/certbot/certbot/_internal/client.py", line 216, in register
    acme = acme_from_config_key(config, key)
  File "/opt/certbot/src/certbot/certbot/_internal/client.py", line 76, in acme_from_config_key
    client = acme_client.BackwardsCompatibleClientV2(net, key, config.server)
  File "/opt/certbot/src/acme/acme/client.py", line 891, in __init__
    directory = messages.Directory.from_json(net.get(server).json())
  File "/opt/certbot/src/acme/acme/client.py", line 1256, in get
    self._send_request('GET', url, **kwargs), content_type=content_type)
  File "/opt/certbot/src/acme/acme/client.py", line 1194, in _send_request
    response = self.session.request(method, url, *args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 565, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='acme-v02.api.letsencrypt.org', port=443): Max retries exceeded with url: /directory (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f37b78806d0>: Failed to establish a new connection: [Errno -3] Try again'))
2022-09-30 10:58:01,502:ERROR:certbot._internal.log:An unexpected error occurred:
2022-09-30 10:58:01,502:ERROR:certbot._internal.log:requests.exceptions.ConnectionError: HTTPSConnectionPool(host='acme-v02.api.letsencrypt.org', port=443): Max retries exceeded with url: /directory (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f37b78806d0>: Failed to establish a new connection: [Errno -3] Try again'))
2

Your topic was moved to its own thread. We like each problem to be its own.

Yes, port 80 must be open if you are using the HTTP Challenge (as compared to a DNS Challenge).

I know there are many ways to configure docker containers. I think other volunteers here are better able to suggest docker setup.

3 Likes
3

Thank you!

Hope someone can help me.

1 Like
4

Using docker is like using multiple servers / VMs.

The edge routing system can only send HTTP (port 80) requests to one IP/system.
Which system handles those HTTP (port 80) request has to handle the ACME challenge requests OR proxy them to the system that handles that FQDN.

4 Likes
5

That's what I figured too so I looked into the tutorials and altered my docker compose.
The tutorials said that I should create a shared volume between certbot and my nginx so they can share the challenges folder.
That's why I added following volume

/root/certbot/www/:/var/www/certbot/:rw

to both containers.

Furthermore I added this

location /.well-known/acme-challenge/ {
    root /var/www/certbot;
}

to my nginx-config file so requests made by let's encrypt to those challenges should be re-routed to the directory where certbot places its challenges.

However Let's Debug does not report any problems for my domain which (as far as I understood it) should mean that requests to /.well-known/acme-challenge/ should go through or am I wrong there?

I am pretty new to the topic so I'd love to learn where I am wrong.

Thank you in advance!

1 Like
6

You can use nginx to reverse proxy certbot:

location /.well-known/acme-challenge/ {
    proxy_pass http://certbot; # requires certbot certonly --standalone
}

But I see you don't have a volume to save your certificates and keys, that's recipe for exhausting your rate limits pretty fast.

Also,

  1. nginx doesn't have any idea when certbot has renewed a certificate, and does not know when to reload. You can make certbot do that but it's a mess, because the image is pretty bare and you'd have to find out how to send nginx a signal by mounting the docker socket in the certbot container.
  2. certbot doesn't know when to run, there is no cron service to tell it to run.
4 Likes
7

Where does certbot store the certificates? Would it work if I set up another shared volume like /root/certbot/www:/var/www/certbot/:ro to share the certificates between both containers or is there more I have to pay attention to?

What would be the difference between the configuration I use at the moment:

location /.well-known/acme-challenge/ {
    root /var/www/certbot;
}

and the configuration you suggested:

location /.well-known/acme-challenge/ {
    proxy_pass http://certbot; # requires certbot certonly --standalone
}

?

I know that there is no triggering for renewing/reloading certificates. I would like to understand the basics and get a basic service running before I try to implement automatic renewing. But if it is easier to do everything at the same time I am free to try that.

8

I'm not sure about the paths with the docker image, but it's usually /etc/letsencrypt

4 Likes
9

Okay, I added this path to my docker compose file. Should prevent future errors but the main error sadly persists.
I still get a Max retries exceeded with url: /directory error.
Or is this the error stating that I exhausted my rate limits?
If I exhausted my rate limits shouldn't --dry-run still work?

Thank you anyway!

1 Like
10

That sounds more like a "slow down" response than a "you're ratelimited" one.

5 Likes
11

Okay got that, thanks!

1 Like
12

This is a python error message that I believe indicates it failed to be able to talk to Let's Encrypt at all. Not an error message from Let's Encrypt's API. You'll have to make sure that certbot can talk to the internet.

5 Likes
13

Okay, I did not know that, thank you.
Do you have an idea how I could test that?
If I start a container with the certbot image it directly shuts down again because of stated error.

1 Like
14

what do you see before the error? can it be a DNS problem, or is it an IP problem?

try adding a --dns 1.1.1.1 option to your docker command.

4 Likes
15

If it doesn't have to be certbot, take a look at:nginxproxy/acme-companion. Adds automatic Let's Encrypt certificates to the Nginx proxy.

16

It definitely does not have to be certbot. Will try if I have some more time, thank you!

1 Like
17

It seems to have been an DNS issue. As I added the DNS everything worked fine and I got my certificate. Thank you very, very much!

1 Like
18

That issue can also be solved by restarting the docker daemon, usually.

4 Likes
19

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.