Docker NGINX Certbot 502 response on new cert

[stevenmz]

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: mindfulprogress.org

I ran this command: docker compose -f docker-compose.prod.yml run --rm --entrypoint "
certbot certonly --webroot -w /var/www/certbot
$staging_arg
$email_arg
$domain_args
--rsa-key-size $rsa_key_size
--agree-tos
--force-renewal" certbot

It produced this output:
Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
Domain: mindfulprogress.org
Type: unauthorized
Detail: 132.148.79.171: Invalid response from http://mindfulprogress.org/.well-known/acme-challenge/_1N0FvP-vWCfzzr1fCZ6vsK131xflNAWu28D0bbFBzM: 502

Domain: www.mindfulprogress.org
Type: unauthorized
Detail: 132.148.79.171: Invalid response from http://www.mindfulprogress.org/.well-known/acme-challenge/6v3VvpHWMRWZiw50skD-rBo3dVDr9ND0JkKQUpH9VBw: 502

Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded from the internet.

My web server is (include version): nginx:latest docker image

The operating system my web server runs on is (include version): ubuntu 22.04

My hosting provider, if applicable, is: GoDaddy

I can login to a root shell on my machine (yes or no, or I don't know): Yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): No

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):

I am following the tutorial here: Nginx and Let’s Encrypt with Docker in Less Than 5 Minutes | by Philipp | Medium

My nginx config looks like this (in part):

server {
listen 80;
listen [::]:80;
server_name mindfulprogress.org;

location /.well-known/acme-challenge/ {
    root /var/www/certbot;
}

location / {
    return 301 https://$host$request_uri;
}

}

my docker compose YAML looks like this (in part):

certbot:
    image: certbot/certbot
    entrypoint: "/bin/sh -c 'trap exit TERM; while :; do certbot renew; sleep 12h & wait $${!}; done;'"
    volumes:
      - ./container_volume/certbot/conf:/etc/letsencrypt:rw
      - ./container_volume/certbot/www:/var/www/certbot:rw

  nginx:
    build: ./nginx
    restart: always
    command: "/bin/sh -c 'while :; do sleep 6h & wait $${!}; nginx -s reload; done & nginx -g \"daemon off;\"'"
    ports:
      - ${NGINX_PORT}:80
      - 443:443
    volumes:
      - ./container_volume/certbot/conf:/etc/letsencrypt:ro
      - ./container_volume/certbot/www:/var/www/certbot:ro
    depends_on:
      - web

[rg305]

Hi @stevenmz, and welcome to the LE community forum

@stevenmz

${NGINX_PORT}:80

What is the value of that variable?

@stevenmz

--force-renewal

That won't magically fix any problem.
Please don't use the force when simply renewing a cert.

@stevenmz

Detail: 132.148.79.171: Invalid response from http://mindfulprogress.org/.well-known/acme-challenge/_1N0FvP-vWCfzzr1fCZ6vsK131xflNAWu28D0bbFBzM: 502

That seems like a back-end issue.
Is there another nginx [proxy] in front of the Docker containerized nginx?

@stevenmz

My web server is (include version): nginx:latest docker image

That doesn't help much.
An actual version number is helpful.

[stevenmz]

Hi @rg305 .

Thank you for your reply.

NGINX is mapped to 80:80.

--force-renewal came from the tutorial, but i will remove it now that i know it is not the command argument i am looking for.

There is nothing else on the server in front of my nginx. This is a bare GoDaddy VPS that i installed docker on.

Docker is pulling NGINX 1.25.3

Thanks,
Steven

[MikeMcQ]

@stevenmz

My nginx config looks like this (in part):

Are there any other server blocks? Because nothing you show would cause a 502 Bad Gateway error. The 502 can happen for any number of reasons like a bad proxypass in nginx.

Also, requests to your domain right now don't get handled like that nginx fragment shows. Your nginx server block should redirect HTTP requests but does not. I realize the redirect won't work properly until HTTPS is working but we should still see the 301 redirect from nginx.

curl -I http://mindfulprogress.org
HTTP/1.1 200 OK
Server: nginx/1.24.0

@stevenmz

Docker is pulling NGINX 1.25.3

Well, an nginx 1.24 is responding to http requests. Something seems amiss in your setup

[stevenmz]

Hi @MikeMcQ ,

Thanks for alerting me that docker wasnt pulling the version i intended. I will fix that by being more explicit in my compose file.

There are two server blocks. Here is the second along with the upstream that points to gunicorn for my Django app:

upstream web_app {
    server web:8000;
}

server {
    listen 443 ssl;
    server_name mindfulprogress.org;

    ssl_certificate /etc/letsencrypt/live/mindfulprogress.org/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/mindfulprogress.org/privkey.pem;    
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
    include /etc/letsencrypt/options-ssl-nginx.conf;

    location / {
        proxy_pass http://web_app;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $host;
        proxy_redirect off;
    }

    location /static/ {
        alias /mindful_progress_project/staticfiles/;
    }

    location /media/ {
        alias /mindful_progress_project/mediafiles/;
    }
}

[MikeMcQ]

Then there are other factors not yet disclosed.

HTTP requests are not being handled by the nginx server block listening on port 80 that you show (because there is no redirect)

HTTPS requests don't reach your nginx at all (port 443 is closed). Where do you Do you still see the 502 error? Because I get this

curl -i https://mindfulprogress.org
curl: (7) Failed to connect to mindfulprogress.org port 443 after 70 ms: 
Connection refused

[rg305]

@stevenmz

NGINX is mapped to 80:80.

Is there a NAT device inline?
[anything that can port map]

@stevenmz

There is nothing else on the server in front of my nginx.

Does the host also run nginx?

[stevenmz]

Hi @MikeMcQ ,

Thanks for trying to point me in the right direction. I am new to docker, nginx, etc so i was blindly following the tutorials.

I decided to go a different direction which is not ideal but it got the little lock icon showing up so thats progress.

I followed the instructions here Certbot Instructions | Certbot (eff.org) to get certbot installed on the VPS host and to get the initial certificate using the standalone flag. That worked.

Then, to get the dockerized nginx to see the cert it downloaded, i modified the volume in docker compose to be:

volumes:
      - /etc/letsencrypt/:/etc/letsencrypt:ro

This is not a complete solution because when i try the dry run for renewing the cert, there is now a conflict on port 80 because my website is currently being served on it. The naive thing will be for me to stop the website, renew the cert, bring the website back up every 3 months. Not ideal, but the website now has SSL.

Thanks again,
Steven

[rg305]

You could try using --webroot.
Which is intended to work with whatever web server service may already exist.
[certbot would need access to the expected challenge path in order to place the required file there]