Unauthorized, Invalid response, 404... but with reverse proxy and docker stuff

[erambeau]

Hello,

I am encountering difficulties when renewing my certificate. I managed to do it at one point, but I am not very comfortable with the configuration and I am not sure I understand what is happening.

To summarize my situation, I have an application (Prefect) that I want to access online. So I prepared a docker-compose with this app, nginx and certbot. I have set up an authentication reverse proxy and enabled HTTPS in my nginx.conf (see below). Finally, I created an A-type DNS record with my host that points to the IP. Everything was working fine for 3 months until the renewal, when I got an error.

Is it possible that the reverse proxy is blocking the renewal?
What can I do to address my issue ?

My domain is:
prefect.40projets.com

I ran this command:
sudo docker-compose run --rm certbot certonly --webroot --webroot-path /var/www/certbot/ --dry-run -v -d prefect.40projets.com

It produced this output:
Creating user_certbot_run ... done
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Certificate is due for renewal, auto-renewing...
Simulating renewal of an existing certificate for prefect.40projets.com
Performing the following challenges:
http-01 challenge for prefect.40projets.com
Using the webroot path /var/www/certbot for all unmatched domains.
Waiting for verification...
Challenge failed for domain prefect.40projets.com
http-01 challenge for prefect.40projets.com

Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
Domain: prefect.40projets.com
Type: unauthorized
Detail: XXX.XXX.XX.XX: Invalid response from http://prefect.40projets.com/.well-known/acme-challenge/m0_ZUiwHWmg_xuFyhoIBY9m4f_FfFzf_XrtKQLCeuTI: 404

Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded from the internet.

Cleaning up challenges
Some challenges have failed.

My web server is (include version):
Nginx 1.25.4 (latest)

The operating system my web server runs on is (include version):
Debian GNU/Linux 12 (bookworm)

My hosting provider, if applicable, is:
LWS

I can login to a root shell on my machine (yes or no, or I don't know):
yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):
no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):
2.9.0

Here is my nginx.conf :

# XX.XX.XXX.XXX stands for the IPV4 of my development server
geo $authentification {
    default "Authentication required";
    XX.XX.XXX.XXX "off";
}

server {
    listen 80;
    listen [::]:80;

    server_name prefect.40projets.com www.prefect.40projets.com;
    server_tokens off;

    location /.well-known/acme-challenge/ {
        root /var/www/certbot;
    }

    location / {
        return 301 https://prefect.40projets.com$request_uri;
    }
}

server {
    listen 443 default_server ssl;
    listen [::]:443 ssl;
    http2 on;

    server_name prefect.40projets.com;

    ssl_certificate /etc/letsencrypt/live/prefect.40projets.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/prefect.40projets.com/privkey.pem;

    location / {
        satisfy any;

        proxy_pass http://prefect-server:4200;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

        auth_basic $authentification;
        auth_basic_user_file /etc/nginx/.htpasswd;
    }
}

And here is the part of docker-compose.yml which is interesting with nginx and certbot.

  nginx:
    image: nginx:latest
    container_name: nginx
    volumes:
      - ./nginx.conf:/etc/nginx/conf.d/default.conf
      - /home/user/nginx/.htpasswd:/etc/nginx/.htpasswd:ro
      - /etc/letsencrypt/:/etc/letsencrypt/:ro
    ports:
      - "80:80"
      - "443:443"
    depends_on:
      - server

  certbot:
    image: certbot/certbot:latest
    volumes:
      - ./certbot/www/:/var/www/certbot/:rw
      - ./certbot/conf/:/etc/letsencrypt/:rw

Thank you for your help.

[Osiris]

I don't have much experience with Docker, but shouldn't /var/www/certbot/ also be mapped as a volume in the nginx container/image/whatever to work?

Also, in the nginx container/image/whatever you're mapping /etc/letsencrypt/ to /etc/letsencrypt/, but in the certbot container/image/whatever you're mapping ./certbot/conf/ to /etc/letsencrypt/? That would mean nginx and Certbot wouldn't use the same directory as /etc/letsencrypt/, right?

[erambeau]

It was the point ! Thanks !
Fixed by modifying the docker-compose accordingly.
I don't remember exactly what happened, it's possible that I had to change the configuration at a moment and made the mistake.

  nginx:
    image: nginx:latest
    container_name: nginx
    volumes:
      - ./nginx.conf:/etc/nginx/conf.d/default.conf
      - /home/user/nginx/.htpasswd:/etc/nginx/.htpasswd:ro
      - ./certbot/conf/:/etc/letsencrypt/:ro
      - ./certbot/www/:/var/www/certbot/:ro
    ports:
      - "80:80"
      - "443:443"
    depends_on:
      - server