SOLVED ! Nginx keeps looping - Docker - Certificate not found

Hello all !

Since a couple of days I have the following error while trying to generate a new certificate for my subdomain that saying the certificates are not found and so the container keeps restarting in loop.

I would like to mention that the only thing that change on the server is the fact that I added some iptables rules because my server got massively DDoS attacked/bruteforced. But nothing else has change in the server side, I didn't change any permissions for folders etc.. or something else.

I would also like to mention that all my subdomains have the same configuration and the other subdomains are working perfectly.

This one used to work perfectly also.

*Server version : *
PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
NAME="Debian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=debian

When I run the initletsencrypt.sh :

Successfully received certificate.
Certificate is saved at: /etc/letsencrypt/live/husky/fullchain.pem
Key is saved at: /etc/letsencrypt/live/husky/privkey.pem
This certificate expires on 2023-07-19.
These files will be updated when the certificate renews.

NEXT STEPS:
- The certificate will need to be renewed before it expires. Certbot can automatically renew the certificate in the background, but you may need to take steps to enable that functionality. See User Guide — Certbot 2.5.0 documentation for instructions.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
If you like Certbot, please consider supporting our work by:

### Reloading nginx ...
Error response from daemon: Container CONTAINER_ID is restarting, wait until the container is running

When I check the logs from the nginx container (docker ps CONTAINER_ID), I have the following error :
I'm more interest about the certificate issue than the server issue :

2023/04/20 19:21:50 [emerg] 1#1: host not found in upstream "server:8000" in /etc/nginx/conf.d/app.conf:70

nginx: [emerg] host not found in upstream "server:8000" in /etc/nginx/conf.d/app.conf:70

2023/04/20 19:21:51 [emerg] 1#1: cannot load certificate "/opt/husky/bsc/release-data/certbot/conf/live/husky/fullchain.pem": BIO_new_file() failed (SSL: error:02001002:system library:fopen:No such file or directory:fopen('/opt/husky/bsc/release-data/certbot/conf/live/husky/fullchain.pem','r') error:2006D080:BIO routines:BIO_new_file:no such file)

nginx: [emerg] cannot load certificate "/opt/husky/bsc/release-data/certbot/conf/live/husky/fullchain.pem": BIO_new_file() failed (SSL: error:02001002:system library:fopen:No such file or directory:fopen('/opt/husky/bsc/release-data/certbot/conf/live/husky/fullchain.pem','r') error:2006D080:BIO routines:BIO_new_file:no such file)

2023/04/20 19:21:52 [emerg] 1#1: cannot load certificate "/opt/husky/bsc/release-data/certbot/conf/live/husky/fullchain.pem": BIO_new_file() failed (SSL: error:02001002:system library:fopen:No such file or directory:fopen('/opt/husky/bsc/release-data/certbot/conf/live/husky/fullchain.pem','r') error:2006D080:BIO routines:BIO_new_file:no such file)

When I check the folder I have the files :

root@synan:/opt/husky/bsc/release-data/certbot/conf/live/husky# ls -l
total 4
-rw-r--r-- 1 root root 692 Apr 20 12:21 README
lrwxrwxrwx 1 root root 31 Apr 20 12:21 cert.pem -> ../../archive/husky/cert1.pem
lrwxrwxrwx 1 root root 32 Apr 20 12:21 chain.pem -> ../../archive/husky/chain1.pem
lrwxrwxrwx 1 root root 36 Apr 20 12:21 fullchain.pem -> ../../archive/husky/fullchain1.pem
lrwxrwxrwx 1 root root 34 Apr 20 12:21 privkey.pem -> ../../archive/husky/privkey1.pem

Also the interesting part of my docker-compose.yml file for this subdomain :
nginx:
image: nginx:1.21-alpine
restart: unless-stopped

volumes:
- ./release-data/nginx:/etc/nginx/conf.d
- ./release-data/certbot/conf:/etc/letsencrypt
- ./release-data/certbot/www:/var/www/certbot
ports:
- "8085:80"
- "4385:443"
command: "/bin/sh -c 'while :; do sleep 6h & wait $${!}; nginx -s reload; done & nginx -g "daemon off;"'"

certbot:
image: certbot/certbot
restart: unless-stopped
volumes:

- ./release-data/certbot/conf:/etc/letsencrypt
- ./release-data/certbot/www:/var/www/certbot
entrypoint: "/bin/sh -c 'trap exit TERM; while :; do certbot renew; sleep 12h & wait $${!}; done;'"

volumes:
redis:

Do you have an idea ?

Thank you !

1 Like

Hello @synan, welcome to the Let's Encrypt community. :slightly_smiling_face:

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is:

I ran this command:

It produced this output:

My web server is (include version):

The operating system my web server runs on is (include version):

My hosting provider, if applicable, is:

I can login to a root shell on my machine (yes or no, or I don't know):

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):

Thank you for assisting us in helping YOU!

2 Likes

What shows?:
ls -l /opt/husky/bsc/release-data/certbot/conf/live/husky/
ls -l /opt/husky/bsc/release-data/certbot/conf/archive/husky/
ls -l /etc/letsencrypt/live/husky/
ls -l /etc/letsencrypt/archive/husky/
find / -name cert1.pem

3 Likes

root@synan:~# ls -l /opt/husky/bsc/release-data/certbot/conf/live/husky/
total 4
-rw-r--r-- 1 root root 692 Apr 20 12:21 README
lrwxrwxrwx 1 root root 31 Apr 20 12:21 cert.pem -> ../../archive/husky/cert1.pem
lrwxrwxrwx 1 root root 32 Apr 20 12:21 chain.pem -> ../../archive/husky/chain1.pem
lrwxrwxrwx 1 root root 36 Apr 20 12:21 fullchain.pem -> ../../archive/husky/fullchain1.pem
lrwxrwxrwx 1 root root 34 Apr 20 12:21 privkey.pem -> ../../archive/husky/privkey1.pem

root@synan:~# ls -l /opt/husky/bsc/release-data/certbot/conf/archive/husky/
total 20
-rw-r--r-- 1 root root 2220 Apr 20 12:21 cert1.pem
-rw-r--r-- 1 root root 3749 Apr 20 12:21 chain1.pem
-rw-r--r-- 1 root root 5969 Apr 20 12:21 fullchain1.pem
-rw------- 1 root root 3272 Apr 20 12:21 privkey1.pem

root@synan:~# ls -l /etc/letsencrypt/live/husky
ls: cannot access '/etc/letsencrypt/live/husky': No such file or directory

It seems that the letsencrypt folder doesn't exist.
But my configuration is working perfectly for the other subdomains and also worked for this one in the past.

root@synan:/etc# find / -name cert1.pem
/var/lib/docker/overlay2/ce4715404141e31fed7b7992a4194957890be58fcf41be6d6a6e0c70288b5942/diff/opt/certbot/src/certbot/certbot/tests/testdata/sample-archive-ec/cert1.pem
/var/lib/docker/overlay2/ce4715404141e31fed7b7992a4194957890be58fcf41be6d6a6e0c70288b5942/diff/opt/certbot/src/certbot/certbot/tests/testdata/sample-archive/cert1.pem
/var/lib/docker/overlay2/d5fa565b503b97a2d40d620b0cd56108205fb16326661ab6c5d6142f3f830c66/merged/opt/certbot/src/certbot/certbot/tests/testdata/sample-archive-ec/cert1.pem
/var/lib/docker/overlay2/d5fa565b503b97a2d40d620b0cd56108205fb16326661ab6c5d6142f3f830c66/merged/opt/certbot/src/certbot/certbot/tests/testdata/sample-archive/cert1.pem
/opt/husky/bsc1/release-data/certbot/conf/archive/husky/cert1.pem
/opt/husky/bsc2/release-data/certbot/conf/archive/husky/cert1.pem
/opt/husky/bsc3/release-data/certbot/conf/archive/husky/cert1.pem
/opt/husky/bsc4/release-data/certbot/conf/archive/husky/cert1.pem
/opt/husky/bsc/release-data/certbot/conf/archive/husky/cert1.pem
/opt/husky/bsc5/release-data/certbot/conf/archive/husky/cert1.pem

That fails to find a file while looking in path:
/opt/husky/bsc/release-data/certbot/conf/live/husky/

But searching shows those files exist in the links for that path:

So...
I'm thinking that permissions are to blame.

3 Likes

Which permissions ? On which files/folders ?

This one is specifically mentioned in the fopen() failure:

3 Likes

Okay I need to wait as I reached the limit to generate a new request and will update the post then !

Thanks mate :slight_smile:

1 Like

Testing and debugging are best done using the Staging Environment as the Rate Limits are much higher. Rate Limits are per week (rolling).

And to assist with debugging there is a great place to start is Let's Debug.

3 Likes

The file got the same right as the others :

root@synan:/opt/husky/bsc/release-data/certbot/conf/live/husky# ls -l
total 4
-rw-r--r-- 1 root root 692 Apr 21 11:15 README
lrwxrwxrwx 1 root root 31 Apr 21 11:15 cert.pem -> ../../archive/husky/cert1.pem
lrwxrwxrwx 1 root root 32 Apr 21 11:15 chain.pem -> ../../archive/husky/chain1.pem
lrwxrwxrwx 1 root root 36 Apr 21 11:15 fullchain.pem -> ../../archive/husky/fullchain1.pem
lrwxrwxrwx 1 root root 34 Apr 21 11:15 privkey.pem -> ../../archive/husky/privkey1.pem

hmm...
I guess that's as far as I can "help".
I'm not familiar with this configuration :frowning:

2 Likes

I solved the issue !

Thank you :slight_smile:

The post you marked as “Solution” does not have any solution in it, please explain what you did or tag the right post. :slightly_smiling_face:

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.