Letsencrypt certbot not issuing new certificate with right config

krs87851992 · January 2, 2022, 6:38pm

I follow the following to setup my website Nginx and Let’s Encrypt with Docker in Less Than 5 Minutes | by Philipp | Medium

My domain is: www.prokom.dev

I ran this command:
My nginx config is

upstream keycloak_server {
    server keycloak:8080;
}

upstream konicaMinolta {
    server km-app:7171;
}

server {
    listen 80;
    server_name prokom.dev www.prokom.dev;
    
    client_max_body_size 25M;
    
    location ^~ /.well-known/acme-challenge/ {
        allow all;
		root /var/www/certbot;
    }
    
    return 301 https://www.prokom.dev$request_uri;
}

server {
    listen 443 ssl;
    server_name prokom.dev;
    
    client_max_body_size 25M;
	
	ssl_certificate /etc/letsencrypt/live/prokom.dev/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/prokom.dev/privkey.pem;
    include /etc/letsencrypt/options-ssl-nginx.conf;
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
	
    
    return 301 https://www.prokom.dev$request_uri;
}



server {
    listen 443 ssl;
    server_name www.prokom.dev;
    
    location ^~ /.well-known/acme-challenge/ {
        allow all;
        root /var/www/certbot;
    }
    
    client_max_body_size 25M;

    location /auth {
        proxy_pass http://keycloak_server;
        proxy_http_version 1.1;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }

    location / {
        proxy_pass http://konicaMinolta;
        proxy_http_version 1.1;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }

    ssl_certificate /etc/letsencrypt/live/prokom.dev/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/prokom.dev/privkey.pem;
    include /etc/letsencrypt/options-ssl-nginx.conf;
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;

}

My docker-compose looks like

 nginx:
    build: nginx-image
    image: km-nginx
    volumes:
      - ./data/certbot/conf:/etc/letsencrypt
      - ./data/certbot/www:/var/www/certbot
    ports:
      - 80:80
      - 443:443
    depends_on:
      - keycloak
      - km-app
    links:
      - keycloak
      - km-app
    environment:
      - PRODUCTION=true
    command: "/bin/sh -c 'while :; do sleep 6h & wait $${!}; nginx -s reload; done & nginx -g \"daemon off;\"'"
      
  certbot:
    image: certbot/certbot
    restart: unless-stopped
    volumes:
      - ./data/certbot/conf:/etc/letsencrypt
      - ./data/certbot/www:/var/www/certbot
    entrypoint: "/bin/sh -c 'trap exit TERM; while :; do certbot renew --webroot -w /var/www/certbot; sleep 12h & wait $${!}; done;'"

The script https://raw.githubusercontent.com/wmnnd/nginx-certbot/master/init-letsencrypt.sh produced this output:

Creating km_certbot_run ... done
Generating a RSA private key
...+++++
......................................+++++
writing new private key to '/etc/letsencrypt/live/prokom.dev/privkey.pem'
-----

### Starting nginx ...
Recreating km_mysql_1 ... done
Recreating km_keycloak_1 ... done
Recreating km_km-app_1   ... done
Recreating km_nginx_1    ... done

### Deleting dummy certificate for prokom.dev ...
Creating km_certbot_run ... done

### Requesting Let's Encrypt certificate for prokom.dev ...
Creating km_certbot_run ... done
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Requesting a certificate for prokom.dev and www.prokom.dev

Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
  Domain: prokom.dev
  Type:   connection
  Detail: Fetching http://prokom.dev/.well-known/acme-challenge/5mZ4xuxyHgpSsAiWZK-Df8HyQFNzvHrNBMiFvIe9bzA: Connection refused

  Domain: www.prokom.dev
  Type:   connection
  Detail: Fetching http://www.prokom.dev/.well-known/acme-challenge/y1Z9n3yY5bUGqA0DKCMmVKIdYThx4QU9qGFRShHy_Wk: Connection refused

Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded from the internet.

Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.

### Reloading nginx ...
ERROR: No container found for nginx_1

For last 2 years this was working. Suddenly today its not able to generate certificates

Osiris · January 2, 2022, 8:10pm

The script that is running sounds like a terrible script called "init-letsencrypt.sh" or something. Where is that script being run in your setup? It isn't mentioned in your Docker file.

Also, at the moment your site is completely down. It isn't answering on port 80 nor on port 443.

I recommend to get your websites basic function up and running again and find out why that terrible init-letsencrypt.sh is running and kill it.. Kill it with fire.

krs87851992 · January 2, 2022, 8:14pm

The script is here https://raw.githubusercontent.com/wmnnd/nginx-certbot/master/init-letsencrypt.sh

I run it manually to create the certfiicates

Osiris · January 2, 2022, 8:16pm

I know the script. It's terrible. Don't use it, especially if you already have certbot certificates issued already, as it deletes perfectly good certificates for no good reason.

I wasn't asking where you downloaded the script from, I was trying to figure out where it's being run in your setup! As it isn't mentioned anywhere in your Docker file. I'll rephrase my post.

krs87851992 · January 2, 2022, 8:19pm

I run it manually. Its not via docker. As this point I am not sure how to get my website up without ssl

Osiris · January 2, 2022, 8:20pm

Your nginx configuration already has SSL configured, the init-letsencrypt.sh should not be necessary.

krs87851992 · January 2, 2022, 8:26pm

Right but when I run directly without the script because the certificate are deleted nginx does not run because it needs cert. Gives the following error

Attaching to km_nginx_1
nginx_1 | 2022/01/02 20:35:41 [emerg] 11#11: cannot load certificate "/etc/letsencrypt/live/prokom.dev/fullchain.pem": BIO_new_file() failed (SSL: error:02001002:system library:fopen:No such file or directory:fopen('/etc/letsencrypt/live/prokom.dev/fullchain.pem','r') error:2006D080:BIO routines:BIO_new_file:no such file)
nginx_1 | nginx: [emerg] cannot load certificate "/etc/letsencrypt/live/prokom.dev/fullchain.pem": BIO_new_file() failed (SSL: error:02001002:system library:fopen:No such file or directory:fopen('/etc/letsencrypt/live/prokom.dev/fullchain.pem','r') error:2006D080:BIO routines:BIO_new_file:no such file)

Osiris · January 2, 2022, 9:14pm

Well.. Yes.. That's because the script is intended (and even then it's terrible the way it's doing things) to be run once, not when there's a perfectly good Let's Encrypt certificate already issued. At least, I assume all the previous certificates for your hostname were already issued to this system.

I don't have any experience with Docker (and I want to keep it that way), so I'm not sure why the script is failing now. I only know nginx isn't running now and apparently the init-letsencrypt.sh doesn't seem to be able to get nginx started too.

My suggestion: comment out the HTTPS server block from your nginx configuration and find a better way to issue certificates using certbot and Docker and forget about that init-letsencrypt.sh script.

krs87851992 · January 2, 2022, 9:56pm

I tried but still get problem generating the cert. This is what I did

Removed nginx config SSL server block. New config looks like

upstream keycloak_server {
    server keycloak:8080;
}

upstream konicaMinolta {
    server km-app:7171;
}

server {
    listen 80;
    listen [::]:80;
    server_name prokom-na.org www.prokom-na.org;
    
    client_max_body_size 25M;
    
    location ^~ /.well-known/acme-challenge/ {
        allow all;
        root /var/www/certbot;
    }
    
    return 301 https://www.prokom-na.org$request_uri;
}

Ran docker-compose up nginx. nginx is now running but website still inaccessible. No error
Ran command to generate cert using certbot

docker-compose run --rm  certbot certonly --webroot -w /var/www/certbot/ --dry-run -d prokom-na.org,www.prokom-na.org

Error

Creating km_certbot_run ... done
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Simulating a certificate request for prokom-na.org and www.prokom-na.org

Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
  Domain: prokom-na.org
  Type:   connection
  Detail: Fetching https://www.prokom-na.org/.well-known/acme-challenge/n9zmMdlitCwQvtKAy9HR9SrwcnxXAtCyuU8HOdYLvtc: Connection refused

  Domain: www.prokom-na.org
  Type:   connection
  Detail: Fetching https://www.prokom-na.org/.well-known/acme-challenge/AvvyoHJfXOT6hhqTZwk8F6R5XJ094a2DXIbDQb-ppU0: Connection refused

Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded from the internet.

Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.

Osiris · January 2, 2022, 9:57pm

There's still a HTTP to HTTPS redirect in your HTTP server block. That redirect isn't working now, as there is no HTTPS server currently listening. You can only set up the HTTPS part again after you get a certificate and only then you need to enable the redirect again.

At least your HTTP nginx is working again it seems

krs87851992 · January 2, 2022, 10:01pm

My understand was that

location ^~ /.well-known/acme-challenge/ {
        allow all;
        root /var/www/certbot;
    }

 would allow HTTP connection and not redirect to HTTPS

Osiris · January 2, 2022, 10:03pm

That soulds plausible indeed, one would expect that. However, from the error message from the Let's Encrypt validation server:

You can see it tried to fetch the token using the https:// protocol. It only does that when it received a HTTP to HTTPS redirect, as the http-01 challenge always starts using http:// on port 80.

So for some reason the location block isn't working. I'm not familiair enough with nginx to debug that properly though.. (Apache guy myself..)

MikeMcQ · January 2, 2022, 10:31pm

You need to wrap your return 301 ... statement in another location block. Otherwise it is always done. Like this

location / {
      return 301 ...
}

rg305 · January 2, 2022, 10:51pm

Try:
echo "Test_File-1234" > /var/www/certbot/Test_File-1234

Then:
http://prokom-na.org/.well-known/acme-challenge/Test_File-1234
http://prokom.dev/.well-known/acme-challenge/Test_File-1234

[not sure why you switched the domain name in the middle of the topic]

rg305 · January 4, 2022, 3:34pm

2 posts were split to a new topic: .well-known/acme-challenge path access 404

system · February 5, 2022, 6:38pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.