Certbot Issue/Renewal fails

[glowfishDE]

Hi there,

i am trying to issue an alternate domain name for an existing certificate.

My setup has a frontend NGINX server that is redirecting traffic to various other internal systems.

So i maintain my LetsEncrypt certificates on the NGINX frontend. This works for all domains and subdomains but not for one. There the challenge fails.

I could once issue the cert by stopping nginx and running the standalone challenge webserver. Thus its saying "renew" certificate now.

So i wonder what am i doing wrong. I can create certificates for all other domains but not for this particular one.

Since incoming traffic is routed to some other server via NGINX i am not even sure if this is the reason. I guess the certbot cannot put its challenge files on a different server. How could i solve that?

The standalone method does only work if i stop and start the nginx by using post and pre deploy hooks. However the hooks seem not to be working fine either as they do not stop nginx in a timely manner as some workers process seems to wait for terminating.

I tried this to test (using certbot 1.11.0)

certbot renew --dry-run

The following simulated renewals failed:
/etc/letsencrypt/live/nginxhostname.domain.tld/fullchain.pem (failure)

---

1 renew failure(s), 0 parse failure(s)

IMPORTANT NOTES:

• The following errors were reported by the server:

  Domain: subdomain.domain.tld
  Type: unauthorized
  Detail: ip.ip.ip.ip: Invalid response from
  https://nginxhostname.domain.tld/.well-known/acme-challenge/S9DAurn6gFd-K0wOXXf9iC_LruMXl1F9hI8mwB5Rbi8:
  401

  To fix these errors, please make sure that your domain name was
  entered correctly and the DNS A/AAAA record(s) for that domain
  contain(s) the right IP address.

[9peppe]

Show us the nginx config for that FQDN.

[glowfishDE]

I could fix the issue but i had to give the subject alternative name its own IP adress on the nginx proxy server. When the new IP was brought up on the proxy and it was given a new section in nginx.conf it was working very well.

What i basically added to nginx.conf was:

upstream subdomain.domain.tld {
server 1.2.3.4:80;
}

And:

server {
listen 5.6.7.8:80;
server_name subdomain.domain.tld 5.6.7.8;
location / {
proxy_pass http://subdomain.domain.tld;
proxy_redirect off;
proxy_max_temp_file_size 0;
proxy_set_header Host $http_host;
proxy_set_header Port $proxy_port;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass_header Authorization;
}
}

1.2.3.4 is the internal IP of the target machine
5.6.7.8 is the public IP on the frontend nginx proxy machine it is listening for incoming traffic for the subdomain

After nginx was running fine with that configuration i could issue my cert with:

certbot --nginx -d subdomain1.domain.tld -d subdomain2.domain.tld -d subdomain3.domain.tld

All challenges went through fine now.

Also i added some http -> https rewrite stuff and adjusted my ciphers etc.

Now it all works like a charm.

This can be closed.
Thanks for pointing me to the nginx.conf way

[glowfishDE]

However i am wondering if there is no other way than adding a new IP to each subdomain. Because if you have like 10 subdomains you are wasting 10 IPs on the frontend. Maybe there is a way that is less of a waste of IPs ?

Whats funny:

i can issue certificates for www.domain.tld and domain.tld under the same IP adress. But when i add a subdomain to the same domain.tld it needs a different IP.

Example:

www.test.com and test.com can have the same IP
shop.test.com needs to have a unique IP in the nginx setup.

Why cant i issue a certificate for www.test.com, shop.test.com and test.com under one IP ?

Does anyone have an idea what i´m doing wrong?
Maybe someone could paste a nginx.conf Sniplet for a working proxy and multiple subdomains under one certificate that is not causing certbot challenge failures.

[9peppe]

You do know that you can host more than one website on the same IP address, by using "Server Name Indication"?

You just listen on every IP, define several server blocks, and what you put in server_name decides the website that gets served.

[glowfishDE]

I think i had a general misunderstanding in my whole 14 years old NGINX concept ^^

Now its working with a single IP as wanted. I used this format for my nginx.conf section.

server {
server_name subdomain1.domain.tld;

location / {
  proxy_pass                http://subdomain1.domain.tld/path/script.cgi;
  proxy_redirect            off;
  proxy_max_temp_file_size  0;
  proxy_set_header          Host                    $http_host;
  proxy_set_header          Port                    $proxy_port;
  proxy_set_header          X-Real-IP               $remote_addr;
  proxy_set_header          X-Forwarded-For         $proxy_add_x_forwarded_for;
  proxy_pass_header         Authorization;
}

location /application {
  proxy_pass                http://subdomain1.domain.tld;
  proxy_redirect            off;
  proxy_max_temp_file_size  0;
  proxy_set_header          Host                    $http_host;
  proxy_set_header          Port                    $proxy_port;
  proxy_set_header          X-Real-IP               $remote_addr;
  proxy_set_header          X-Forwarded-For         $proxy_add_x_forwarded_for;
  proxy_pass_header         Authorization;
}

location /application-images {
  proxy_pass                http://anothersubdomain.domain.tld;
  proxy_redirect            off;
  proxy_max_temp_file_size  0;
  proxy_set_header          Host                    $http_host;
  proxy_set_header          Port                    $proxy_port;
  proxy_set_header          X-Real-IP               $remote_addr;
  proxy_set_header          X-Forwarded-For         $proxy_add_x_forwarded_for;
  proxy_pass_header         Authorization;
}
location /application-web {
  proxy_pass                http://anothersubdomain.domain.tld;
  proxy_redirect            off;
  proxy_max_temp_file_size  0;
  proxy_set_header          Host                    $http_host;
  proxy_set_header          Port                    $proxy_port;
  proxy_set_header          X-Real-IP               $remote_addr;
  proxy_set_header          X-Forwarded-For         $proxy_add_x_forwarded_for;
  proxy_pass_header         Authorization;
}



listen 1.2.3.4:443 ssl;
ssl_certificate /etc/letsencrypt/live/subdomain1.domain.tld/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/subdomain1.domain.tld/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

}

[rg305]

@glowfishDE

1.2.3.4 is the internal IP of the target machine
5.6.7.8 is the public IP on the frontend nginx proxy machine it is listening for incoming traffic for the subdomain

@glowfishDE

upstream subdomain.domain.tld {
server 1.2.3.4:80;           <<<<<<<<<<<<<<< EXTERNAL IP
}

That seems like a loop.

@glowfishDE

server {
listen 5.6.7.8:80;
server_name subdomain.domain.tld 5.6.7.8;     <<<<<<<<<< INTERNAL SITE
location / {
proxy_pass http://subdomain.domain.tld;       <<<<<<<<<< PROXY TO EXTERNAL IP
...

That seems like a loop.