Can't get an extra certificate with NGINX (already have a few working)

Help
1

Hi, I've an issue with one of my subdomain. On the same server, I use letsencrypt for a while but when I wanted to create another certificate this last week, it failed (see details below)

I'm wondering if something changed on letsencrypt or NGINX behavior because I have a dozen certificates on this server and if you test both domains jellyfin-test.ehretic.fr and jellyfin.ehretic.fr you will see that the "non-test" instance works and already have a LetsEncrypt certificate.

I run the same command in previous certificate requests without any issue. Nothing changed in my configurations except the new server block added for this new instance.

I've automatic redirection of port 80 to 443 on the NGINX box with this:

   # Redirect all HTTP traffic to HTTPS
    server {
        listen 80 default_server;
        server_name _;
        return 301 https://$host$request_uri;
    }

My domain is: jellyfin-test.ehretic.fr

I ran this command: certbot --nginx -d jellyfin-test.ehretic.fr -v

It produced this output:

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator nginx, Installer nginx
Requesting a certificate for jellyfin-test.ehretic.fr
Performing the following challenges:
http-01 challenge for jellyfin-test.ehretic.fr
Waiting for verification...
Challenge failed for domain jellyfin-test.ehretic.fr
http-01 challenge for jellyfin-test.ehretic.fr

Certbot failed to authenticate some domains (authenticator: nginx). The Certificate Authority reported these problems:
  Domain: jellyfin-test.ehretic.fr
  Type:   connection
  Detail: During secondary validation: 80.13.195.192: Fetching http://jellyfin-test.ehretic.fr/.well-known/acme-challenge/ZiwwOke8k9xWpGv8gCjNTq9BpkQSDRZWdKz7ID10sDA: Timeout during connect (likely firewall problem)

Hint: The Certificate Authority failed to verify the temporary nginx configuration changes made by Certbot. Ensure the listed domains point to this nginx server and that it is accessible from the internet.

Cleaning up challenges
Running post-hook command: systemctl reload nginx
Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.
[root@srv-rpxy-01 ~]# vi /etc/nginx/nginx.conf
[root@srv-rpxy-01 ~]# certbot --nginx -d jellyfin-test.ehretic.fr -v
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator nginx, Installer nginx
Requesting a certificate for jellyfin-test.ehretic.fr
Performing the following challenges:
http-01 challenge for jellyfin-test.ehretic.fr
Waiting for verification...
Challenge failed for domain jellyfin-test.ehretic.fr
http-01 challenge for jellyfin-test.ehretic.fr

Certbot failed to authenticate some domains (authenticator: nginx). The Certificate Authority reported these problems:
  Domain: jellyfin-test.ehretic.fr
  Type:   connection
  Detail: During secondary validation: 80.13.195.192: Fetching http://jellyfin-test.ehretic.fr/.well-known/acme-challenge/uCzcvGgDbuLrVjFsZaug50qFBIZGCpKBsISWUKV6SsU: Timeout during connect (likely firewall problem)

Hint: The Certificate Authority failed to verify the temporary nginx configuration changes made by Certbot. Ensure the listed domains point to this nginx server and that it is accessible from the internet.

Cleaning up challenges
Running post-hook command: systemctl reload nginx
Some challenges have failed.

My web server is (include version): nginx/1.25.4

The operating system my web server runs on is (include version): Almalinux 8.9

My hosting provider, if applicable, is: myself

I can login to a root shell on my machine (yes or no, or I don't know): yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): certbot 1.22.0

Thanks a lot for your help.

2

The "During secondary validation" means the primary Let's Encrypt servers were able to reach you but one/some of its secondary sites could not.

Does this problem repeat? Sometimes temp comms problems can cause this.

There was a recent change in that Let's Encrypt has added a 4th global validation point. But, I don't think this is causing the above problem. Just be sure that your firewall allows HTTP access from any IP / geographic region and you should be fine. I see you have recently renewed some of your other certs which is why I say I don't think it it cause of above.

3 Likes
3

Hi Mike,

You nailed it! I didn't update it recently but it was caused by my geoIP limitations.
I always authorized a few countries I travel often + USA (and in the past the verifications were apparently always done from the US).

Now I can see IPs from other countries like Singapore (or at least GeoIP thinks it's there) or Sweden

I think it is a bit of a problem not having a specific whitelist for Let's Encrypt servers but if it not planned or desired for security reasons, I get it too.
Well, now I have too many failed attemps, so I guess I'll just have to wait!

Thanks a lot :wink:

3 Likes
4

Terrific. Yeah, they don't publish the IP addresses. More details at this FAQ answer:

3 Likes
5

Yeah I know but what would be good/sufficiant is either to know the service provider where services are hosted (AWS for instance), or a list of susceptible countries.

It's not the right time to allow countries like Russia, China or Ukraine through your firewal those days! :innocent:
(IT security speaking of course - I have nothing against people)

1 Like
6

You sound skilled enough to watch the IP's and see the current patterns. Yes, AWS features prominently as secondary validation sites. They have plans to add a 5th validation farm soon. The primary is not AWS (today)

They have always coached that their IP's can change at a moment's notice. They are looking at adding a different provider. Their have been other substantial changes. If they were to publish these they would want to automate alerting of changes or developing an API to publish them. I am certain that at least some firewall operators would not react promptly :slight_smile: So, just trading one set of problems for another. Possible expansion of surface for attack as well.

They need reliable comms from all their validation sites so I don't see them using any of the countries you mention any time soon.

These are my personal thoughts not at all official. For some official extra info see below post and even that whole thread.

2 Likes
7

You sound skilled enough to realize that HTTP doesn't have to be blocked/protected [at all].
It merely needs to do only these two things:

  • handle the ACME challenge requests
  • redirect all other traffic to HTTPS

HTTPS [as well as all other ports] can/should be fully blocked/protected.

4 Likes
8

Well, I still wanted to drop "shady countries" traffic completely before hitting my RP but now I opened it "a little" more!!! :grin:
(I have way more power on my hardware FW compared to my RP VM, if it makes sense... and I know, shady traffic can come from anywhere)

3 Likes
9

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.