Certificate won't renew (I think due to the server forcing views to https with outdated cert)

Help
1

I have tried many different alternatives to get the certificate updated. I have gone to manual to try and identify the issue. It seems like LetsEncrypt is not able to read the file because my server is forcing things to https but I can't figure out how.

I have tried a couple different domains on the same server and they all have the same problem.

The certificates were updating automatically but that stopped.

My domain is: externs.com

I ran this command:
certbot certonly -a manual -d externs.com

It produced this output:
I have tried many different alternatives. I have gone to manual to try and identify the issue. It seems like LetsEncrypt is not able to read the file because the server is forcing things to https but I can't figure out how. The file works fine

http://externs.com/.well-known/acme-challenge/v5y57yfJ2Cmv_2TH64LWZrknp3JzSLCeCx41lLc1JDg

after you manually accept that the https certificate is not being used. But I guess the LetsEncrypt process can't deal with that.

Fetching https://externs.com/.well-known/acme-challenge/v5y57yfJ2Cmv_2TH64LWZrknp3JzSLCeCx41lLc1JDg: Connection refused

I can't find the .conf files anywhere. No files are in /etc/nginx has sites-available or sites-enabled.

My web server is (include version):
nginx version: nginx/1.10.3 (Ubuntu)

The operating system my web server runs on is (include version):
Ubuntu 16.04.7 LTS
I know that is old, that is another one of things I am trying to update...

I can login to a root shell on my machine (yes or no, or I don't know):
Yes I can.

2

Uhm, this should work.

$ curl -iLk http://externs.com/.well-known/acme-challenge/v5y57yfJ2Cmv_2TH64LWZrknp3JzSLCeCx41lLc1JDg
HTTP/1.1 302 Moved Temporarily
Server: nginx/1.10.0
Date: Fri, 24 Nov 2023 21:50:35 GMT
Content-Type: text/html
Content-Length: 161
Connection: keep-alive
Location: https://externs.com/.well-known/acme-challenge/v5y57yfJ2Cmv_2TH64LWZrknp3JzSLCeCx41lLc1JDg

HTTP/1.1 200 OK
Server: nginx/1.10.0
Date: Fri, 24 Nov 2023 21:50:36 GMT
Content-Type: application/octet-stream
Content-Length: 88
Last-Modified: Fri, 24 Nov 2023 21:38:39 GMT
Connection: keep-alive
ETag: "656117df-58"
Accept-Ranges: bytes

v5y57yfJ2Cmv_2TH64LWZrknp3JzSLCeCx41lLc1JDg.ex83JuQpoz2-Ml-7Pgd93FSrbz7gn0HzRpaC77YsKaE

This looks like some kind of firewall or security thingy blocking Let's Encrypt's connections.

3 Likes
3

FYI, I just completed the process of renewing a certificate for another domain on the server that had a current good certificate (the issue from my original comment is for a domain where the certificate expired). That processed worked (when I did it manually, like I did for the example above).

4

The fact that the current certificate is expired doesn't matter. Validation servers don't check if the certificate is valid when you redirect to https.

The issue is somewhere else. It might have been a random transient error.

4 Likes
5

The problem is with the IPv6 address in your DNS AAAA record. Let's Encrypt prefers IPv6 if possible and connections to that fail. Even from my own test server to your "home" page

You should correct the IPv6 address and make sure your nginx and Passenger system support it. Or, remove the AAAA record

curl -i6 -m10 http://externs.com
curl: (7) Failed to connect to externs.com port 80 after 208 ms: Connection refused

curl -i4 -m10 http://externs.com
HTTP/1.1 302 Moved Temporarily
Server: nginx/1.10.0
Location: https://externs.com/
5 Likes
6

Thanks for your help.

I have been working on this for several hours and have a couple domains that are failing (that each have the cert expired). That one I just tried that didn't have an expired cert worked. I am trying another one with an active cert to see what happens.

1 Like
7

Adding to my previous comment about IPv6.

Your DNS

nslookup externs.com
A    Address: 172.104.117.22
AAAA Address: 2400:8902::f03c:91ff:fedf:92cb
3 Likes
8

Thanks, I am looking into this now.

2 Likes
9

Actually, the IPv6 is failing outright with connection failure not a timeout so the IPv4 retry is not involved. My mistake

Still, IPv6 is the reason the challenge fails

3 Likes
10

Thanks I don't use IPv6 at all (as far as I know). There are AAAA records for it on the domain record. It doesn't appear to be totally straight forward how to "turn off" IPv6 for the domain? So it may take me some time to figure out how to do that.

1 Like
11

Just remove the AAAA record :slight_smile:

3 Likes
12

That is what I was going to try but the host says that isn't possible? So trying to figure out what that means... Maybe they automatically recreate it if you remove it... I know I never added any IPv6 stuff; they just did it themselves.

1 Like
13

Oh. You might have to talk with Linode about that.

Are they saying it is required?

3 Likes
14

What was the domain name that worked? I'm curious to compare the DNS and HTTP connectivity

We could also look at your failing nginx config. If you show result of this

sudo nginx -T

Must be upper case T. Output will be long so sometimes easier to upload the config.txt file that results from this

sudo nginx -T >config.txt
3 Likes
15

The AAAA record is gone. You should be able to renew now. You do not have to wait for worldwide TTL propagation. Let's Encrypt looks directly at the authoritive DNS servers which I see are updated.

2 Likes
16

Yes, so I deleted the AAAA IPv6 records.
It did let me manually update the cert.

Thanks.

I guess I need to restart the webserver for the updated certs to show up. I am a bit worried as there are things I don't understand. Such as unfindable sites-available/sites-enabled. And sudo nginx -T >config.txt failing

nginx: [emerg] open() "/etc/nginx/nginx.conf" failed (2: No such file or directory)
nginx: configuration file /etc/nginx/nginx.conf test failed

(but right now the webserver is actually serving pages...). I am a bit worried a restart will fail... and then I am in worse shape :frowning:

2 Likes
17

Yes, you need to at least reload nginx to pickup new cert. And, I see your new cert in the public logs but it only has the root name. In the past your cert had the root and the www subdomain so you probably should get a fresh one with both names.

Your server responds with

X-Powered-By: Phusion Passenger 5.0.28
Server: nginx/1.10.0 + Phusion Passenger 5.0.28

I don't know anything about Passenger so hesitant to say much more.

4 Likes
18

Thanks for all your help.

service nginx reload and alternative wording options fail. It is saying nginx "nginx.service is not active, cannot reload."

But the websites are being served (and I can see nginx in top) so it must be active. I don't have the mental energy to keep up with this today. I will try again tomorrow. If anyone has suggestions please let me know. I want to avoid breaking all the sites in case somehow nginx is working now but will fail on restart (nginx -T also fails "[emerg] open() "/etc/nginx/nginx.conf" failed (2: No such file or directory)" - which doesn't make much sense as it is running now...

1 Like
19

I found my sites-available/sites-enabled files. They are in the /opt/nginx/conf folder. I don't understand why the server is working fine now (indicating that things are running fine and setup properly). But then when I try nginx -T it fails as noted above.

I have tried reload and it fails.
"nginx.service is not active, cannot reload."

I guess it is in some kind of "loaded" status but is messed up somehow. Because the websites are all being servered (so ngnix has to be running).

> sudo systemctl status nginx
● nginx.service - A high performance web server and a reverse proxy server
   Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code)

I am a bit paranoid about trying to stop and restart because the "test" is indicating it will fail. Any suggestions would be greatly appreciated.

I will probably try now copying the file to /etc/nginx/nginx.conf
But that seems odd as it has been at this location /opt/nginx/conf/nginx.conf and things have been working fine so I am a bit worried things will somehow get gummed up. But at least if I can get nginx -T to succeed I will be less worried about restarting.

20

You might have more than one nginx installed. /opt usually means something that has been installed outside of your OS's package management framework.

4 Likes