Nginx failed restart

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: www.gmental.com, gmental.com

I ran this command: sudo certbot certonly -n -v --nginx --agree-tos --force-renew -d gmental.com,www.gmental.com -m green@gmental.com}

It produced this output:
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator nginx, Installer nginx
Renewing an existing certificate for gmental.com and www.gmental.com
Reloading nginx server after certificate issuance

Successfully received certificate.
Certificate is saved at: /etc/letsencrypt/live/gmental.com/fullchain.pem
Key is saved at: /etc/letsencrypt/live/gmental.com/privkey.pem
This certificate expires on 2022-09-28.
These files will be updated when the certificate renews.
Certbot has set up a scheduled task to automatically renew this certificate in the background.

NEXT STEPS:

  • The certificate was saved, but was not successfully loaded by the installer (nginx) due to the installer failing to reload. After fixing the error shown below, try reloading nginx manually.

nginx restart failed:
nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Unknown error)
nginx: [emerg] bind() to [::]:80 failed (98: Unknown error)
nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Unknown error)
nginx: [emerg] bind() to [::]:80 failed (98: Unknown error)
nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Unknown error)
nginx: [emerg] bind() to [::]:80 failed (98: Unknown error)
nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Unknown error)
nginx: [emerg] bind() to [::]:80 failed (98: Unknown error)
nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Unknown error)
nginx: [emerg] bind() to [::]:80 failed (98: Unknown error)
nginx: [emerg] still could not bind()

Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.

My web server is (include version):
nginx version: nginx/1.18.0 (Ubuntu)

The operating system my web server runs on is (include version):
Linux ip-172-31-14-124 5.15.0-1014-aws #18-Ubuntu SMP Wed Jun 15 20:04:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
My hosting provider, if applicable, is:
AWS

I can login to a root shell on my machine (yes or no, or I don't know):yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):certbot 1.28.0

Please don't use this option unless it's absolutely necessary. And I don't see any reason in your situation. You've already issued 5 certificates the last few days: crt.sh | gmental.com of which 4 are most likely issued unnecessarily.

Anyway, is there an actual issue with your nginx? Or is it running as expected?

I'm also tagging @certbot-devs as I think there's something fishy going on here: why would Certbot complain about the cert being "not successfully loaded by the installer (nginx)" when Certbot was run with the certonly subcommand?

7 Likes

Well, I stopped and started the AWS instance which caused AWS to assign a new ip address to the nginx web server. I believed this caused connection issues to my site, so I wanted to renew the certificate.

I attached
letsencrypt.txt (31.2 KB)
in case it'll help.

I noticed the ip address in the log: 13.57.31.58 (the old IP for my AWS instance) is different than the current ip address of my AWS instance: 54.241.45.216

Let's Encrypt certificates don't care about IP addresses, so no need to renew the certificate.

6 Likes

Though I don't think it's directly related to what's going on here. The difference under certonly --nginx is ultimately whether Certbot reloads nginx 1 time or 2 times. If it's failing on the 2nd reload, something has gone wrong in the environment, maybe with the nginx pid file.

6 Likes

@_az I was specifically referring to the message that claims that Certbot tried to "load" the certificate by the installer. While not literally saying so, this sounds like "installing into" to me. But with certonly, while reloading a few times, there should be no actual "loading" or "installing" of the certificate.

Edit:
Or does it mean like, if it's a renewal? With an already configured certificate? It probably does. In that case it makes more sense.. I'd have to figure out if this message would also present itself if there is no HTTPS configuration for the certificate yet.

6 Likes

I see. The error message can probably be reworded for clarity about what's happening.

6 Likes

I believe currently Certbot only checks for an installer error and if certonly was used, but does not check for the (renewed) certificate actually being in use:

I often use the --apache plugin to get a certificate for hostnames Apache isn't even configured for (yes, that seems to work for me), so any error should, in those cases, not mentioning anything about "loading" :slight_smile:

Not sure if it's easily possible to check for existance of the certificate in the webservers configuration at that point tho.

5 Likes

Yes, if you don't have an Elastic IP for your EC2 instance you would also need to update the DNS records to point to your new IP. The Let's Encrypt server (and anyone else) uses DNS to find your IP.

These errors can happen for various reasons. One, nginx should be running before you use the certbot command with --nginx - was it? And, two, sometimes if you have perl active - do you? I recently described a problem with curl perl and the certbot nginx plug-in in this thread. Let us know about these.

@_az Sometime back you were going to look into handling the case better when the nginx restart fails and you start nginx without systemctl. Did you have any luck with that? I don't know that's what is happening here but I can't help being reminded of that.

Edited to improve wording in "These errors" paragraph.

7 Likes

Isn't this a conflict? I think of certonly as meaning no installer so why does it show installer nginx? I would expect to see nginx plug-in just for the authenticator. Or, a warning that certonly is being ignored.

Sorry if I am repeating the discussion you already started.

7 Likes

See the Github issue above linked by Alex.

6 Likes

I did but color me dense then. I still don't understand why installer logic is even used with certonly.

7 Likes

It's for the very fringe situation when someone already has a certificate installed, but doesn't use renew to renew it (e.g. for changing certain settings such as "must staple") and therefore uses certonly but with an installer set. So there is a installer plugin loaded (can also be for selecting hostnames), but due to certonly it doesn't actually install anything. In that case, the installer plugin will reload the webserver even when certonly is being used.

Personally, I think it's useless/redundant, because if a certain certificate is already installed, there is nothing wrong with using the run subcommand: Certbot will notice the cert already being installed. It won't suddenly duplicate the HTTPS VirtualHost or something like that. Using certonly doesn't add anything beneficial to the whole thing, but does make things more difficult.

So while I understand what the issue was and what the PR is about, I don't think it was worth the effort. But it might be that I'm missing something too. :slight_smile:

6 Likes

Thanks for the details @Osiris . I have some follow up I'll take up elsewhere as I don't want to distract from helping siralbert. Cheers.

@siralbert It looks like you still have a problem as your port 443 is closed. To cleanup your nginx after the "bind port 80" error you can try this "quick and dirty" way:

sudo killall nginx
sudo systemctl start nginx

Confirm with:
sudo systemctl status nginx 
(active running time should be just seconds ago)

I am assuming the bind error was caused by reasons in my post #9. Of course, other things can cause these problems but I think this has good chance of helping.

Let us know how it works out.

8 Likes

Wait a minute. I still don't see port 443 open but I see Apache responding on port 80.

@siralbert You used the nginx plug-in in your first post but Apache is running on port 80.

If you used the nginx plug-in without a running nginx server you can get the symptom you showed (bind port 80). The nginx plug-in would have tried to reload nginx during authentication but would have failed if no running nginx. The plug-in would then try to start nginx but that would fail since Apache has control of port 80.

Can you explain more about what you are trying to do?

curl -I http://gmental.com
HTTP/1.1 200 OK
Date: Thu, 30 Jun 2022 21:13:42 GMT
Server: Apache/2.4.52 (Ubuntu)
Link: <https://gmental.com/wp-json/>; rel="https://api.w.org/"
Content-Type: text/html; charset=UTF-8
7 Likes

I am in the process of migrating my website to AWS because I read that nginx servers are generally faster and load-balances web traffic better.

Unfortunately, I stopped and restarted the AWS instance from the AWS management console because I was unsure about its pricing policies. This caused the IP address to change as I mentioned above.

I thought simply changing the @ record from my existing hosting provider would fix the connection issue to the AWS , but it didn't. The next step I tried was renewing the certificate with the command I mentioned above and testing it out in an install script:

sudo certbot certonly -n -v --nginx --agree-tos --force-renew -d gmental.com, www.gmental.com -m green@gmental.com

Anyways, I resolved the connection issue with your help. I checked the port on AWS and it was open. The next thing I did was stopping the apache2 server and starting the nginx server. This fixed the issue. One last thing though. . . I ran the curl -I command below and got a certificate expired message. Should I be concerned about that?

curl -I https://www.gmental.com

curl: (60) SSL certificate problem: certificate has expired
More details here: curl - SSL CA Certificates

curl performs SSL certificate verification by default, using a "bundle"
of Certificate Authority (CA) public keys (CA certs). If the default
bundle file isn't adequate, you can specify an alternate file
using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
the bundle, the certificate verification probably failed due to a
problem with the certificate (it might be expired, or the name might
not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
the -k (or --insecure) option.

Yes, you should, it should not do that. But, it looks ok to me now. I tried the same curl and it works fine. Also, this SSL Decoder test site says it is fine too.

Does that message still appear for you?

Note I got redirected from the www domain to your apex with above curl but there is nothing wrong with that.

curl -I https://www.gmental.com
HTTP/1.1 301 Moved Permanently
Server: nginx/1.18.0 (Ubuntu)
Date: Fri, 01 Jul 2022 01:10:18 GMT
Content-Type: text/html; charset=UTF-8
Connection: keep-alive
X-Redirect-By: WordPress
Location: https://gmental.com/

curl -I https://gmental.com
HTTP/1.1 200 OK
Server: nginx/1.18.0 (Ubuntu)
Date: Fri, 01 Jul 2022 01:10:27 GMT
Content-Type: text/html; charset=UTF-8
Connection: keep-alive
Link: <https://gmental.com/wp-json/>; rel="https://api.w.org/"
8 Likes

Yes, it still does. What does this mean?

What system are you running that curl from? Your new AWS EC2?

And, can you show result of these:

curl --version
curl -kv https://www.gmental.com
2 Likes

Are you using the Apache server at all?
If not, then you might want to uninstall it.

3 Likes