Renewal failure - cannot bind to port


#1

Hi,

Just ran the update cert scripts and had this error shown below. The command used via cron is:

/usr/bin/letsencrypt renew

I see that the script is trying to bind to port 443, which looks daft because Nginx runs on port 443.
Can the port be changed?

How can I correct this?

Best wishes, Sophie

O/S Debian 9.4 with these packages:

ii certbot 0.19.0-1~bpo9+1 all automatically configure HTTPS using Let’s Encrypt
ii python-certbot 0.19.0-1~bpo9+1 all main library for certbot
ii python-certbot-nginx 0.19.0-1~bpo9+1 all Nginx plugin for Certbot

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Cert is due for renewal, auto-renewing…
Could not choose appropriate plugin: The manual plugin is not working; there may be problems with your existing configuration.
The error was: PluginError(‘An authentication script must be provided with --manual-auth-hook when using the manual plugin non-interactively.’,)
Attempting to renew cert (webmail.example.co.uk) from /etc/letsencrypt/renewal/webmail.example.co.uk.conf produced an unexpected error: The manual plugin is not working; there may be problems with your existing configuration.
The error was: PluginError(‘An authentication script must be provided with --manual-auth-hook when using the manual plugin non-interactively.’,). Skipping.
Cert not yet due for renewal
Cert is due for renewal, auto-renewing…
Could not choose appropriate plugin: The manual plugin is not working; there may be problems with your existing configuration.
The error was: PluginError(‘An authentication script must be provided with --manual-auth-hook when using the manual plugin non-interactively.’,)
Attempting to renew cert (example.co.uk) from /etc/letsencrypt/renewal/example.co.uk.conf produced an unexpected error: The manual plugin is not working; there may be problems with your existing configuration.
The error was: PluginError(‘An authentication script must be provided with --manual-auth-hook when using the manual plugin non-interactively.’,). Skipping.
Cert is due for renewal, auto-renewing…
Plugins selected: Authenticator standalone, Installer None
Renewing an existing certificate
Performing the following challenges:
tls-sni-01 challenge for mx10.example.co.uk
Cleaning up challenges
Attempting to renew cert (mx10.example.co.uk) from /etc/letsencrypt/renewal/mx10.example.co.uk.conf produced an unexpected error: Problem binding to port 443: Could not bind to IPv4 or IPv6… Skipping.
All renewal attempts failed. The following certs could not be renewed:
/etc/letsencrypt/live/webmail.example.co.uk/fullchain.pem (failure)
/etc/letsencrypt/live/example.co.uk/fullchain.pem (failure)
/etc/letsencrypt/live/mx10.example.co.uk/fullchain.pem (failure)


Processing /etc/letsencrypt/renewal/webmail.example.co.uk.conf


Processing /etc/letsencrypt/renewal/wm10.example.co.uk.conf


Processing /etc/letsencrypt/renewal/example.co.uk.conf


Processing /etc/letsencrypt/renewal/mx10.example.co.uk.conf


The following certs are not due for renewal yet:
/etc/letsencrypt/live/wm10.example.co.uk/fullchain.pem (skipped)
All renewal attempts failed. The following certs could not be renewed:
/etc/letsencrypt/live/webmail.example.co.uk/fullchain.pem (failure)
/etc/letsencrypt/live/example.co.uk/fullchain.pem (failure)
/etc/letsencrypt/live/mx10.example.co.uk/fullchain.pem (failure)

3 renew failure(s), 0 parse failure(s)

Sorry for ugly formatting. Blockquotes not working for this text.


#2

That’s probably because you requested the certificate in the beginning with the standalone plugin (which will spin up its own server temporarily) and the tls-sni-01 challenge (which has been deprecated).

You can change it to port 80 for the http-01 challenge, but I’m guessing there’s a webserver running there too :wink:

This suggests you used the manual plugin for this certificate… Without a script to manage the challenge, the renew command will skip all the certificates with the manual plugin, because, well, you can’t manually do stuff when the client is ran through cron.

I would suggest you start over all the certificates you initially made with the manual or standalone plugins. Those plugins were a poor choice to start with when you want to renew without any user input or without stopping any servers. The webroot and/or ngingx plugin are the best choice of plugins.


#3

Hi,

Thanks for the explanation .

Those plugins were a poor choice to start with…

Were a sensible choice based on functionality available by letsencrypt at that time based on a reasonable expectation said functionality would remain. Especially since the webroot option is not available because of security requirements based on our security policy. A pity.


#4

Any user input during the use of the manual plugin cannot be done during automatic renewal through cron unfortunately. However, the manual plugin can use scripts to do the “”“manual”"" part for you, so you can use this plugin with automatic renwal. See for more information the documentation on the manual plugin and the Pre and Post Validation Hooks section.

As for the standalone plugin: you can use hooks to stop and start the webserver. See the Renewing certificates section of the manual for more information about the different kind of hooks available.


#5

Hi,

I recall I used this command ( but the .bash_history did get wiped )

certbot certonly --register-unsafely-without-email --agree-tos --standalone -d mx10.example.com

I decided I should to ditch letsencrypt and buy from GoDaddy.

I need reliability but the LetsEncrypt automated plugins and methods can change at a moments notice, which is not helpful for my production servers.

For folks who have time to check often that functionality does not change, then LetsEncrypt is good and I am pleased to see this available. But not for time stressed system admins like me.

Best, Soph.


#6

That isn’t daft, because you told certbot to use port 443 by itself when you did:

Because you used standalone, it is trying to bind to port 443.

You see, this isn’t certbot/letsencrypts fault. It is only doing what you told the program.

I’m very sorry you’re somewhat disappointed with Let’s Encrypt, but all is related by some form of lack of enough information about how Let’s Encrypt and the client certbot works.


#7

Hi,

Good point.

I have to revisit this and to find a way to automate the update without using a well known port, because those are all blocked except specifically permitted well known ports (e.g 80/443 for nginx).

I have 20 days to figure this out because the certs expire in 22 days. If I fail I can fall back to a paid ssl cert.

Best, Soph


#8

The CA/Browser forum rules for Domain Validated certificates require using a well-known port when connecting to the applicant’s server as a way of verifying control over the domain name. The main reason for this is shared hosting where a large number of users may have shell access and be able to bind other ports. (Another reason is probably the risk of protocol-in-protocol attacks.)

See https://cabforum.org/wp-content/uploads/CA-Browser-Forum-BR-1.5.6.pdf (you can search for “Authorized Ports”).

If you don’t want to use webroot, one option would be to CNAME _acme-challenge.webmail.klunky.co.uk (etc.) to a resource on any DNS provider that supports update via API (some people here have been recommending CloudFlare, although there are several options). Then you can get new certificates just by using an API to post TXT records to the resource that’s the target of the CNAME, which doesn’t have to be on your existing DNS servers at all!


#9

Hi,

Just wondering, but could something like this work, depending on how long the renewal process took? The web server has to be reloaded afterwards anyway for the certificates to be picked up anyway.

certbot renew --pre-hook “service nginx stop” --post-hook “service nginx start”

Would I have to adjust the config files to include a preferred challenge?

/etc/letsencrypt/renewal/mx10.example.co.uk.conf


#10

Something like that is a pretty common pattern when using standalone, and it can go in the renewal configuration file. It will also be saved for you there if you re-issue the certificate once with certonly and all of your preferred options (which requires running certonly with --force-renew if the certificate isn’t near expiry).


#11

Hi Schoen,

If I understood, I can:

1 Leave my configuration files untouched.
2 Attempt to renew the certificates by running the command:

certbot renew --pre-hook “service nginx stop” --post-hook “service nginx start”

And adding the —force-renew option if the certs are not ready for renewal.

Many thanks, Soph.


#12

I believe you want certbot certonly -d mx10.example.com --pre-hook “service nginx stop” --post-hook “service nginx start” instead (having to do with a rule about renew not modifying the renewal configuration files on disk).


#13

Are you sure? It seems to update them for me…


#14

I’m not absolutely positive because there were a number of discussions about changing this, so I’ll have to look into it to confirm the current behavior.


#15

I tried this but no luck. I have to find a way around the plugin method, for a start.

$ certbot renew --pre-hook “service nginx stop” --post-hook “service nginx start”
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Processing /etc/letsencrypt/renewal/webmail.example.co.uk.conf
Cert is due for renewal, auto-renewing…
Could not choose appropriate plugin: The manual plugin is not working; there may be problems with your existing configuration.
The error was: PluginError(‘An authentication script must be provided with --manual-auth-hook when using the manual plugin non-interactively.’,)
Attempting to renew cert (webmail.example.co.uk) from /etc/letsencrypt/renewal/webmail.example.co.uk.conf produced an unexpected error: The manual plugin is not working; there may be problems with your existing configuration.
The error was: PluginError(‘An authentication script must be provided with --manual-auth-hook when using the manual plugin non-interactively.’,). Skipping.

This looked like this could work:

$ certbot certonly -d mx10.example.co.uk --pre-hook “service nginx stop” --post-hook “service nginx start”
Saving debug log to /var/log/letsencrypt/letsencrypt.log
How would you like to authenticate with the ACME CA?
1: Nginx Web Server plugin - Alpha (nginx)
2: Spin up a temporary webserver (standalone)
3: Place files in webroot directory (webroot)
Select the appropriate number [1-3] then [enter] (press ‘c’ to cancel):

But, I cancelled with a ^C because I am unsure what actually will happen. I think I want option 2.


#16

Yes, you want the standalone option if you’re stopping your nginx before the validation.


#17

This worked. Thank-you everybody for all your advice.

$ certbot certonly --force-renew -d mx10.example.com --pre-hook “service nginx stop” --post-hook “service nginx start”
Saving debug log to /var/log/letsencrypt/letsencrypt.log
How would you like to authenticate with the ACME CA?
1: Nginx Web Server plugin - Alpha (nginx)
2: Spin up a temporary webserver (standalone)
3: Place files in webroot directory (webroot)

Select the appropriate number [1-3] then [enter] (press ‘c’ to cancel): 2
Plugins selected: Authenticator standalone, Installer None
Running pre-hook command: service nginx stop
Renewing an existing certificate
Performing the following challenges:
tls-sni-01 challenge for mx10.example.co.uk
Waiting for verification…
Cleaning up challenges
Running post-hook command: service nginx start

IMPORTANT NOTES:

  • Congratulations! Your certificate and chain have been saved at:
    /etc/letsencrypt/live/mx10.example.com/fullchain.pem
    Your key file has been saved at:
    /etc/letsencrypt/live/mx10.example.com/privkey.pem
    Your cert will expire on 2018-07-07. To obtain a new or tweaked
    version of this certificate in the future, simply run certbot
    again. To non-interactively renew all of your certificates, run
    “certbot renew”

One can I automate option 2 from above by adding --standalone.

certbot certonly --force-renew --standalone -d webmail.example.co.uk --pre-hook "service nginx stop" --post-hook "service nginx start"

However, removing the --force-new causes interaction, which messes up cron:

 /usr/bin/certbot certonly --standalone -d mx10.example.co.uk --pre-hook "service nginx stop" --post-hook "service nginx start" && /etc/init.d/dovecot restart && /etc/init.d/postfix restart
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator standalone, Installer None
Cert not yet due for renewal
You have an existing certificate that has exactly the same domains or certificate name you requested and isn't close to expiry.
(ref: /etc/letsencrypt/renewal/mx10.example.co.uk.conf)
What would you like to do?
1: Keep the existing certificate for now
2: Renew & replace the cert (limit ~5 per 7 days)
Select the appropriate number [1-2] then [enter] (press 'c' to cancel): ^CExiting abnormally:

How could I have it default to option 1. Keep existing cert., ?
Perhaps by adding the --non-interactive option? Gave these results:-

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator standalone, Installer None
Cert not due for renewal, but simulating renewal for dry run
Running pre-hook command: service nginx stop
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for webmail.example.co.uk
Waiting for verification...
Cleaning up challenges
Running post-hook command: service nginx start

#18

I’m a little confused about this. I would expect that once you have the hooks and everything that you expect, you could run certbot renew from cron instead of running a specific certonly command. Then certbot renew will only attempt renewal when the certificate is near expiry.


#19

Hi, No the renew always failed, which is why this thread was started. This was because of how I created the cert in the first place.

e.g:

$ certbot renew
…SNIP…
Processing /etc/letsencrypt/renewal/example.co.uk.conf
Cert is due for renewal, auto-renewing…
Could not choose appropriate plugin: The manual plugin is not working; there may be problems with your existing configuration.
The error was: PluginError(‘An authentication script must be provided with --manual-auth-hook when using the manual plugin non-interactively.’,)
Attempting to renew cert (example.co.uk) from /etc/letsencrypt/renewal/example.co.uk.conf produced an unexpected error: The manual plugin is not working; there may be problems with your existing configuration.
The error was: PluginError(‘An authentication script must be provided with --manual-auth-hook when using the manual plugin non-interactively.’,). Skipping.


#20

If you run just once with certonly --force-renew and some specific authentication method, that authentication method should be saved and used later with certbot renew for subsequent renewals. It should replace the manual authentication in your renewal configuration for that cert.

Do you have an indication that this isn’t happening? Could you also run certbot certificates just to be sure that we’re not getting confused about several separate Certbot-managed certificates on your system?