First I created the certificate, this worked fine (after I stopped nginx), except that the instructions have a mistake in that they say do letsencrypt certonly when obviously in fact you must do sudo letsencrypt certonly. Apart from that it was all good.
Then I tried to test the renewal process as (almost) suggested by the instructions: sudo letsencrypt renew --dry-run. This produced a warning and an apparently ‘fatal warning’ (!):
WARNING:letsencrypt.client:Registering without email! (Not true, I did provide an email address when I created the certificate.)
WARNING:letsencrypt.cli:Attempting to renew cert from /etc/letsencrypt/renewal/example.conf produced an unexpected error: Missing command line flag or config entry for this setting: Please read the Terms of Service at https://letsencrypt.org/documents/LE-SA-v1.0.1-July-27-2015.pdf. You must agree in order to register with the ACME server at https://acme-staging.api.letsencrypt.org/directory (Not true, I did agree to the ToS when I created the certificate.)
I can of course add the --agree-tos command-line parameter, but if that’s supposed to be required then clearly the instructions are wrong. If the instructions aren’t wrong then the program is. Either way I still get the ‘missing email’ warning.
Finally, if I do run it with --agree-tos then it fails instead because nginx is of course using TCP/80. I understand that perhaps automated renewal is not supposed to be possible currently with nginx, but if so the instructions at certbot.eff.org are wrong to imply that it should work.
Also of course taking the web server down as suggested twice a day every day to renew (or check renewal) for certificates is unlikely to be acceptable for most users. Is there some reason the standalone auth code doesn’t pick a random available port <1024 for doing the verification?
A random port won’t work. ACME currently offers three possible validation choices
Respond to a HTTP (not HTTPS) GET of /.well-known/acme-challenge/stuffChosenByTheACMEprotocol correctly on port 80 of the host to be validated
Respond to a fake HTTPS connection that does SNI for an invalid hostname in a particular way on port 443 of the host to be validated.
Provision a DNS TXT record in your domain with specified contents
The Standalone mode can do the first two of those options by itself, but as you’ve observed it can’t do this while you have a real web server running. “Webroot” mode lets you achieve the first option, if you can configure a place on your filesystem from which such a response will be served, it will create the file needed, tell the Let’s Encrypt servers it’s in place and wait for them to check it’s OK. You will still need to arrange to any HTTP server reload or whatever.
DNS mode is good if you’ve got programmatic control of DNS for an organisation, but the certbot software doesn’t (last I checked) offer this mode so you need to look for third party scripts that show how it’s done.
The instructions suggest using the webroot plugin if you're running a web server:
This will allow you interactively select the plugin and options used to obtain your certificate. If you already have a webserver running, we recommend chosing the "webroot" plugin.
That's more than a little unclear - these instructions are for people who have already explicitly indicated that they have a nginx running, and yet the instructions aren't suitable for those people.
I’m not sure I follow. The phrasing could be better, but it does indicate that you get to pick the plugin (“interactively select the plugin”) and that webroot would be the best option if you’re running a web server.
Ah, in that case the problem is that the instructions are wrong: you don't get to interactively select the plugin. It asks your email, it asks to agree to ToS, it asks the domain name. It doesn't ask for any plugin choices.
Ok I’ve tried with certonly --webroot and it does not work interactively:
The webroot plugin is not working; there may be problems with your existing configuration. The error was: PluginError('Missing parts of webroot configuration; please set either --webroot-path and --domains, or --webroot-map. Run with --help webroot for examples.',)
It does work if I specify the document root and domain name manually on the command-line.
However renew still does not work inasmuch as it still wrongly thinks that (a) there has been no email address specified and (b) that the ToS have not been agreed to. If I do --dry-run --agree-tos then it looks like it works (but still has the warning about no email).
Looks like you're right! Best I can tell, the client version shipped by Ubuntu does not offer interactive plugin selection (nor interactive webroot mode). I've created an issue to update the website:
I did some testing and managed to reproduce this at some point, but I'm not entirely sure what caused it. If I had to guess, I'd say the client somehow fails to persist those settings if the first attempt of getting a certificate fails, while the ACME account creation succeeds. At least that's the only difference I can think of between my two attempts - both on a clean installation of Xenial. You could try setting the email value in your .conf file in /etc/letsencrypt/renewal manually.
I think there were some changes to how these things are stored in more recent client versions, so it's quite possible this is something that's already been fixed outside of the Xenial package.
Ok cool. Thanks for looking into that. Yes setting email and tos in the .conf file looks like it works - there must be some bug that means that file gets created wrong initially. But unfortunately the Ubuntu version seems to not have the quiet and post-hook options so automatic renewal will still be a bit tricky (do errors reliably go out on stderr and other noise on stdout?). I don’t suppose letsencrypt renew does anything helpful like return a non-zero return code unless at least one certificate was renewed? Also the documentation seems to imply that the script auto-updates, but obviously it isn’t doing that?
The Ubuntu package is probably stuck at the current version for good. Not entirely sure what the policy is, but I think only fixes for security issues and major bugs are backported. The auto-update bit in the documentation is about letsencrypt-auto/certbot-auto and doesn’t apply to packaged versions.
Your best option might actually be to switch to certbot-auto, especially if you want to use renewal hooks and such. The instructions for Trusty work on Xenial. I think a PPA for certbot is also being worked on, but don’t quote me on that.
