Certbot --help incomprehensible

Help
#1

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. https://crt.sh/?q=example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: davekimble.net

I ran this command: certbot certonly -d mail.davekimble.net

It produced this output:
Challenge failed for domain mail.davekimble.net
http-01 challenge for mail.davekimble.net
Cleaning up challenges
Some challenges have failed.

IMPORTANT NOTES:

My web server is (include version): Apache 2.4.41

The operating system my web server runs on is (include version): Ubuntu 20.04

My hosting provider, if applicable, is: at home

I can login to a root shell on my machine (yes or no, or I don't know): yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): 0.40.0

I have 10 years experience with LINUX, but I find your "certbot --help" to be absolutely incomprehensible. This definitely needs some examples, and an overview paragraph. The messages output during running are also incomprehensible. What does "spin up a webserver" mean and what does it do?

2 Likes
#2

Hi Dave, and welcome to the LE community forum!

I can't do anything about the --help messages, but I can try to help you get a cert.
It seems that your system is requiring authentication to reach the challenge files:

curl -Iki http://mail.davekimble.net/.well-known/acme-challenge/yNcKgZzdTRNfYTVHO9bzPKAUTIPuCYluDfDfZzjezcI
HTTP/1.1 403 Forbidden
Date: Sat, 28 Nov 2020 06:38:07 GMT
Server: Apache/2.4.41 (Ubuntu)
Content-Type: text/html; charset=iso-8859-1

This will undoubtedly break the certificate authentication process.

Before continuing testing with the LE production system, I suggest you review the vhost config settings and place a sample test file in the expected challenge location to ensure it can be reached from the Internet without requiring to login/authenticate.
Once the test file can be reached, you can resume your LE testing with the --staging environment.
Once you are able to obtain a cert from the --staging environment, you can then switch to production.
Once you have switched to production, you can automate the renewal process.
If you then need to test the automated renewal process, you should do so against production [if the renewals should fail, you should then switch back to --staging and perhaps open another topic here for that discussion]

3 Likes
#3

Thank you for your assistance.
There was no yNcKgZzdTRNfYTVHO9bzPKAUTIPuCYluDfDfZzjezcI file in /var/www/html/, no /acme-challenge folder, no /.well-known folder, and nothing told ME to create them. I thought YOU were going to create them. It's your Apache2 after all.

What is the " --staging environment" ? Where do I find out?
Just some EXAMPLES would help.

1 Like
#4

Welcome to the Let's Encrypt Community, Dave :slightly_smiling_face:

We're more than happy to help. As many around here can tell you, I'm not the biggest fan of certbot's current documentation. That said, there are literally thousands of searchable examples (of a sort) in this community.

"Spinning-up a webserver" just means "running a webserver process to listen for connections". This is useful when you either don't have a functioning webserver (a.g. apache, nginx, IIS) or you stop your current webserver because it doesn't play well with certbot. This is accomplished with the standalone authenticator of certbot. If you have a webserver already running, you could instead use the webroot authenticator and specify the webroot directory of your website with -w. There are also specific apache and nginx authenticators with corresponding installers to install your certificate (i.e. add the appropriate lines to your configuration files).

You can read more in the official user guide:

We check for that structure because that's how Let's Encrypt expects it to be. The authentication filename(s) will change though. I usually check /.well-known/acme-challenge/test and hope for a 404 Not Found. You can use Let's Debug as a starting point.

Apache2 is your webserver software.

Much further reading can be found here:

3 Likes
#5

Are you trying to secure a mail server or a webpage to access your mail? The http-01 challenge (via /.well-known/acme-challenge/) expects a response via port 80 (http), which is fine for a webserver, but not usually the correct port for a mail server. You can forward the Let's Encrypt requests from port 80 to port 443 (https), but the process must start at port 80. Keep in mind, I'm only referring to the authentication process to acquire a certificate. Once you have acquired your certificate, you are welcome to use it with whatever port you wish. What I'm getting-at is: make sure a webserver is going to answer on port 80 to facilitate acquiring your certificate. You can't expect your smtp server to do this. You could use a dns-01 challenge instead, but it's usually not as friendly and much more difficult to automate renewals.

2 Likes
#6

Thanks all round.
Despite my please for EXAMPLES, there are still no examples given.. You have all ignored the bit where I said I can't understand the certbot help, and talked about things where I have to know how to call certbot.

I have Apache running on my production server. It has /.well-known/acme-challenge/ . I want a Letsencrypt certificate and private key for a mailserver I am writing, on my development computer of course. Can I put the token file on the production server, and get the .crt.pem and pkey.pem files in filestore somewhere, and then copy them across ? Please give an example of how I could do this, not a reference to documentation for experts, which I can't understand.

How do I get hold of these token files? There was no sign of them when I tried various ways. Again please give example.

#7

Probably because there is pretty good documentation to read first. After understanding how Let's Encrypt (more specifically: the ACME protocol) works, there's a good chance you'll understand certbot better.
Further more, a link to the certbot documentation page has been given. That page has multiple examples among all the documentation about the different features of certbot.

So I'm not really sure where your demand for examples comes from, as by that I'm assuming you haven't read the certbot documentation site yet. Please read that first.

4 Likes
#8

Happy birthday, Osiris!

3 Likes
#9

Thanks Jim! My fifth year on this Community already :slight_smile:

4 Likes
#10

And you don't have to.
certbot will create it and then delete it when finished.

I don't understand this statement.
I am NOT certbot.
I don't write their code.
I don't work for LE nor ISRG.
I am a volunteer here.
Just another fellow user trying to help those that need help.
As with most of the replies you will receive, keep in mind that they come from users - just like YOU.

You've been using Linux 10 years and you don't know who's Apache it is?
It belongs to the ADMIN - presumably YOU.

My first suggestion was:

Have you done that? NO? How did I guess?
Please show the output of this command:
sudo apachectl -S

From there we can find out exactly which file is responsible for the FQDN in question.
But let's not get ahead of ourselves...
We are here to help and will walk YOU through this one step at a time (if needed)
[with or without working examples or comprehensible --help output]

5 Likes
#11

@davekimble2

I also suspect issue(s) exists within your apache and possibly your Ubuntu configuration(s).

When you ran certbot the folder structure was created automatically by the process.
find /var/www/html/ -type d -name ".well-known" will demonstrate that unless the folder was manually removed.

When I browse to http://mail.davekimble.net/.well-known/ I am served a "403 Forbidden" error, not a 404 Not Found which would be expected.

This suggests a configuration issue exists that should be corrected to achieve your goal.

It seems that you have implied that you are looking for a "cut and paste" solution, which in the end will come back to haunt you.

Someone I have come to respect greatly on this forum (although I have never bought him a beer) once stated:

Whenever I come across something I don’t understand, I try to read about it and learn about it until I understand it.
Happy Birthday @Osiris!

So if you resist the documentation(s) provided because they are "incomprehensible" or otherwise lacking, just know that LOTS of folks have raised this issue and @griffin has taken on the task of fixing it. (This will take a bunch of his volunteered time)

In the meantime please contribute information to the people who are only trying to help.
@rg305 asked for some info in the post previous to mine, and IMHO that is the first step in resolving this thread.

3 Likes
#12

@davekimble2

The token comes from the Let's Encrypt server and is written by certbot during the authentication process. This is why you need to run certbot on the machine responding for the domain name being certified.

My guess would be, based on the information you've provided , your mail server certbot command will look something like this:

sudo certbot run --apache -d "mail.davekimble.net" --keep

Now, given the extensive Let's Encrypt certificate history of davekimble.net that dates back 4 years and having performed a dig to ensure that davekimble.net and mail.davekimble.net have the same IP address, I would personally recommend using the following certbot command to create a compound certificate:

sudo certbot run --cert-name davekimble.net --apache -d "davekimble.net,www.davekimble.net,mail.davekimble.net" --keep

After that, you should have an automatic process setup to run certbot renew daily to keep your certificate from expiring. The renew command, like adding --keep will only update your certificate when it is near expiry (within 30 days). You can run with either of those as often as you like without being concerned about hitting the duplicate certificate rate-limit.

1 Like
#13

Two things @griffin:

  1. Using run will result in certbot trying to install the certificate into a virtualhost containing the mail subdomain, which might not be available (please, no discussion about the poorly chosen name of the "run" command.…);
  2. --keep only has any effect when there already is a certificate with the same set of hostnames. As the set of hostnames has changed, --keep isn't relevant here and won't do anything. Perhaps a more logical option is --expand here.
2 Likes
#14

You make a fair point here. Hmm... :thinking: Certbot basically assumes that the vHosts will be for http and not smtp.

The idea here is "training wheels". If the certificate is successfully acquired, running the command again will just attempt to reinstall the existing certificate rather than acquire a duplicate. Keep in mind that --cert-name is more powerful than --expand because we know exactly which certificate will end up being the active one.

2 Likes
#15

Do we even know what the cert will be used for?

  • SMTP only
  • Web Mail only
  • SMTP + Web Mail
2 Likes
#16

Not sure, honestly. With my mail I have both with two different ports. If it's a a webmail access then the port 443 SSL will be important.

1 Like
#17

You're suggesting to use the apache plugin, also as installer plugin, so yes, it looks for an Apache virtualhost. An option to prevent this might be by using -a apache in stead of --apache. I think this would result in certbot choosing the "null" installer plugin, which of course wouldn't try to install anything.

Users shouldn't run a command a second time if it succeeded previously anyway.. Also, without --keep the user would have to purposely choose to issue a new cert again in stead of reinstalling it.
You can only prevent users doing stupid things that much by adding more and more options.. In the end they'll always find a way to mess things up. I'm personally enclined not to "pamper" the users too much :wink:

Not a clue, I mainly got the idea @davekimble2 came here to complain about the --help output..

3 Likes
#18

In all fairness that is a valid complaint.

2 Likes
#19

Do children in Holland not run into the street? :wink:

With --keep the user can't choose to issue a duplicate certificate. That's the whole point. I don't believe they're given the option. I may be wrong. It's a "latch" in my world.

2 Likes
#20

Survival of the "fittest" - the ones that did took that trait to the grave - LOL

2 Likes