Domain Server how-to


#1

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. https://crt.sh/?q=example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is:huntrods.com

I ran this command: ./letsencrypt-auto certonly --standalone -d huntrods.com

It produced this output: (can’t copy / paste but essentially: Invalid response from http://huntrods.com/.well-known/acme-challenge/

My web server is (include version): Apache 2.2 (I think - the default with OpenBSD 5.1)

The operating system my web server runs on is (include version): OpenBSD 5.1

My hosting provider, if applicable, is:me … I built my own server - OpenBSD 5.1 running Apache 2.2 (I think). the httpd.conf has deny all on all directories currently.

I can login to a root shell on my machine (yes or no, or I don’t know):yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel):no

Basically, I know that I have to modify Apache’s httpd.conf file to enable the Acme challenge-response to be able to write a file on my server. But there is no guide as to how to do this.


#2

Hi @sunnyboy,

If you expect to complete the challenge by serving a static file from an existing web server, than --standalone is not the right method. The --standalone method is intended for machines that don’t have an existing web server. You should use --webroot (if you have a location on disk that serves static files) or --apache (if you want Certbot to try to modify your Apache configuration in order to complete the challenges and install the certificate itself).


#3

You will have to create the challenge directory and accept requests to it.
Using --webroot is probably the simplest first test.


#4

I’ve done some reading and created the directory .well-known and acme-challenge off -webroot. I’ve modified httpd.conf settings so I can see it from a browser, but the actual command ./letseencrypt-auto certonly --webroot -d huntrods.com
still fails (I specify / as webroot when asked).
"Invalid response from http://huntrods.com/.well-known/acme-challenge/

My httpd.conf lines I added are:
<Directory “/home/huntrods/public_html/.well-known”>
Options Indexes FollowSymLinks MultiViews
AllowOveride All
Order allow,deny
Allow from all

So far internet searches aren’t helping. Again, with the above changes to httpd.conf, I can see a directory listing with http://huntrods.com/.well-known/acme-challenge


#5

Try making that:
<Directory “/home/huntrods/public_html/.well-known/acme-challenge”>
[create full directory path if it doesn’t exist]

Then also, place a simple test file in that folder to reach from Internet.
Like: http://huntrods.com/.well-known/acme-challenge/1234


#6

Also, this command doesn’t specify the webroot path.
(and maybe add the “www” fqnd as well)

Try:
./letseencrypt-auto certonly --webroot -w /home/huntrods/public_html -d huntrods.com -d www.huntrods.com


#7

OK. I changed the <Directory to include acme-challenge as you suggested, and also put a test file 1234 in the directory.
Trying http://huntrods/com/.well-known/acme-challenge/1234 displays the contents of that file.
My domain is huntrods.com, no www.
For --webroot, I specified / as in

./letseencrypt-auto certonly --webroot -w / -d huntrods.com -d www.huntrods.com

If I use ./letseencrypt-auto certonly --webroot -w /home/huntrods/public_html -d huntrods.com -d www.huntrods.com
then letsencrypt complains that the path does not exist (which it doesn’t as the virtualhost section in httpd.conf already maps this to ‘/’.)

But… it still fails with the same error. It’s trying to write this random named file, which it can’t write for some reason, or so it seems. I can’t just create the file because the random filename changes each time.

Thanks for the assistance so far, but I’m still stuck with the same error message.


#8

Oh yes, one more thing. I made the owner and group of acme-challenge www:www since that’s ‘who’ is actually owner of the httpd process.

http://huntrods.com/.well-known/acme-challenge/1234

does show the contents of the file, and the directory permissions are 755 for www:www.


#9

[using / is NOT a good idea]
If your have no other choice, then the challenge token should be found at:
/.well-known/acme-challenge/
create that folder and try it again.

You should at least try something like -w /test
and create the required:
/test/.well-known/acme-challenge/


#10

/ is equivalent to /home/huntrods/public_html.

Checking my logs, here’s what letsencrypt is sending and receiving:

“GET /.well-known/acme-challenge/qAN6euLXpA2WToTwf4VrK6vFvYvD3qQvLsSkVX680Ns HTTP/1.1” 404 276

/ is not my server root, but rather the real webroot for this user.

For example, if I open a telnet session to huntrods.com on port 80, and type
GET /.well-known/acme-challenge/1234
I get the proper response of “abcd” (the contents of the file).
So it’s working, but when letsencrypt asks for this random filenamed file, it isn’t there. I guess the question now is … how do I get myserver to create the file with the long random name that letsencrypt is asking for via GET?


#11

This relies on the document root statement in the web service.
--w /path does NOT follow that; it should go to “/path”

Certbot is NOT part of the web service (when --webroot is used).
Certbot relies on the web service to give external access to the location provided to certbot ("/path").
The web service is unmodified and knows nothing about “/path” and will always serve “/.well-known/acme-challenge/1234” as $document_root/.well-known/acme-challenge/1234
Those are now two different locations.


#12

If I give letsencrypt a document root (-w) of /home/huntrods/public_html, letsencrypt fails immediately with
/home/huntrods/public_html does not exist or is not a directory.
If I use a document root of / , the command runs but fails because the file didn’t get created.
I’m now trying to figure out how to get the file created.


#13

This works:
http://huntrods.com/.well-known/acme-challenge/1234
Where is the ‘1234’ file located?
That file location should be the --w “path”.
If certbot puts anything in that folder the world should see it at:
http://huntrods.com/.well-known/acme-challenge/{whatever.it.put}


#14

1234 is physically /home/huntrods/public_html/.well-known/acme-challenge/1234
but resides relative to docroot as /.well-known/acme-challenge/1234

At the moment, nothing is in that folder after I run letsencrypt except the 1234 file which I manually put there.
I thought it might be a permissions problem, but the owner of acme-challenge is www, and permissions 755.


#15

Then the webroot must be:

or
/home/huntrods/public_html/

Is docroot a global variable?
Can certbot see docroot or is it just visible inside the web service?

Have you tried running letsencrypt-auto with sudo (or whichever way as root)?


#16

I’m not sure, possibly just visible inside the apache web server. The httpd.conf creates several VirtualHost entities (it’s serving multiple domains on different static IPs) and the VirtualHost entry in httpd.conf for this domain specifies:

<VirtualHost 66.18.225.244>
DocumentRoot /home/huntrods/public_html
ServerName huntrods.com
<Directory “/home/huntrods/public_html/.well-known/acme-challenge”
Options Indexes FollowSymLinks
AllowOverride None
Order allow,deny
Allow from all

(weird. the editor is removing the end tags for Directory and VirtualHost

Looking at the httpd.conf, ServerRoot is /var/www/" and there is a htdocs directory there, but it’s not tied to huntrods.com due to the virtualhost item above


#17

For:
ServerName huntrods.com
The document root is:
DocumentRoot /home/huntrods/public_html
So that is the --w /path that needs to be used:
--w /home/huntrods/public_html

But the challenge file will be placed in
/home/huntrods/public_html/.well-known/acme-challenge
So don’t delete nor change that folder.


#18

OK. I see part of the problem.
When I type ./letsencrypt-auto certonly --webroot -w /home/huntrods/public_html -d huntrods.com -d www.huntrods.com --test
I get the instant response:
/home/huntrods/public_html does not exist or is not a directory

So I thought… what if it’s looking for this directory on the current local machine?
So - I created the directory /home/huntrods/public_html and re-ran the command

Sure enough, it runs partially: that is, it runs but gives the same error message as it’s always given:

./letsencrypt-auto certonly --webroot -w /home/huntrods/public_html -d huntrods.com -d www.huntrods.com --test

Saving debug log to /var/log/letsencrypt/letsencrypt.log Plugins selected: Authenticator webroot, Installer None Obtaining a new certificate Performing the following challenges: http-01 challenge for huntrods.com http-01 challenge for www.huntrods.com Using the webroot path /home/huntrods/public_html for all unmatched domains. Waiting for verification... Cleaning up challenges Failed authorization procedure. www.huntrods.com (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://www.huntrods.com/.well-known/acme-challenge/ypHWrSAjwCfsGKpbF15ibxkbQamZqkf7x2yLHjjAzN4: Not found:huntrods.com (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://huntrods.com/.well-known/acme-challenge/_863KkBhgMI7CngL_ReEI94CIjYIidf_DhpGvqgz9xQ: Not Found IMPORTANT NOTES: - The following errors were reported by the server:

Domain: www.huntrods.com
Type: unauthorized
Detail: Invalid response from
http://www.huntrods.com/.well-known/acme-challenge/ypHWrSAjwCfsGKpbF15ibxkbQamZqkf7x2yLHjjAzN4:
Not found
Domain: huntrods.com
Type: unauthorized
Detail: Invalid response from
http://huntrods.com/.well-known/acme-challenge/_863KkBhgMI7CngL_ReEI94CIjYIidf_DhpGvqgz9xQ:
Not found
To fix these errors, please make sure that your domain name was
entered correctly and the DNS A/AAAA record(s) for that domain
contain(s) the right IP address.

But here's the problem I've identified. I am running this command on a different machine, NOT the domain machine and NOT the machine running Apache as huntrods.com.

Here’s what I was doing:

  • on a local PC (inside the huntrods domain but a local IP) I created a virtualbox Ubuntu 18.04.1 machine. That’s the machine I want to install JupyterHub on, so that’s the machine that I wanted the cert and key to be sitting on.
  • the domain and Apache server are on a totally different box, running OpenBSD and Apache. That’s the domain server that I’m trying to contact to get the cert authorized. That’s also where the official ‘/home/huntrods/public_html’ directory resides, and the one that Apache on huntrods.com uses.

So - two totally different machines. I’m wanting my domain machine at huntrods.com to verify that I’m “me” and issue the cert to the virtual box machine so I can use it for JupyterHub.


#19

In https://certbot.eff.org/docs/intro.html, we say

Certbot is meant to be run directly on your web server, not on your personal computer. If you’re using a hosted service and don’t have direct access to your web server, you might not be able to use Certbot. Check with your hosting provider for documentation about uploading certificates or using certificates issued by Let’s Encrypt.

You can do what you’re trying to do via Certbot’s --manual-auth-hook feature if you can write an appropriate shell script to upload the challenge files to the right place; otherwise, Certbot is probably not suitable for you at all. You might consider GetSSL’s for its “remote webroot” feature.


#20

Thanks. This is awesome news, mostly because it confirms that while I was trying to do the wrong thing, I wasn’t totally crazy! :wink:

I will definitely look into the --manual-auth-hook feature. I should be able to do this now that I know what’s going on.

Just to provide some background for this thread… what I’m trying to do is create a JupyterHub server for testing purposes. I really can’t play on my OpenBSD primary server, and I’ve already created/deleted several Ubuntu servers on my development PC while getting the hang of Jupyter (i.e. JupyterLab for one user on Ubuntu 18.10 virtualbox, then a few others, and now JupyterHub on 18.04.)

Ultimately I’m hoping to get JupyterHub on AWS (Amazon Web Services) for my university course, but I really ‘need’ to understand what I’m doing first. The current primary goal is to get JupyterHub running on this virtual PC but accessible from outside my local intranet. I already have other servers ‘forwarded’ by my firewall, so this should be doable. But… Installing JupyterHub - all the documentation says you need the cert so that you can run secure. As the ultimate goal is secure on AWS, this is a necessary step.

Again, thanks to all who have been patient with me so far.