SSL cert setup for Wiki on OS X Server

Server
1

I currently have a Wildcard certificate from GoDaddy installed on an OS X Server. The only thing this server hosts that can be accessed from outside the network is the company Wiki. I’d like to replace the current wildcard cert (it’s expiring at the end of the month) with a LetsEncrypt SSL. I’ve read a few How-To guides on here, but have not been able to get the cert to download, as yet. Most recently, I have been confronted with the following errors:

When attempting to download a cert for the specific subdomain

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for companyserver.companyurl.com
Using the webroot path /Volumes/Redundancy/IT/SSL Certificates/LetsEncrypt for all unmatched domains.
Waiting for verification...
Cleaning up challenges
Failed authorization procedure. companyserver.companyurl.com (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://companyserver.companyurl.com/.well-known/acme-challenge/KYwKgFcTaYzpzcQpfPlflSKRngBGvshTJfsfT6vqEMA: "<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p"

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: companyserver.companyurl.com
   Type:   unauthorized
   Detail: Invalid response from
   http://companyserver.companyurl.com/.well-known/acme-challenge/KYwKgFcTaYzpzcQpfPlflSKRngBGvshTJfsfT6vqEMA:
   "<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
   <html><head>
   <title>404 Not Found</title>
   </head><body>
   <h1>Not Found</h1>
   <p"

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A record(s) for that domain
   contain(s) the right IP address.

When attempting to download the cert using the top domain:

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for companyurl.com
http-01 challenge for www.companyurl.com
Using the webroot path /Volumes/Redundancy/IT/SSL Certificates/LetsEncrypt for all unmatched domains.
Waiting for verification...
Cleaning up challenges
Failed authorization procedure. www.companyurl.com (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://www.companyurl.com/.well-known/acme-challenge/8tCE7dgOb6cmakgN2vgQBbj-ykF9vBoL0ihBB3-sotg: "<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> 
<html xmlns="http", companyurl.com (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://companyurl.com/.well-known/acme-challenge/VvTy7m2Y9_VJe9hibkC0PKAEyi-QcUdTQofAVSjhCSM: "<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> 
<html xmlns="http"

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: www.companyurl.com
   Type:   unauthorized
   Detail: Invalid response from
   http://www.companyurl.com/.well-known/acme-challenge/8tCE7dgOb6cmakgN2vgQBbj-ykF9vBoL0ihBB3-sotg:
   "<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
   "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
   <html xmlns="http"

   Domain: companyurl.com
   Type:   unauthorized
   Detail: Invalid response from
   http://companyurl.com/.well-known/acme-challenge/VvTy7m2Y9_VJe9hibkC0PKAEyi-QcUdTQofAVSjhCSM:
   "<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
   "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
   <html xmlns="http"
2

Hi @WilhelmSturmer,

Could you please provide the commands you are using to get the certificate?.

Are you sure the DocumentRoot for your domains is /Volumes/Redundancy/IT/SSL Certificates/LetsEncrypt ?. Note: I didn’t check it but maybe having a space in your path is not a good idea.

Anyway, It is easy to check, create a test file without extension inside /Volumes/Redundancy/IT/SSL Certificates/LetsEncrypt/.well-known/acme-challenge/ and try to reach that file with your browser or with curl from command line.

mkdir -p "/Volumes/Redundancy/IT/SSL Certificates/LetsEncrypt/.well-known/acme-challenge/"
echo -n "This is a test" > "/Volumes/Redundancy/IT/SSL Certificates/LetsEncrypt/.well-known/acme-challenge/test"

Now try to get the file:

With your browser:
http://companyserver.companyurl.com/.well-known/acme-challenge/test

Or command line:
curl -ikL http://companyserver.companyurl.com/.well-known/acme-challenge/test

And you should get the text “This is a test”, if you get any other thing, the document root could not be the right one.

Note: It would be easier to debug if you provide the real domain.

Cheers,
sahsanu

3

The directory I choose was arbitrary. It is not the root directory for the Wiki. I’m actually unsure where that directory is located. I may have to do more digging to figure out where that is, if it is required for the command to generate the certificate.

The command I was using:
sudo letsencrypt certonly --webroot -w /Volumes/Redundancy/IT/SSL\ Certificates/LetsEncrypt -d companyurl.com -d www.companyurl.com

4

It can't be arbitrary, it must be the document root for your domain, you shoud check it in your web server conf.

1 Like
5

Thanks for the info. Finding the directory root for the Wiki has been challenging.

6

It might not have one if the web server is configured to forward all requests to the wiki software. In that case, it's possible that the web server isn't serving any files directly from a location on disk. Nonetheless, it should be possible in this situation to configure the web server to do so for specific directories.

7

I figured out the root directory for the Wiki. I was going about that the completely wrong way. I was able to run the command referenced with the proper directory and it created a .pem file. However, I’ve lost the site I was referencing for how to generate and install the LetsEncrypt certificate.

A simple Google search lead me to this tutorial, but I’ve run into a new issue. After generating the two files and attempting the first, test run of get_cert.sh, terminal gives the error “Permission denied”. I don’t understand this as everything has been done under the admin user for the Mac Server. Any insight would be appreciated.

OR, if anyone can direct me on what to do with the .pem file LetsEncrypt generated, that would also be helpful.

8

Did you create get_cert.sh yourself? Did you mark it executable with chmod before trying to run it?

Text files are normally not executable on Unix, but have to be marked as executable via something like chmod +x before they are run.

Which PEM files do you have now?

How is your existing wildcard certificate configured in your web server?

9

I did create the .sh & .ini files myself. I realized after your post I made an error in the .sh file and was able to run it, though it doesn't appear to be functioning correctly. I'll provide an update to that after a bit more testing.

When running this command, referenced earlier in the thread, with the proper root folder location, one .pem file was created at /etc/letsencrypt/csr labeled 0000_csr-certbot.pem.

Currently, the GoDaddy wildcard is configured to apply to *.companytopdomain.com.

10

So, here are the contents of my cert.ini & get_cert.sh files:

cert.ini

# Use a 4096 bit RSA key instead of 2048
rsa-key-size = 4096 
# Register with the specified e-mail address
email = info@companyurl.com
# Generate certificates for the specified domains.
domains = orgsubdomain.orgtopdomain.com
# Uncomment to use a text interface instead of ncurses
# text = True
# To use the webroot authenticator.
authenticator = webroot
webroot-path = /Library/Server/Web/Data/Sites/Default

get_cert.sh

#!/bin/sh

DOMAIN_DEFAULT= orgsubdomain.orgtopdomain.com
PEM_FOLDER="/etc/letsencrypt/live/${orgsubdomain.orgtopdomain.com}/" 
LOG_FOLDER="/Users/administrator/letsencrypt/my_script/logs" 
DATE=$(date +"%d-%m-%y")
LOG_FILE="${LOG_FOLDER}/${DATE}.log" 

# Retrieve certificate - DELETE --dry-run AFTER THE TEST RUN WORKED
sudo /Users/administrator/letsencrypt/letsencrypt-auto certonly -c cert.ini --dry-run

# Check that everything went fine
LE_STATUS=$?

if [ "$LE_STATUS" != 0 ]; then
    echo Automated Get certificate failed:
    cat $LOG_FILE
    exit 1
fi
 
# Generate a passphrase - UNCOMMENT THE NEXT LINE AFTER THE TEST RUN WORKED
#PASS=$(openssl rand -base64 45 | tr -d /=+ | cut -c -30)

# Transform the pem files into a OS X Valid p12 file - UNCOMMENT THE NEXT LINE AFTER THE TEST RUN WORKED
#sudo openssl pkcs12 -export -inkey "${PEM_FOLDER}privkey.pem" -in "${PEM_FOLDER}cert.pem" -certfile "${PEM_FOLDER}fullchain.pem" -out "${PEM_FOLDER}letsencrypt_sslcert.p12" -passout pass:$PASS

# import the p12 file in keychain - UNCOMMENT THE NEXT LINE AFTER THE TEST RUN WORKED
#sudo security import "${PEM_FOLDER}letsencrypt_sslcert.p12" -f pkcs12 -k /Library/Keychains/System.keychain -P $PASS -T /Applications/Server.app/Contents/ServerRoot/System/Library/CoreServices/ServerManagerDaemon.bundle/Contents/MacOS/servermgrd

When I run get_cert.sh through the dry run, I get the following output:

server:my_script administrator$ ~/letsencrypt/my_script/get_cert.sh
/Users/administrator/letsencrypt/my_script/get_cert.sh: line 3: orgsubdomain.orgtopdomain.com: command not found
/Users/administrator/letsencrypt/my_script/get_cert.sh: line 4: /etc/letsencrypt/live/${orgsubdomain.orgtopdomain.com}/: bad substitution
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Cert not due for renewal, but simulating renewal for dry run
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for orgsubdomain.orgtopdomain.com
Using the webroot path /Library/Server/Web/Data/Sites/Default for all unmatched domains.
Waiting for verification...
Cleaning up challenges
Unable to clean up challenge directory /Library/Server/Web/Data/Sites/Default/.well-known/acme-challenge

IMPORTANT NOTES:
 - The dry run was successful.

Here is my get_cert_live.sh file. This is a copy of the get_cert.sh file with the --dry --run tag removed and the relevant lines uncommented.

get_cert_live.sh

#!/bin/sh

DOMAIN_DEFAUT= orgsubdomain.orgtopdomain.com
PEM_FOLDER="/etc/letsencrypt/live/${orgsubdomain.orgtopdomain.com}/" 
LOG_FOLDER="/Users/administrator/letsencrypt/my_script/logs" 
DATE=$(date +"%d-%m-%y")
LOG_FILE="${LOG_FOLDER}/${DATE}.log" 

# Retrieve certificate - DELETE --dry-run AFTER THE TEST RUN WORKED
sudo /Users/administrator/letsencrypt/letsencrypt-auto certonly -c cert.ini

# Check that everything went fine
LE_STATUS=$?

if [ "$LE_STATUS" != 0 ]; then
    echo Automated Get certificate failed:
    cat $LOG_FILE
    exit 1
fi
 
# Generate a passphrase - UNCOMMENT THE NEXT LINE AFTER THE TEST RUN WORKED
PASS=$(openssl rand -base64 45 | tr -d /=+ | cut -c -30)

# Transform the pem files into a OS X Valid p12 file - UNCOMMENT THE NEXT LINE AFTER THE TEST RUN WORKED
sudo openssl pkcs12 -export -inkey "${PEM_FOLDER}privkey.pem" -in "${PEM_FOLDER}cert.pem" -certfile "${PEM_FOLDER}fullchain.pem" -out "${PEM_FOLDER}letsencrypt_sslcert.p12" -passout pass:$PASS

# import the p12 file in keychain - UNCOMMENT THE NEXT LINE AFTER THE TEST RUN WORKED
sudo security import "${PEM_FOLDER}letsencrypt_sslcert.p12" -f pkcs12 -k /Library/Keychains/System.keychain -P $PASS -T /Applications/Server.app/Contents/ServerRoot/System/Library/CoreServices/ServerManagerDaemon.bundle/Contents/MacOS/servermgrd

Here is the output from that file:

server:my_script administrator$ ~/letsencrypt/my_script/get_cert_live.sh
/Users/administrator/letsencrypt/my_script/get_cert_live.sh: line 3: orgsubdomain.orgtopdomain.com: command not found
/Users/administrator/letsencrypt/my_script/get_cert_live.sh: line 4: /etc/letsencrypt/live/${orgsubdomain.orgtopdomain.com}/: bad substitution
Password:
Upgrading certbot-auto 0.14.2 to 0.15.0...
Replacing certbot-auto...
Creating virtual environment...
Installing Python packages...
The directory '/Users/administrator/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/Users/administrator/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Installation succeeded.
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Cert not yet due for renewal

You have an existing certificate that has exactly the same domains or certificate name you requested and isn't close to expiry.
(ref: /etc/letsencrypt/renewal/orgsubdomain.orgtopdomain.com.conf)

What would you like to do?
-------------------------------------------------------------------------------
1: Keep the existing certificate for now
2: Renew & replace the cert (limit ~5 per 7 days)
-------------------------------------------------------------------------------
Select the appropriate number [1-2] then [enter] (press 'c' to cancel): 2
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for orgsubdomain.orgtopdomain.com
Using the webroot path /Library/Server/Web/Data/Sites/Default for all unmatched domains.
Waiting for verification...
Cleaning up challenges
Unable to clean up challenge directory /Library/Server/Web/Data/Sites/Default/.well-known/acme-challenge

IMPORTANT NOTES:
 - Congratulations! Your certificate and chain have been saved at
   /etc/letsencrypt/live/orgsubdomain.orgtopdomain.com/fullchain.pem. Your
   cert will expire on 2017-09-06. To obtain a new or tweaked version
   of this certificate in the future, simply run letsencrypt-auto
   again. To non-interactively renew *all* of your certificates, run
   "letsencrypt-auto renew"
 - If you like Certbot, please consider supporting our work by:

   Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
   Donating to EFF:                    https://eff.org/donate-le

unable to write 'random state'
Error opening input file cert.pem
cert.pem: No such file or directory
security: Error reading infile letsencrypt_sslcert.p12: No such file or directory

As you can see, there are some errors, most importantly that the letsencrypt_sslcert.p12 file isn't created. The article states that if this file isn't created, I probably didn't provide the right domain in the .sh file. However, the domain provided is the only domain that will work. A wild card domain is not optionable, and I can not use my top domain, as it isn't hosted on this server. The subdomain provided is specific to this server and the Wiki hosted on it.

11

That file is not really useful for anything. :slight_smile: For present purposes, it can be seen as an intermediate temporary file that Certbot keeps around permanently for historical reasons and probably shouldn't. In fact, I've suggested that Certbot be modified to stop saving these files.

12

I’ll try to comment on problems with your script in a bit.

1 Like
13

The first problem which may be responsible for a lot of other problems is that you didn’t modify the script the way it was intended. They wanted you to literally fill in your domain name as the value of DOMAIN_DEFAUT [sic], as in

DOMAIN_DEFAUT=example.com

The ${DOMAIN_DEFAUT} is a Bourne shell variable substitution, which fills in the value of the variable DOMAIN_DEFAUT. If you say ${example.com}, you are asking for the value of the variable example.com, not the string “example.com”. However, shell variable names are not allowed to contain a period. That is the reason for the bad substitution error. Then, the paths used by later commands are wrong because the value of the PEM_FOLDER variable could not be constructed.

14

So, you definitely talked above my head a bit, however, I went back and edited the two .sh files based on your input. One error I noticed is that I had a space in front of the domain. Here is the edited live .sh file.

get_cert_live.sh

#!/bin/sh

DOMAIN_DEFAUT=orgsubdomain.orgtopdomain.com
PEM_FOLDER="/etc/letsencrypt/live/${DOMAIN_DEFAUT}/" 
LOG_FOLDER="/Users/administrator/letsencrypt/my_script/logs" 
DATE=$(date +"%d-%m-%y")
LOG_FILE="${LOG_FOLDER}/${DATE}.log" 

# Retrieve certificate - DELETE --dry-run AFTER THE TEST RUN WORKED
sudo /Users/administrator/letsencrypt/letsencrypt-auto certonly -c cert.ini

# Check that everything went fine
LE_STATUS=$?

if [ "$LE_STATUS" != 0 ]; then
    echo Automated Get certificate failed:
    cat $LOG_FILE
    exit 1
fi
 
# Generate a passphrase - UNCOMMENT THE NEXT LINE AFTER THE TEST RUN WORKED
PASS=$(openssl rand -base64 45 | tr -d /=+ | cut -c -30)

# Transform the pem files into a OS X Valid p12 file - UNCOMMENT THE NEXT LINE AFTER THE TEST RUN WORKED
sudo openssl pkcs12 -export -inkey "${PEM_FOLDER}privkey.pem" -in "${PEM_FOLDER}cert.pem" -certfile "${PEM_FOLDER}fullchain.pem" -out "${PEM_FOLDER}letsencrypt_sslcert.p12" -passout pass:$PASS

# import the p12 file in keychain - UNCOMMENT THE NEXT LINE AFTER THE TEST RUN WORKED
sudo security import "${PEM_FOLDER}letsencrypt_sslcert.p12" -f pkcs12 -k /Library/Keychains/System.keychain -P $PASS -T /Applications/Server.app/Contents/ServerRoot/System/Library/CoreServices/ServerManagerDaemon.bundle/Contents/MacOS/servermgrd

Another issue I found is that, based on the script, the cert.ini file was in the wrong location. I made a copy of the file and placed it in the location specified by line 10 of the script.

Finally, I've noticed that no log files are being generated.

Here's the output of the above .sh file:

server:~ administrator$ ~/letsencrypt/my_script/get_cert_live.sh
Password:
usage: 
  letsencrypt-auto [SUBCOMMAND] [options] [-d DOMAIN] [-d DOMAIN] ...

Certbot can obtain and install HTTPS/TLS/SSL certificates.  By default,
it will attempt to use a webserver both for obtaining and installing the
certificate. 
certbot: error: File not found: cert.ini
Automated Get certificate failed:
cat: /Users/administrator/letsencrypt/my_script/logs/09-06-17.log: No such file or directory

Based on this output, the script can't seem to find cert.ini, but I'm unsure why.

15

The trouble is that the location specified by line 10 is the current working directory. :slight_smile:

The shell command

sudo /Users/administrator/letsencrypt/letsencrypt-auto certonly -c cert.ini

does not mean that cert.ini is in /Users/administrator/letsencrypt. It means that the program letsencrypt-auto is in /Users/administrator/letsencrypt. The command to specify that cert.ini is in /Users/administrator/letsencrypt would be

sudo /Users/administrator/letsencrypt/letsencrypt-auto certonly -c /Users/administrator/letsencrypt/cert.ini

Every unqualified Unix file name, meaning one that doesn't explicitly specify a directory, refers to a file in the current working directory. From the point of view of someone writing or running a shell script, that's the directory that you were in in the shell at the moment when you ran the script (unless the script or another program explicitly changes into a different directory, which isn't the case here).

When you're running that script, you appear to be in your home directory (abbreviated ~) and so that's where the cert.ini file is expected to be. You could change the script to indicate that it's somewhere else if you have somewhere that you'd prefer for it to be.

I'm kind of confused about the LOG_FILE. As far as I can tell, this is a bug in the script from the tutorial because nothing is ever done to set up logging into this file, so there's no way it could exist or contain logs. I don't think that's a result of anything that you did.

Normally Certbot itself is going to log into /var/log/letsencrypt, and it doesn't seem that the script has attempted to change this.

1 Like
16

You can find the full name of the current directory at any moment by running pwd.

1 Like
17

You have been most helpful and educating. I’ll make the necessary changes and update on my progress.

18

@schoen So, the dry run .sh file seems to be working.

server:~ administrator$ cd /Users/administrator/letsencrypt/my_script
server:my_script administrator$ ~/letsencrypt/my_script/get_cert.sh
Password:
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for orgsubdomain.orgtopdomain.com
Using the webroot path /Library/Server/Web/Data/Sites/Default for all unmatched domains.
Waiting for verification...
Cleaning up challenges
Unable to clean up challenge directory /Library/Server/Web/Data/Sites/Default/.well-known/acme-challenge

IMPORTANT NOTES:
 - The dry run was successful.

I see that "Unable to clean..." statement every time I run either .sh file. Should I be concerned about that?

19

I believe this means that you have an old challenge file in that directory and so Certbot is unwilling to delete the directory.

This doesn’t harm anything, but you could probably get rid of the message with

sudo rm -i /Library/Server/Web/Data/Sites/Default/.well-known/acme-challenge/*

1 Like
20

Now I feel silly... that file is the test.html.

1 Like