Redirect on macOS from http to https makes certbot fail

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. https://crt.sh/?q=example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is:
blindpromo.com

I ran this command:

#!/bin/sh

DOMAIN_NAME=“blindpromo.com

HTDOCS_ROOT="/Library/WebServer"
PEM_FOLDER="/etc/letsencrypt/live/{DOMAIN_NAME}/" LOG_FOLDER="/Users/leuser/letsencrypt/my_script/logs" DATE=(date +"%y-%m-%d")
LOG_FILE="{LOG_FOLDER}/{DATE}.log"

PATH=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Applications/Server.app/Contents/ServerRoot/usr/bin:/Applications/Server.app/Contents/ServerRoot/usr/sbin

create domain specific log dir

------------------------------

[ -e {LOG_FOLDER}/{DOMAIN_NAME} ] || mkdir -p {LOG_FOLDER}/{DOMAIN_NAME}

Retrieve certificate

--------------------

sudo certbot certonly --logs-dir {LOG_FOLDER}/{DOMAIN_NAME} --webroot -w {HTDOCS_ROOT}/{DOMAIN_NAME}/htdocs -n -d {DOMAIN_NAME} -d www.{DOMAIN_NAME}

Check that everything went fine

-------------------------------

LE_STATUS=$?

if [ “$LE_STATUS” != 0 ]; then
echo Automated Get certificate failed:
cat $LOG_FILE
exit 1
fi

Generate a passphrase

---------------------

PASS=$(openssl rand -base64 45 | tr -d /=+ | cut -c -30)

Transform the pem files into an OS X / macOS Valid p12 file

-----------------------------------------------------------

sudo openssl pkcs12 -export -inkey “{PEM_FOLDER}privkey.pem" -in "{PEM_FOLDER}cert.pem” -certfile “{PEM_FOLDER}fullchain.pem" -out "{PEM_FOLDER}letsencrypt_sslcert.p12” -passout pass:$PASS

import the p12 file into the OS X / macOS keychain

--------------------------------------------------

sudo security import “${PEM_FOLDER}letsencrypt_sslcert.p12” -f pkcs12 -k /Library/Keychains/System.keychain -P $PASS -T /Applications/Server.app/Contents/ServerRoot/System/Library/CoreServices/ServerManagerDaemon.bundle/Contents/MacOS/servermgrd

It produced this output:

Domain: blindpromo.com
Type: connection
Detail: Fetching http://blindpromo.com/.well-known/acme-challenge/MR24YudjX3OREjqnUcG2tevfDCU-vXPL7ImQWt4vJeQ: Connection refused

Domain: www.blindpromo.com
Type: connection
Detail: Fetching http://www.blindpromo.com/.well-known/acme-challenge/3zQ0dwExqHxGBMpLKrSuYbkzIMBrLVXVwXrd3Mh1vMA: Connection reset by peer

To fix these errors, please make sure that your domain name was entered correctly and the DNS A/AAAA record(s) for that domain contain(s) the right IP address. Additionally, please check that your computer has a publicly routable IP address and that no firewalls are preventing the server from communicating with the client. If you’re using the webroot plugin, you should also verify that you are serving files from the webroot path you provided.
2019-04-15 10:51:25,025:DEBUG:certbot.error_handler:Encountered exception:
Traceback (most recent call last):
File “/usr/local/Cellar/certbot/0.30.2/libexec/lib/python3.7/site-packages/certbot/auth_handler.py”, line 82, in handle_authorizations
self._respond(aauthzrs, resp, best_effort)
File “/usr/local/Cellar/certbot/0.30.2/libexec/lib/python3.7/site-packages/certbot/auth_handler.py”, line 161, in _respond
self._poll_challenges(aauthzrs, chall_update, best_effort)
File “/usr/local/Cellar/certbot/0.30.2/libexec/lib/python3.7/site-packages/certbot/auth_handler.py”, line 232, in _poll_challenges
raise errors.FailedChallenges(all_failed_achalls)
certbot.errors.FailedChallenges: Failed authorization procedure. blindpromo.com (http-01): urn:ietf:params:acme:error:connection :: The server could not connect to the client to verify the domain :: Fetching http://blindpromo.com/.well-known/acme-challenge/MR24YudjX3OREjqnUcG2tevfDCU-vXPL7ImQWt4vJeQ: Connection refused, www.blindpromo.com (http-01): urn:ietf:params:acme:error:connection :: The server could not connect to the client to verify the domain :: Fetching http://www.blindpromo.com/.well-known/acme-challenge/3zQ0dwExqHxGBMpLKrSuYbkzIMBrLVXVwXrd3Mh1vMA: Connection reset by peer

2019-04-15 10:51:25,025:DEBUG:certbot.error_handler:Calling registered functions
2019-04-15 10:51:25,026:INFO:certbot.auth_handler:Cleaning up challenges
2019-04-15 10:51:25,026:DEBUG:certbot.plugins.webroot:Removing /Library/WebServer/blindpromo.com/htdocs/.well-known/acme-challenge/MR24YudjX3OREjqnUcG2tevfDCU-vXPL7ImQWt4vJeQ
2019-04-15 10:51:25,027:DEBUG:certbot.plugins.webroot:Removing /Library/WebServer/blindpromo.com/htdocs/.well-known/acme-challenge/3zQ0dwExqHxGBMpLKrSuYbkzIMBrLVXVwXrd3Mh1vMA
2019-04-15 10:51:25,027:DEBUG:certbot.plugins.webroot:All challenges cleaned up
2019-04-15 10:51:25,027:DEBUG:certbot.log:Exiting abnormally:
Traceback (most recent call last):
File “/usr/local/bin/certbot”, line 11, in
load_entry_point(‘certbot==0.30.2’, ‘console_scripts’, ‘certbot’)()
File “/usr/local/Cellar/certbot/0.30.2/libexec/lib/python3.7/site-packages/certbot/main.py”, line 1364, in main
return config.func(config, plugins)
File “/usr/local/Cellar/certbot/0.30.2/libexec/lib/python3.7/site-packages/certbot/main.py”, line 1249, in certonly
lineage = _get_and_save_cert(le_client, config, domains, certname, lineage)
File “/usr/local/Cellar/certbot/0.30.2/libexec/lib/python3.7/site-packages/certbot/main.py”, line 116, in _get_and_save_cert
renewal.renew_cert(config, domains, le_client, lineage)
File “/usr/local/Cellar/certbot/0.30.2/libexec/lib/python3.7/site-packages/certbot/renewal.py”, line 310, in renew_cert
new_cert, new_chain, new_key, _ = le_client.obtain_certificate(domains, new_key)
File “/usr/local/Cellar/certbot/0.30.2/libexec/lib/python3.7/site-packages/certbot/client.py”, line 353, in obtain_certificate
orderr = self._get_order_and_authorizations(csr.data, self.config.allow_subset_of_names)
File “/usr/local/Cellar/certbot/0.30.2/libexec/lib/python3.7/site-packages/certbot/client.py”, line 389, in _get_order_and_authorizations
authzr = self.auth_handler.handle_authorizations(orderr, best_effort)
File “/usr/local/Cellar/certbot/0.30.2/libexec/lib/python3.7/site-packages/certbot/auth_handler.py”, line 82, in handle_authorizations
self._respond(aauthzrs, resp, best_effort)
File “/usr/local/Cellar/certbot/0.30.2/libexec/lib/python3.7/site-packages/certbot/auth_handler.py”, line 161, in _respond
self._poll_challenges(aauthzrs, chall_update, best_effort)
File “/usr/local/Cellar/certbot/0.30.2/libexec/lib/python3.7/site-packages/certbot/auth_handler.py”, line 232, in _poll_challenges
raise errors.FailedChallenges(all_failed_achalls)
certbot.errors.FailedChallenges: Failed authorization procedure. blindpromo.com (http-01): urn:ietf:params:acme:error:connection :: The server could not connect to the client to verify the domain :: Fetching http://blindpromo.com/.well-known/acme-challenge/MR24YudjX3OREjqnUcG2tevfDCU-vXPL7ImQWt4vJeQ: Connection refused, www.blindpromo.com (http-01): urn:ietf:params:acme:error:connection :: The server could not connect to the client to verify the domain :: Fetching http://www.blindpromo.com/.well-known/acme-challenge/3zQ0dwExqHxGBMpLKrSuYbkzIMBrLVXVwXrd3Mh1vMA: Connection reset by peer

My web server is (include version):

Apache 2.4

The operating system my web server runs on is (include version):

macOS Sierra

I’m using a control panel to manage my site (no, or provide the name and version of the control panel):

Server.app

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you’re using Certbot):

certbot 0.30.2

I have the following redirect defined in Server.app:

^/$ https://blindpromo.com permanent status 301

This works to redirect anything from http to https but certbot fails. When I remove the redirect certbot renews the certificates and the world is fine.

What would I have to do so that anything http is redirected to https but certbot gets what it needs? Please if possible refer to Server.app from Apple and the settings inside the Web Section.

Thanks for your help

Hi @gugus

Letsencrypt doesn’t see a redirect:

There is the http fetched - with a connection refused.

And I can’t see such a redirect ( https://check-your-website.server-daten.de/?q=blindpromo.com ).

Domainname Http-Status redirect Sec. G
http://blindpromo.com/
213.189.151.242 200 0.110 H
http://www.blindpromo.com/
213.189.151.242 200 0.103 H
https://blindpromo.com/
213.189.151.242 200 0.887 B
https://www.blindpromo.com/
213.189.151.242 200 0.640 B
http://blindpromo.com/.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de
213.189.151.242 404 0.190 A
Not Found
Visible Content: Not Found The requested URL /.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de was not found on this server. Apache Server at blindpromo.com Port 80
http://www.blindpromo.com/.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de
213.189.151.242 404 0.100 A
Not Found
Visible Content: Not Found The requested URL /.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de was not found on this server. Apache Server at www.blindpromo.com Port 80

But there is no Connection refused.

Is there a firewall that blocks some countries?

I have currently removed the redirect so that certbot works (which it does).

The redirect was created like in the screenshot. I have reestablished it. It allows me to type http://blindpromo.com and I get https://blindpromo.com. This domain currently does not have any subpages. I maintain several other domains on the same server and all are identically set up, just currently without the redirects. As I need the other domains to work I did not reestablish the redirect, but blindpromo.com is currently not productively used and thus ideal for testing.

The redirect has two errors:

If you create a redirect, you should always redirect subdirectories and files correct.

And the redirect goes to …com without / at the end.

Thanks, that is fine, will change it

This I don’t understand, What would a correct redirect look like?

Thanks a lot for your patience

I don’t know what that system supports.

A standard redirect http -> https is something like

RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,QSA,R=permanent]

defined in the http vHost. So every request

http://example.com/folder/subfolder/file

is redirected to

https://example.com/folder/subfolder/file

There may be help pages with samples, check these.

Thanks for your help, I think the “trick” with the / at the end of the URL did it. Just tried it out and renewal did get through. We’ll see how things go in 2 months from now when the next auto renewal is due.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.