Renewal failure on MacOS 10.14.x using launchd homebrew apache

delaat · August 5, 2019, 1:49pm

I ran this command: from launchd for automated renewal: certbot renew

It produced this output:
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Cert is due for renewal, auto-renewing…
Non-interactive renewal: random delay of 57 seconds
Plugins selected: Authenticator apache, Installer apache
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for ipv4.worku.delaat.net
http-01 challenge for ipv6.worku.delaat.net
http-01 challenge for catch.work.delaat.net
http-01 challenge for work.delaat.net
http-01 challenge for worku.delaat.net
http-01 challenge for ipv6.work.delaat.net
http-01 challenge for ipv4.work.delaat.net
Cleaning up challenges
Attempting to renew cert (work.delaat.net-0001) from /etc/letsencrypt/renewal/work.delaat.net-0001.conf produced an unexpected error: Unable to find a virtual host listening on port 80 which is currently needed for Certbot to prove to the CA that you control your domain. Please add a virtual host for port 80… Skipping.
All renewal attempts failed. The following certs could not be renewed:
/etc/letsencrypt/live/work.delaat.net-0001/fullchain.pem (failure)
1 renew failure(s), 0 parse failure(s)

My web server is (include version): under homebrew apache: Server version: Apache/2.4.39 (Unix)

The operating system my web server runs on is (include version): MacOSX 10.14.6

My hosting provider, if applicable, is:do it myself on mac at home and at university

I can login to a root shell on my machine (yes or no, or I don’t know): Yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel):

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you’re using Certbot): certbot 0.36.0

Additional info:

The strange thing is that if I do the renew typing “sudo certbot renew” all goes fine.
For the automated renewal I created a plist

You can find that on https://delaat.net/setup

that I load with:
sudo launchctl load /Library/LaunchDaemons/com.letsencrypt.renew.plist

In an earlier try I got this from the debug log file:

2019-06-11 08:49:04,662:DEBUG:acme.client:Storing nonce: hlfmebzGmcAMbx_O93TXP2XEx3Codru6fQfmjQyFQtg
2019-06-11 08:49:04,663:INFO:certbot.auth_handler:Performing the following challenges:
2019-06-11 08:49:04,663:INFO:certbot.auth_handler:http-01 challenge for ipv6.ehw.delaat.net
2019-06-11 08:49:04,663:INFO:certbot.auth_handler:http-01 challenge for ehw.delaat.net
2019-06-11 08:49:04,664:INFO:certbot.auth_handler:http-01 challenge for eh.delaat.net
2019-06-11 08:49:04,664:INFO:certbot.auth_handler:http-01 challenge for catch.eh.delaat.net
2019-06-11 08:49:04,664:INFO:certbot.auth_handler:http-01 challenge for ipv6.eh.delaat.net
2019-06-11 08:49:04,687:DEBUG:certbot.error_handler:Encountered exception:
Traceback (most recent call last):
File “/usr/local/Cellar/certbot/0.35.0/libexec/lib/python3.7/site-packages/certbot/auth_handler.py”, line 69, in handle_authorizations
resps = self.auth.perform(achalls)
File “/usr/local/Cellar/certbot/0.35.0/libexec/lib/python3.7/site-packages/certbot_apache/configurator.py”, line 2264, in perform
http_response = http_doer.perform()
File “/usr/local/Cellar/certbot/0.35.0/libexec/lib/python3.7/site-packages/certbot_apache/http_01.py”, line 75, in perform
self._mod_config()
File “/usr/local/Cellar/certbot/0.35.0/libexec/lib/python3.7/site-packages/certbot_apache/http_01.py”, line 110, in _mod_config
for vh in self._relevant_vhosts():
File “/usr/local/Cellar/certbot/0.35.0/libexec/lib/python3.7/site-packages/certbot_apache/http_01.py”, line 165, in _relevant_vhosts
" {0}.".format(http01_port))
certbot.errors.PluginError: Unable to find a virtual host listening on port 80 which is currently needed for Certbot to prove to the CA that you control your domain. Please add a virtual host for port 80.

So why can it find vhosts in the cli mode but not when invoked via a launchd.

Best regards,
Cees

JuergenAuer · August 5, 2019, 2:14pm

Hi @delaat

there are duplicated port 80 vHost entries.

What says

apachectl -S

delaat · August 5, 2019, 2:38pm

Hi,

Please find below. I have to clarify: I have several servers, delaat.net , ext.delaat.net , work.delaat.net and even a few others with family members. The files on delaat.net/setup are pieces that I used to setup my main site. I never got automatic renewing to work so I tried different things but ended up doing the renewal by hand (i.e. typing sudo certbot renew). That works, it does not work when the exact same comes from launchd. So the file below is from the current machine that is within 30 days: work.delaat.net

VirtualHost configuration:
*:80 is a NameVirtualHost
default server catch.work.delaat.net (/usr/local/etc/httpd/extra/httpd-vhosts.conf:23)
port 80 namevhost catch.work.delaat.net (/usr/local/etc/httpd/extra/httpd-vhosts.conf:23)
port 80 namevhost work.delaat.net (/usr/local/etc/httpd/extra/httpd-vhosts.conf:28)
alias ipv4.work.delaat.net
alias ipv6.work.delaat.net
alias worku.delaat.net
alias ipv4.worku.delaat.net
alias ipv6.worku.delaat.net
*:443 work.delaat.net (/usr/local/etc/httpd/extra/httpd-vhosts-le-ssl.conf:2)
ServerRoot: “/usr/local/opt/httpd”
Main DocumentRoot: “/usr/local/var/www”
Main ErrorLog: “/usr/local/var/log/httpd/error_log”
Mutex rewrite-map: using_defaults
Mutex ssl-stapling-refresh: using_defaults
Mutex ssl-stapling: using_defaults
Mutex ssl-cache: using_defaults
Mutex default: dir="/usr/local/var/run/httpd/" mechanism=default
PidFile: “/usr/local/var/run/httpd/httpd.pid”
Define: DUMP_VHOSTS
Define: DUMP_RUN_CFG
User: name="_www" id=70
Group: name="_www" id=70

in next post I put the vhost file.

thanks,
Cees.

delaat · August 5, 2019, 2:39pm

and this is the vhost conf file:

<VirtualHost *:80>
DocumentRoot “/usr/local/var/www”
ServerName catch.work.delaat.net

<VirtualHost *:80>
DocumentRoot “/usr/local/var/www”
ServerName work.delaat.net
ServerAlias ipv4.work.delaat.net ipv6.work.delaat.net
ServerAlias worku.delaat.net ipv4.worku.delaat.net ipv6.worku.delaat.net
RewriteEngine on
RewriteCond %{SERVER_NAME} =work.delaat.net [OR]
RewriteCond %{SERVER_NAME} =ipv4.work.delaat.net [OR]
RewriteCond %{SERVER_NAME} =ipv6.work.delaat.net [OR]
RewriteCond %{SERVER_NAME} =worku.delaat.net [OR]
RewriteCond %{SERVER_NAME} =ipv4.worku.delaat.net [OR]
RewriteCond %{SERVER_NAME} =ipv6.worku.delaat.net
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]

<Directory “/Users/delaat/Sites/”>
Options Indexes FollowSymLinks
AllowOverride All
Require all granted

JuergenAuer · August 5, 2019, 2:50pm

You have two different vHosts, one with catch, the other with the other domains.

But you want one certificate with all subdomain names.

Create one certificate per vHost -> one certificate with catch, the other certificate with all the other domain names.

delaat · August 5, 2019, 2:58pm

But I have one certificate with all (sub)domain names for that server. I get that certificate if I do sudo certbot --apache
or
sudo certbot renew

See:
sudo certbot certificates

Saving debug log to /var/log/letsencrypt/letsencrypt.log

Found the following certs:
Certificate Name: work.delaat.net-0001
Domains: work.delaat.net catch.work.delaat.net ipv4.work.delaat.net ipv4.worku.delaat.net ipv6.work.delaat.net ipv6.worku.delaat.net worku.delaat.net
Expiry Date: 2019-08-31 14:32:18+00:00 (VALID: 25 days)
Certificate Path: /etc/letsencrypt/live/work.delaat.net-0001/fullchain.pem
Private Key Path: /etc/letsencrypt/live/work.delaat.net-0001/privkey.pem

delaat · August 5, 2019, 3:04pm

I have also the debug log of the renew attempt from earlier today:

2019-08-05 15:28:36,723:DEBUG:certbot.error_handler:Calling registered functions
2019-08-05 15:28:36,723:INFO:certbot.auth_handler:Cleaning up challenges
2019-08-05 15:28:36,946:WARNING:certbot.renewal:Attempting to renew cert (work.delaat.net-0001) from /etc/letsencrypt/renewal/work.delaat.net-0001.conf produced an unexpected error: Unable to find a virtual host listening on port 80 which is currently needed for Certbot to prove to the CA that you control your domain. Please add a virtual host for port 80… Skipping.
2019-08-05 15:28:36,947:DEBUG:certbot.renewal:Traceback was:
Traceback (most recent call last):
File “/usr/local/Cellar/certbot/0.36.0/libexec/lib/python3.7/site-packages/certbot/renewal.py”, line 449, in handle_renewal_request
main.renew_cert(lineage_config, plugins, renewal_candidate)
File “/usr/local/Cellar/certbot/0.36.0/libexec/lib/python3.7/site-packages/certbot/main.py”, line 1207, in renew_cert
renewed_lineage = _get_and_save_cert(le_client, config, lineage=lineage)
File “/usr/local/Cellar/certbot/0.36.0/libexec/lib/python3.7/site-packages/certbot/main.py”, line 115, in _get_and_save_cert
renewal.renew_cert(config, domains, le_client, lineage)
File “/usr/local/Cellar/certbot/0.36.0/libexec/lib/python3.7/site-packages/certbot/renewal.py”, line 307, in renew_cert
new_cert, new_chain, new_key, _ = le_client.obtain_certificate(domains, new_key)
File “/usr/local/Cellar/certbot/0.36.0/libexec/lib/python3.7/site-packages/certbot/client.py”, line 349, in obtain_certificate
orderr = self._get_order_and_authorizations(csr.data, self.config.allow_subset_of_names)
File “/usr/local/Cellar/certbot/0.36.0/libexec/lib/python3.7/site-packages/certbot/client.py”, line 385, in _get_order_and_authorizations
authzr = self.auth_handler.handle_authorizations(orderr, best_effort)
File “/usr/local/Cellar/certbot/0.36.0/libexec/lib/python3.7/site-packages/certbot/auth_handler.py”, line 69, in handle_authorizations
resps = self.auth.perform(achalls)
File “/usr/local/Cellar/certbot/0.36.0/libexec/lib/python3.7/site-packages/certbot_apache/configurator.py”, line 2303, in perform
http_response = http_doer.perform()
File “/usr/local/Cellar/certbot/0.36.0/libexec/lib/python3.7/site-packages/certbot_apache/http_01.py”, line 76, in perform
self._mod_config()
File “/usr/local/Cellar/certbot/0.36.0/libexec/lib/python3.7/site-packages/certbot_apache/http_01.py”, line 111, in _mod_config
for vh in self._relevant_vhosts():
File “/usr/local/Cellar/certbot/0.36.0/libexec/lib/python3.7/site-packages/certbot_apache/http_01.py”, line 166, in _relevant_vhosts
" {0}.".format(http01_port))
certbot.errors.PluginError: Unable to find a virtual host listening on port 80 which is currently needed for Certbot to prove to the CA that you control your domain. Please add a virtual host for port 80.

2019-08-05 15:28:36,947:ERROR:certbot.renewal:All renewal attempts failed. The following certs could not be renewed:
2019-08-05 15:28:36,947:ERROR:certbot.renewal: /etc/letsencrypt/live/work.delaat.net-0001/fullchain.pem (failure)
2019-08-05 15:28:36,947:DEBUG:certbot.log:Exiting abnormally:
Traceback (most recent call last):
File “/usr/local/bin/certbot”, line 11, in
load_entry_point(‘certbot==0.36.0’, ‘console_scripts’, ‘certbot’)()
File “/usr/local/Cellar/certbot/0.36.0/libexec/lib/python3.7/site-packages/certbot/main.py”, line 1381, in main
return config.func(config, plugins)
File “/usr/local/Cellar/certbot/0.36.0/libexec/lib/python3.7/site-packages/certbot/main.py”, line 1286, in renew
renewal.handle_renewal_request(config)
File “/usr/local/Cellar/certbot/0.36.0/libexec/lib/python3.7/site-packages/certbot/renewal.py”, line 474, in handle_renewal_request
len(renew_failures), len(parse_failures)))
certbot.errors.Error: 1 renew failure(s), 0 parse failure(s)
guest33:~ delaat$

delaat · August 5, 2019, 9:22pm

SOLVED

I created a script with the content

#!/bin/bash
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
sudo certbot renew

and updated the launchd plist file to run that script. That worked. So I assume that somehow the PATH variable is not set correctly for certbot renew via launchd.

Thanks,
Cees

system · September 4, 2019, 9:22pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Renewals failing Help	11	805	January 17, 2019
Renewals stopped working, looking for debugging advice Help	9	886	December 8, 2018
Renewal failure ubuntu 18.04 apache Help	3	845	December 17, 2019
Failed to renew certificate on OSX running apache homebrew Server	5	2395	March 17, 2017
Failure on renew suddenly on update to code Help	15	306	August 4, 2024

Renewal failure on MacOS 10.14.x using launchd homebrew apache

Related topics