My domain is: arpbooks.org
I ran this command: /usr/bin/certbot renew
It produced this output: (see below)
My web server is (include version): Apache 2.4.6
The operating system my web server runs on is (include version): CentOS 7.4
My hosting provider, if applicable, is: n/a
I can login to a root shell on my machine (yes or no, or I don't know): yes
I'm using a control panel to manage my site (no, or provide the name and version of the control panel): no
The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): certbot 0.31.0
I have several sites with LE setups that all exhibit the following issue:
Auto-renewal of certs setup via cron fails due the URL accessed for the http-01 challenge returning a 403. This only happens when the renewal is run from the cron task.
Running the same command on the command line renews the cert without problem.
The user running the task and the user when using the command line are the same; root.
This is a snippet from the letsencrypt.log for a failed auto-renew:
2019-08-04 18:09:22,160:INFO:certbot.auth_handler:Performing the following challenges:
2019-08-04 18:09:22,161:INFO:certbot.auth_handler:http-01 challenge for arpbooks.org
2019-08-04 18:09:22,161:INFO:certbot.auth_handler:http-01 challenge for www.arpbooks.org
2019-08-04 18:09:22,360:DEBUG:certbot_apache.http_01:Adding a temporary challenge validation Include for name: fakename in: /etc/httpd/sites-enabled/99_catchall.conf
2019-08-04 18:09:22,361:DEBUG:certbot_apache.http_01:Adding a temporary challenge validation Include for name: arpbooks.org in: /etc/httpd/sites-enabled/10_arpbooks.org.conf
2019-08-04 18:09:22,362:DEBUG:certbot_apache.http_01:Adding a temporary challenge validation Include for name: arpbooks.org in: /etc/httpd/sites-enabled/20_arpbooks.org_ssl.conf
2019-08-04 18:09:22,363:DEBUG:certbot_apache.http_01:writing a pre config file with text:
RewriteEngine on
RewriteRule ^/\.well-known/acme-challenge/([A-Za-z0-9-_=]+)$ /var/lib/letsencrypt/http_challenges/$1 [END]
2019-08-04 18:09:22,363:DEBUG:certbot_apache.http_01:writing a post config file with text:
<Directory /var/lib/letsencrypt/http_challenges>
Require all granted
</Directory>
<Location /.well-known/acme-challenge>
Require all granted
</Location>
2019-08-04 18:09:22,508:DEBUG:certbot.reverter:Creating backup of /etc/httpd/sites-enabled/20_arpbooks.org_ssl.conf
2019-08-04 18:09:22,508:DEBUG:certbot.reverter:Creating backup of /etc/httpd/sites-enabled/99_catchall.conf
2019-08-04 18:09:22,509:DEBUG:certbot.reverter:Creating backup of /etc/httpd/sites-enabled/10_arpbooks.org.conf
2019-08-04 18:09:26,067:INFO:certbot.auth_handler:Waiting for verification...
...
"type": "http-01",
"status": "invalid",
"error": {
"type": "urn:ietf:params:acme:error:unauthorized",
"detail": "Invalid response from http://arpbooks.org/.well-known/acme-challenge/m-6835HOzzADnXC30DFkMU9KzOIzhxMOtnaUnqN9www [138.197.133.251]: \"\u003c!DOCTYPE HTML PUBLIC \\\"-//IETF//DTD HTML 2.0//EN\\\"\u003e\\n\u003chtml\u003e\u003chead\u003e\\n\u003ctitle\u003e403 Forbidden\u003c/title\u003e\\n\u003c/head\u003e\u003cbody\u003e\\n\u003ch1\u003eForbidden\u003c/h1\u003e\\n\u003cp\"",
"status": 403
},
This is a snippet from the same log for the renewal succeeding from the command line:
2019-08-04 19:13:14,853:INFO:certbot.auth_handler:Performing the following challenges:
2019-08-04 19:13:14,853:INFO:certbot.auth_handler:http-01 challenge for arpbooks.org
2019-08-04 19:13:14,853:INFO:certbot.auth_handler:http-01 challenge for www.arpbooks.org
2019-08-04 19:13:15,002:DEBUG:certbot_apache.http_01:Adding a temporary challenge validation Include for name: fakename in: /etc/httpd/sites-enabled/99_catchall.conf
2019-08-04 19:13:15,002:DEBUG:certbot_apache.http_01:Adding a temporary challenge validation Include for name: arpbooks.org in: /etc/httpd/sites-enabled/10_arpbooks.org.conf
2019-08-04 19:13:15,002:DEBUG:certbot_apache.http_01:Adding a temporary challenge validation Include for name: arpbooks.org in: /etc/httpd/sites-enabled/20_arpbooks.org_ssl.conf
2019-08-04 19:13:15,003:DEBUG:certbot_apache.http_01:writing a pre config file with text:
2019-08-04 19:13:15,003:DEBUG:certbot_apache.http_01:writing a post config file with text:
2019-08-04 19:13:15,072:DEBUG:certbot.reverter:Creating backup of /etc/httpd/sites-enabled/20_arpbooks.org_ssl.conf
2019-08-04 19:13:15,073:DEBUG:certbot.reverter:Creating backup of /etc/httpd/sites-enabled/99_catchall.conf
2019-08-04 19:13:15,073:DEBUG:certbot.reverter:Creating backup of /etc/httpd/sites-enabled/10_arpbooks.org.conf
2019-08-04 19:13:18,430:INFO:certbot.auth_handler:Waiting for verification...
...
2019-08-04 19:13:23,708:DEBUG:acme.client:Received response:
2019-08-04 19:13:23,717:DEBUG:certbot.storage:Writing new private key to /etc/letsencrypt/archive/arpbooks.org/privkey3.pem.
2019-08-04 19:13:23,718:DEBUG:certbot.storage:Writing certificate to /etc/letsencrypt/archive/arpbooks.org/cert3.pem.
2019-08-04 19:13:23,718:DEBUG:certbot.storage:Writing chain to /etc/letsencrypt/archive/arpbooks.org/chain3.pem.
2019-08-04 19:13:23,718:DEBUG:certbot.storage:Writing full chain to /etc/letsencrypt/archive/arpbooks.org/fullchain3.pem.
2019-08-04 19:13:24,074:DEBUG:certbot.storage:Writing new config /etc/letsencrypt/renewal/arpbooks.org.conf.new.
2019-08-04 19:13:24,076:DEBUG:certbot.plugins.storage:Plugin storage file /etc/letsencrypt/.pluginstorage.json was empty, no values loaded
2019-08-04 19:13:24,314:DEBUG:certbot.plugins.selection:Requested authenticator apache and installer apache
2019-08-04 19:13:24,315:DEBUG:certbot.plugins.selection:Selecting plugin: * apache
2019-08-04 19:13:24,316:DEBUG:certbot.renewal:no renewal failures
This is the renewal config:
# renew_before_expiry = 30 days
version = 0.31.0
archive_dir = /etc/letsencrypt/archive/arpbooks.org
cert = /etc/letsencrypt/live/arpbooks.org/cert.pem
privkey = /etc/letsencrypt/live/arpbooks.org/privkey.pem
chain = /etc/letsencrypt/live/arpbooks.org/chain.pem
fullchain = /etc/letsencrypt/live/arpbooks.org/fullchain.pem
# Options used in the renewal process
[renewalparams]
authenticator = apache
installer = apache
account = ab6b3d4f2aab0381abf4410f2aa9910c
server = https://acme-v02.api.letsencrypt.org/directory
My assumption is that the command run on the command line should be seeing the same issue as the cron task, which is run as the same user (root). Clearly that's not the case.
I would appreciate any hints of what to look at next, or how to figure out why the cron task has issues, while the direct call works fine.
Thanks,
Ben