Challenges Failed Due To Invalid Response From .well-known

Do I need to do something with my .well-known directory permissions?

Would it be easier to use Apache Rewrite to strip www off before it call https?

Or would that be admitting defeat?

[are we there yet?]
We're not there yet.

That won't overcome all situations.
See:

NEVER !!!
lol

Let's have a look at the file:
/usr/local/etc/certbot/certs/renewal/walkershire.net.conf

3 Likes

Let's have a look at the file:
/usr/local/etc/certbot/certs/renewal/walkershire.net.conf

# renew_before_expiry = 30 days
version = 1.28.0
archive_dir = /usr/local/etc/certbot/certs/archive/walkershire.net
cert = /usr/local/etc/certbot/certs/live/walkershire.net/cert.pem
privkey = /usr/local/etc/certbot/certs/live/walkershire.net/privkey.pem
chain = /usr/local/etc/certbot/certs/live/walkershire.net/chain.pem
fullchain = /usr/local/etc/certbot/certs/live/walkershire.net/fullchain.pem
# Options used in the renewal process
[renewalparams]
account = 1d30db034ea2ca462fe7e24c284ffeca
config_dir = /usr/local/etc/certbot/certs
work_dir = /usr/local/etc/certbot
logs_dir = /usr/local/etc/certbot/logs
authenticator = standalone
server = https://acme-v02.api.letsencrypt.org/directory
key_type = rsa

Sorry for the Q in the middle of A

Getting late here

1 Like

Good information. Saved a me from going down that rabbit hole! Thank you.

1 Like

That needs to change.
I'd try:
authenticator = apache

And we should review:
certbot certificates

3 Likes

Do I just change it:

certbot --authenticator apache

Or it’s better to just issue a “new” certificate and include --apache

CERTBOT ERRORS

While exploring how to change authenticator I stumbled upon the Certbot Log Directory. Must be the only log not in the MacOS Console App. I has about 17 log file which are grey but fortunately the latest log can be opened so I copied it. It is very long.

Many lines with:
Server: nginx

I copied the Segments of yesterday’s log that look like they are may be helpful.

Segment A
PermissionError: [Errno 13] Permission denied: '/etc/apache2/.certbot.lock'
2022-07-17 21:57:24,873:DEBUG:certbot._internal.plugins.disco:Other error:(PluginEntryPoint#apache): Unable to create a lock file in /etc/apache2. Are you running Certbot with sufficient privileges to modify your Apache configuration?

Segment B
During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/Cellar/certbot/1.28.0/libexec/lib/python3.10/site-packages/certbot/_internal/plugins/disco.py", line 160, in prepare
self._initialized.prepare()
File "/usr/local/Cellar/certbot/1.28.0/libexec/lib/python3.10/site-packages/certbot_apache/_internal/configurator.py", line 397, in prepare
raise errors.PluginError(
certbot.errors.PluginError: Unable to create a lock file in /etc/apache2. Are you running Certbot with sufficient privileges to modify your Apache configuration?
2022-07-17 21:57:24,874:DEBUG:certbot._internal.plugins.disco:No installation (PluginEntryPoint#nginx): Could not find a usable 'nginx' binary. Ensure nginx exists, the binary is executable, and your PATH is set correctly.
Traceback (most recent call last):
File "/usr/local/Cellar/certbot/1.28.0/libexec/lib/python3.10/site-packages/certbot/_internal/plugins/disco.py", line 160, in prepare
self._initialized.prepare()
File "/usr/local/Cellar/certbot/1.28.0/libexec/lib/python3.10/site-packages/certbot_nginx/_internal/configurator.py", line 194, in prepare
raise errors.NoInstallationError(
certbot.errors.NoInstallationError: Could not find a usable 'nginx' binary. Ensure nginx exists, the binary is executable, and your PATH is set correctly.
2022-07-17 21:57:24,874:DEBUG:certbot._internal.plugins.selection:Multiple candidate plugins: * standalone
Description: Spin up a temporary webserver
Interfaces: Authenticator, Plugin
Entry point: standalone = certbot._internal.plugins.standalone:Authenticator
Initialized: <certbot._internal.plugins.standalone.Authenticator object at 0x10f58c220>
Prep: True

Segment C
Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded from the internet.

2022-07-17 21:58:46,632:DEBUG:certbot._internal.error_handler:Encountered exception:
Traceback (most recent call last):
File "/usr/local/Cellar/certbot/1.28.0/libexec/lib/python3.10/site-packages/certbot/_internal/auth_handler.py", line 106, in handle_authorizations
self._poll_authorizations(authzrs, max_retries, best_effort)
File "/usr/local/Cellar/certbot/1.28.0/libexec/lib/python3.10/site-packages/certbot/_internal/auth_handler.py", line 206, in _poll_authorizations
raise errors.AuthorizationError('Some challenges have failed.')
certbot.errors.AuthorizationError: Some challenges have failed.

2022-07-17 21:58:46,632:DEBUG:certbot._internal.error_handler:Calling registered functions
2022-07-17 21:58:46,632:INFO:certbot._internal.auth_handler:Cleaning up challenges
2022-07-17 21:58:46,632:DEBUG:certbot._internal.plugins.webroot:Removing /usr/local/var/www/.well-known/acme-challenge/vvxpygO7x57sLWt6lzkZNKJSxO0_EB6sT-uK8szaE5w
2022-07-17 21:58:46,632:DEBUG:certbot._internal.plugins.webroot:Removing /usr/local/var/www/.well-known/acme-challenge/kKFlEx0azvr44kA9H8vTO4M4e1Wf3zOJXooHSR2Kb74
2022-07-17 21:58:46,632:DEBUG:certbot._internal.plugins.webroot:Removing /usr/local/var/www/.well-known/acme-challenge/duMCp0K9Htm88aNZ1HS4dMxdxuX0Bd6-KONs5O4OdbU
2022-07-17 21:58:46,633:DEBUG:certbot._internal.plugins.webroot:Removing /usr/local/var/www/.well-known/acme-challenge/mFl3BFmXMFOWlRqDqejWfr2mKfGg65l5vSKHJe4OhcM
2022-07-17 21:58:46,633:DEBUG:certbot._internal.plugins.webroot:All challenges cleaned up
2022-07-17 21:58:46,633:DEBUG:certbot._internal.log:Exiting abnormally:
Traceback (most recent call last):
File "/usr/local/bin/certbot", line 33, in
sys.exit(load_entry_point('certbot==1.28.0', 'console_scripts', 'certbot')())
File "/usr/local/Cellar/certbot/1.28.0/libexec/lib/python3.10/site-packages/certbot/main.py", line 19, in main
return internal_main.main(cli_args)
File "/usr/local/Cellar/certbot/1.28.0/libexec/lib/python3.10/site-packages/certbot/_internal/main.py", line 1744, in main
return config.func(config, plugins)
File "/usr/local/Cellar/certbot/1.28.0/libexec/lib/python3.10/site-packages/certbot/_internal/main.py", line 1591, in certonly
lineage = _get_and_save_cert(le_client, config, domains, certname, lineage)
File "/usr/local/Cellar/certbot/1.28.0/libexec/lib/python3.10/site-packages/certbot/_internal/main.py", line 141, in _get_and_save_cert
lineage = le_client.obtain_and_enroll_certificate(domains, certname)
File "/usr/local/Cellar/certbot/1.28.0/libexec/lib/python3.10/site-packages/certbot/_internal/client.py", line 513, in obtain_and_enroll_certificate
cert, chain, key, _ = self.obtain_certificate(domains)
File "/usr/local/Cellar/certbot/1.28.0/libexec/lib/python3.10/site-packages/certbot/_internal/client.py", line 441, in obtain_certificate
orderr = self._get_order_and_authorizations(csr.data, self.config.allow_subset_of_names)
File "/usr/local/Cellar/certbot/1.28.0/libexec/lib/python3.10/site-packages/certbot/_internal/client.py", line 493, in _get_order_and_authorizations
authzr = self.auth_handler.handle_authorizations(orderr, self.config, best_effort)
File "/usr/local/Cellar/certbot/1.28.0/libexec/lib/python3.10/site-packages/certbot/_internal/auth_handler.py", line 106, in handle_authorizations
self._poll_authorizations(authzrs, max_retries, best_effort)
File "/usr/local/Cellar/certbot/1.28.0/libexec/lib/python3.10/site-packages/certbot/_internal/auth_handler.py", line 206, in _poll_authorizations
raise errors.AuthorizationError('Some challenges have failed.')
certbot.errors.AuthorizationError: Some challenges have failed.
2022-07-17 21:58:46,634:ERROR:certbot._internal.log:Some challenges have failed.

I thought it might work by just editing that file and making that change within it.
If that fails, we may need to use the --webroot approach, seeing as:

[certbot doesn't know where your Apache is at]

2 Likes

I changed authenticator to apache in the config and re did
certbot certonly -d www.D7036.com ........etc

Failed with:
Detail: 70.89.220.117: Invalid response from https://www.walkershire.net/.well-known/acme-challenge/G.......: 404

Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded from the internet.

How is it going to create the acme-challenge directory and file in name directory without write permission?

Restarted httpd and discovered that the log now only had 5 standalones, down from 43.

/usr/local/etc/certbot/logs/letsencrypt.log:4: 2022-07-18 13:20:23,455:DEBUG:certbot._internal.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#apache,PluginEntryPoint#manual,PluginEntryPoint#nginx,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
/usr/local/etc/certbot/logs/letsencrypt.log:59: 2022-07-18 13:20:23,881:DEBUG:certbot._internal.plugins.selection:Multiple candidate plugins: * standalone
/usr/local/etc/certbot/logs/letsencrypt.log:62: Entry point: standalone = certbot._internal.plugins.standalone:Authenticator
/usr/local/etc/certbot/logs/letsencrypt.log:62: Entry point: standalone = certbot._internal.plugins.standalone:Authenticator
/usr/local/etc/certbot/logs/letsencrypt.log:63: Initialized: <certbot._internal.plugins.standalone.Authenticator object at 0x10d240220>

The last log now has nginx 14 times down from 23

Last post was wrong, nginx mentions did not change.

These line look like apache is a problem:

/usr/local/etc/certbot/logs/letsencrypt.log:4: 2022-07-18 13:20:23,455:DEBUG:certbot._internal.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#apache,PluginEntryPoint#manual,PluginEntryPoint#nginx,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
/usr/local/etc/certbot/logs/letsencrypt.log:7: 2022-07-18 13:20:23,581:DEBUG:certbot_apache._internal.configurator:Apache version is 2.4.54
/usr/local/etc/certbot/logs/letsencrypt.log:7: 2022-07-18 13:20:23,581:DEBUG:certbot_apache._internal.configurator:Apache version is 2.4.54
/usr/local/etc/certbot/logs/letsencrypt.log:8: 2022-07-18 13:20:23,877:WARNING:certbot_apache._internal.configurator:ssl_module is statically linked but --apache-bin is missing; not disabling session tickets.
/usr/local/etc/certbot/logs/letsencrypt.log:8: 2022-07-18 13:20:23,877:WARNING:certbot_apache._internal.configurator:ssl_module is statically linked but --apache-bin is missing; not disabling session tickets.
/usr/local/etc/certbot/logs/letsencrypt.log:9: 2022-07-18 13:20:23,879:DEBUG:certbot_apache._internal.configurator:Encountered error:
/usr/local/etc/certbot/logs/letsencrypt.log:11: File "/usr/local/Cellar/certbot/1.28.0/libexec/lib/python3.10/site-packages/certbot_apache/_internal/configurator.py", line 394, in prepare
/usr/local/etc/certbot/logs/letsencrypt.log:25: PermissionError: [Errno 13] Permission denied: '/etc/apache2/.certbot.lock'
/usr/local/etc/certbot/logs/letsencrypt.log:26: 2022-07-18 13:20:23,880:DEBUG:certbot._internal.plugins.disco:Other error:(PluginEntryPoint#apache): Unable to create a lock file in /etc/apache2. Are you running Certbot with sufficient privileges to modify your Apache configuration?
/usr/local/etc/certbot/logs/letsencrypt.log:26: 2022-07-18 13:20:23,880:DEBUG:certbot._internal.plugins.disco:Other error:(PluginEntryPoint#apache): Unable to create a lock file in /etc/apache2. Are you running Certbot with sufficient privileges to modify your Apache configuration?
/usr/local/etc/certbot/logs/letsencrypt.log:26: 2022-07-18 13:20:23,880:DEBUG:certbot._internal.plugins.disco:Other error:(PluginEntryPoint#apache): Unable to create a lock file in /etc/apache2. Are you running Certbot with sufficient privileges to modify your Apache configuration?
/usr/local/etc/certbot/logs/letsencrypt.log:28: File "/usr/local/Cellar/certbot/1.28.0/libexec/lib/python3.10/site-packages/certbot_apache/_internal/configurator.py", line 394, in prepare
/usr/local/etc/certbot/logs/letsencrypt.log:42: PermissionError: [Errno 13] Permission denied: '/etc/apache2/.certbot.lock'
/usr/local/etc/certbot/logs/letsencrypt.log:49: File "/usr/local/Cellar/certbot/1.28.0/libexec/lib/python3.10/site-packages/certbot_apache/_internal/configurator.py", line 397, in prepare
/usr/local/etc/certbot/logs/letsencrypt.log:51: certbot.errors.PluginError: Unable to create a lock file in /etc/apache2. Are you running Certbot with sufficient privileges to modify your Apache configuration?
/usr/local/etc/certbot/logs/letsencrypt.log:51: certbot.errors.PluginError: Unable to create a lock file in /etc/apache2. Are you running Certbot with sufficient privileges to modify your Apache configuration?

After brew installed Apache the it is located at:
/usr/local/etc/httpd/

But the log show it is also looking at the old location, /etc/Apache2/
/usr/local/etc/certbot/logs/letsencrypt.log:25: PermissionError: [Errno 13] Permission denied: '/etc/apache2/.certbot.lock'
/usr/local/etc/certbot/logs/letsencrypt.log:26: 2022-07-18 13:20:23,880:DEBUG:certbot._internal.plugins.disco:Other error:(PluginEntryPoint#apache): Unable to create a lock file in /etc/apache2. Are you running Certbot with sufficient privileges to modify your Apache configuration?
/usr/local/etc/certbot/logs/letsencrypt.log:42: PermissionError: [Errno 13] Permission denied: '/etc/apache2/.certbot.lock'
/usr/local/etc/certbot/logs/letsencrypt.log:51: certbot.errors.PluginError: Unable to create a lock file in /etc/apache2. Are you running Certbot with sufficient privileges to modify your Apache configuration?

There is a way to tell certbot where all the Apache stuff is located.
But (to me) it seems much simpler to ignore the web server altogether and just use --webroot instead.
Try:

certbot certonly --cert-name walkershire.net --webroot \
-w /usr/local/var/www/walkershire -d "walkershire.net,www.walkershire.net" \
-w /usr/local/var/www/beyond      -d beyond.cleanair.com \
-w /usr/local/var/www/envirotemps -d "envirotemps.com,www.envirotemps.com" \
-w /usr/local/var/www/hi-tech     -d "hi-tech.rent.www.hi-tech.rent" \
-w /usr/local/var/www/D7036       -d "D7036.com,www.D7036.com" \
-w /usr/local/var/www/Hi-TechRent -d "Hi-TechRent.com.www.Hi-TechRent.com" --dry-run 

[copy all that as one single command - include the "--dry-run" (for initial testing)]

3 Likes

After changing 2 periods to commas it worked:
Simulating renewal of an existing certificate for walkershire.net and 10 more domains

The dry run was successful.

lowerlevel@lowerlevels-Mac-mini ~ %

WWW.walkrshire.net. WORKS. THANK YOU, More 4BEER.

3 Likes

only 2 periods to commas?
LOL

Ok, then run the altered version (without --dry-run) and obtain a real cert.

4 Likes

You are too kind.
Again, thanks for the :beers:
Cheers!

4 Likes