The client lacks sufficient authorization


#1

Hi.

i use letsencrypt about 2 years now. Everything works fine since. But i have some problem with one server to renew his certificate.

Last time it run was 10 september. Now we have this error if we try :

Attempting to renew cert ( ) from /etc/letsencrypt/renewal/ .conf produced an unexpected error: Failed authorization procedure. (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http:// /.well-known/acme-challenge/LkLMydnzrrTPUvJfyfsyInPNsFCf9OvRcX7XUzifoUU: “\n\n403 Forbidden\n\n

Forbidden

\n<p”. Skipping.

_ Domain: _
_ Type: unauthorized_
_ Detail: Invalid response from_
_ http:// /.well-known/acme-challenge/LkLMydnzrrTPUvJfyfsyInPNsFCf9OvRcX7XUzifoUU:_
_ "\n\n403_
_ Forbidden\n\n

Forbidden

\n<p"_

_ To fix these errors, please make sure that your domain name was_
_ entered correctly and the DNS A/AAAA record(s) for that domain_
_ contain(s) the right IP address._

I’ve found a lot of help on this forum, but none can really help.

I’ve try to recreate the .well-known subdirectory with a file in to be sure the rights are ok, and it’s ok.

We check in the DNS A/AAAA and correct it for the ip address not the domain name, and it’s ok.

We use CentOs 6 with apache, and certbot-auto is at the latest version 0.27.1.

crontab : certbot-auto renew --quiet

For me nothing really changed on this server, maybe yum update for the OS, nothing really new.

Now the certificate is expired, Can i just remove this certificate and recreate it ?

Thanks.


#2

Hi @Stayfans,

That won’t necessarily help. What’s the domain name?


#3

Sorry, blogue.uqtr.ca


#4

This seems to indicate that user/pass are required to access this file/folder.
When I try it, I get “404 Not Found”.
I would try placing a test file in the challenge folder:
http://blogue.uqtr.ca/.well-known/acme-challenge/1234


#5

Which Certbot authenticator plugin are you using? (Did you specify --webroot, --standalone, or something else?) You can find this information in /etc/letsencrypt/renewal/blogue.uqtr.ca.conf on the line that says authenticator.


#6

I created the file 1234 with the right owner apache. And it work.


#7

Thanks for your help everyone, you’re fast.

renew_before_expiry = 30 days

version = 0.27.1
archive_dir = /etc/letsencrypt/archive/blogue.uqtr.ca
cert = /etc/letsencrypt/live/blogue.uqtr.ca/cert.pem
privkey = /etc/letsencrypt/live/blogue.uqtr.ca/privkey.pem
chain = /etc/letsencrypt/live/blogue.uqtr.ca/chain.pem
fullchain = /etc/letsencrypt/live/blogue.uqtr.ca/fullchain.pem

Options used in the renewal process

[renewalparams]
authenticator = apache
installer = apache
account = 86d7c259dce7b91d47294496b8a8f3e5
server = https://acme-v02.api.letsencrypt.org/directory


#8

Huh, I’d expect that should be working.

Could you try the certbot renew command again and post the resulting log from /var/log/letsencrypt?


#9

Yes, try it without the quiet
Let us hear all the noise! - LOL


#10

2018-12-10 15:18:20,307:DEBUG:certbot.main:certbot version: 0.29.1
2018-12-10 15:18:20,308:DEBUG:certbot.main:Arguments:
2018-12-10 15:18:20,308:DEBUG:certbot.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#apache,PluginEntryPoint#manual,PluginEntryPoint#nginx,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
2018-12-10 15:18:20,342:DEBUG:certbot.log:Root logging level set at 20
2018-12-10 15:18:20,343:INFO:certbot.log:Saving debug log to /var/log/letsencrypt/letsencrypt.log
2018-12-10 15:18:20,409:DEBUG:certbot.plugins.selection:Requested authenticator <certbot.cli._Default object at 0x7f364cea3860> and installer <certbot.cli._Default object at 0x7f364cea3860>
2018-12-10 15:18:20,464:DEBUG:certbot.storage:Should renew, less than 30 days before certificate expiry 2018-12-09 03:01:46 UTC.
2018-12-10 15:18:20,464:INFO:certbot.renewal:Cert is due for renewal, auto-renewing…
2018-12-10 15:18:20,464:DEBUG:certbot.plugins.selection:Requested authenticator apache and installer apache
2018-12-10 15:18:20,532:DEBUG:certbot_apache.configurator:Apache version is 2.2.15
2018-12-10 15:18:20,748:DEBUG:certbot.plugins.selection:Single candidate plugin: * apache
Description: Apache Web Server plugin
Interfaces: IAuthenticator, IInstaller, IPlugin
Entry point: apache = certbot_apache.entrypoint:ENTRYPOINT
Initialized: <certbot_apache.override_centos.CentOSConfigurator object at 0x7f364cf17be0>
Prep: True
2018-12-10 15:18:20,751:DEBUG:certbot.plugins.selection:Single candidate plugin: * apache
Description: Apache Web Server plugin
Interfaces: IAuthenticator, IInstaller, IPlugin
Entry point: apache = certbot_apache.entrypoint:ENTRYPOINT
Initialized: <certbot_apache.override_centos.CentOSConfigurator object at 0x7f364cf17be0>
Prep: True
2018-12-10 15:18:20,752:DEBUG:certbot.plugins.selection:Selected authenticator <certbot_apache.override_centos.CentOSConfigurator object at 0x7f364cf17be0> and installer <certbot_apache.override_centos.CentOSConfigurator object at 0x7f364cf17be0>
2018-12-10 15:18:20,752:INFO:certbot.plugins.selection:Plugins selected: Authenticator apache, Installer apache
2018-12-10 15:18:20,757:DEBUG:certbot.main:Picked account: <Account(RegistrationResource(new_authzr_uri=‘https://acme-v01.api.letsencrypt.org/acme/new-authz’, body=Registration(contact=(‘mailto:wpadmin@uqtr.ca’,), status=None, only_return_existing=None, terms_of_service_agreed=None, agreement=‘https://letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf’, key=JWKRSA(key=<ComparableRSAKey(<cryptography.hazmat.backends.openssl.rsa._RSAPublicKey object at 0x7f364cea3fd0>)>), external_account_binding=None), uri=‘https://acme-v01.api.letsencrypt.org/acme/reg/9110367’, terms_of_service=‘https://letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf’), 86d7c259dce7b91d47294496b8a8f3e5, Meta(creation_host=‘wpblogue.uqtr.ca’, creation_dt=datetime.datetime(2017, 2, 2, 18, 12, 48, tzinfo=)))>
2018-12-10 15:18:20,763:DEBUG:acme.client:Sending GET request to https://acme-v02.api.letsencrypt.org/directory.
2018-12-10 15:18:20,767:DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): acme-v02.api.letsencrypt.org
2018-12-10 15:18:21,161:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 “GET /directory HTTP/1.1” 200 658
2018-12-10 15:18:21,162:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Content-Type: application/json
Content-Length: 658
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800
Expires: Mon, 10 Dec 2018 20:18:21 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Mon, 10 Dec 2018 20:18:21 GMT
Connection: keep-alive

{
“keyChange”: “https://acme-v02.api.letsencrypt.org/acme/key-change”,
“meta”: {
“caaIdentities”: [
letsencrypt.org
],
“termsOfService”: “https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf”,
“website”: “https://letsencrypt.org
},
“newAccount”: “https://acme-v02.api.letsencrypt.org/acme/new-acct”,
“newNonce”: “https://acme-v02.api.letsencrypt.org/acme/new-nonce”,
“newOrder”: “https://acme-v02.api.letsencrypt.org/acme/new-order”,
“revokeCert”: “https://acme-v02.api.letsencrypt.org/acme/revoke-cert”,
“rnM0ye9qF8k”: “Adding random entries to the directory
}
2018-12-10 15:18:21,163:INFO:certbot.main:Renewing an existing certificate
2018-12-10 15:18:21,459:DEBUG:certbot.crypto_util:Generating key (2048 bits): /etc/letsencrypt/keys/0032_key-certbot.pem
2018-12-10 15:18:21,464:DEBUG:certbot.crypto_util:Creating CSR: /etc/letsencrypt/csr/0032_csr-certbot.pem
2018-12-10 15:18:21,465:DEBUG:acme.client:Requesting fresh nonce
2018-12-10 15:18:21,465:DEBUG:acme.client:Sending HEAD request to https://acme-v02.api.letsencrypt.org/acme/new-nonce.
2018-12-10 15:18:21,736:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 “HEAD /acme/new-nonce HTTP/1.1” 204 0
2018-12-10 15:18:21,737:DEBUG:acme.client:Received response:
HTTP 204
Server: nginx
Replay-Nonce: Qj0IfJW92HShvWkPtunpB-xqwlSNkNe5hLEWA0xjvwE
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800
Expires: Mon, 10 Dec 2018 20:18:21 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Mon, 10 Dec 2018 20:18:21 GMT
Connection: keep-alive

2018-12-10 15:18:21,737:DEBUG:acme.client:Storing nonce: Qj0IfJW92HShvWkPtunpB-xqwlSNkNe5hLEWA0xjvwE
2018-12-10 15:18:21,738:DEBUG:acme.client:JWS payload:
b’{\n “identifiers”: [\n {\n “type”: “dns”,\n “value”: “blogue.uqtr.ca”\n }\n ]\n}’
2018-12-10 15:18:21,742:DEBUG:acme.client:Sending POST request to https://acme-v02.api.letsencrypt.org/acme/new-order:
{
“payload”: “ewogICJpZGVudGlmaWVycyI6IFsKICAgIHsKICAgICAgInR5cGUiOiAiZG5zIiwKICAgICAgInZhbHVlIjogImJsb2d1ZS51cXRyLmNhIgogICAgfQogIF0KfQ”,
“signature”: “K9bWZ2dDaDsdOkhP3jDUlmJMV7IbTqyVeKP4_ZaM-vIGG4KVZNhKb5XSEV7gSK2cyvLhJFF30Vy2t6YJTCLW9aulgc0yLmhdlVZUg4Y_EVpDMyQ4AZ1k5CsZnvUgRlVSBL9uNh5FGilz6xV-ztRTohDZZaZ7Fxh1LU_7bMoecNK3PiG0gU0qsA7dywOvDpNfG_3hpGZZfZ9WjUwMCTByYuAW9ortvNx_B7C-f4F9o7t3VerGccU8dAhJ7hJaXGDuxq0XHcEJn09m2V-RNV4dLogATgvNzd_8YLvOlcCh-0PlcmiFBzDQFL1dNP1O0GKpEQpK2lcoE1S0SkaFDjShrQ”,
“protected”: “eyJraWQiOiAiaHR0cHM6Ly9hY21lLXYwMS5hcGkubGV0c2VuY3J5cHQub3JnL2FjbWUvcmVnLzkxMTAzNjciLCAiYWxnIjogIlJTMjU2IiwgIm5vbmNlIjogIlFqMElmSlc5MkhTaHZXa1B0dW5wQi14cXdsU05rTmU1aExFV0EweGp2d0UiLCAidXJsIjogImh0dHBzOi8vYWNtZS12MDIuYXBpLmxldHNlbmNyeXB0Lm9yZy9hY21lL25ldy1vcmRlciJ9”
}
2018-12-10 15:18:21,880:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 “POST /acme/new-order HTTP/1.1” 201 372
2018-12-10 15:18:21,881:DEBUG:acme.client:Received response:
HTTP 201
Server: nginx
Content-Type: application/json
Content-Length: 372
Boulder-Requester: 9110367
Location: https://acme-v02.api.letsencrypt.org/acme/order/9110367/214973067
Replay-Nonce: 4yEVdvhyCv_z6M51I7XRAC-pkapLyeZ3ZNnEk3SfQeE
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800
Expires: Mon, 10 Dec 2018 20:18:21 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Mon, 10 Dec 2018 20:18:21 GMT
Connection: keep-alive

{
“status”: “pending”,
“expires”: “2018-12-17T20:18:21.820218525Z”,
“identifiers”: [
{
“type”: “dns”,
“value”: “blogue.uqtr.ca”
}
],
“authorizations”: [
https://acme-v02.api.letsencrypt.org/acme/authz/2LT7sc2Ks21BpW1mMlYNXcn50UHi9mE2YbKfWUDelrg
],
“finalize”: “https://acme-v02.api.letsencrypt.org/acme/finalize/9110367/214973067
}
2018-12-10 15:18:21,881:DEBUG:acme.client:Storing nonce: 4yEVdvhyCv_z6M51I7XRAC-pkapLyeZ3ZNnEk3SfQeE
2018-12-10 15:18:21,882:DEBUG:acme.client:JWS payload:
b’’
2018-12-10 15:18:21,884:DEBUG:acme.client:Sending POST request to https://acme-v02.api.letsencrypt.org/acme/authz/2LT7sc2Ks21BpW1mMlYNXcn50UHi9mE2YbKfWUDelrg:
{
“payload”: “”,
“signature”: “YPwgTSKbEjgEjwaTgVRy-IJA3Ao_xLbIuFvx1cLmpYFH8mfS-tF6kn4EGm7FOXWBTjigFxJSFs2oV1PYeWElqZdkZRh3_CvL-xySF1OqUfBmX8UWouocEguJngEhmh65AdLq8nj-K8Y1J6-u0wdPupQ6647o-9wvavFcfhCqSW_jZqq4FQbCMLLZ79gzTILoJayWCfQOHDIBz6nXwA_IS_AVpZVXaX8zvFqcNPRb4vy8QXSYWOEcac4ykXlWjP11fFM03BVAzLt0xWVSsYpvxupxGqZJAc_h62RcbYOSgztJq-ctapCdAbnry1TPBtue1qFfJpgOAOKV_EGviGdpPA”,
“protected”: “eyJraWQiOiAiaHR0cHM6Ly9hY21lLXYwMS5hcGkubGV0c2VuY3J5cHQub3JnL2FjbWUvcmVnLzkxMTAzNjciLCAiYWxnIjogIlJTMjU2IiwgIm5vbmNlIjogIjR5RVZkdmh5Q3ZfejZNNTFJN1hSQUMtcGthcEx5ZVozWk5uRWszU2ZRZUUiLCAidXJsIjogImh0dHBzOi8vYWNtZS12MDIuYXBpLmxldHNlbmNyeXB0Lm9yZy9hY21lL2F1dGh6LzJMVDdzYzJLczIxQnBXMW1NbFlOWGNuNTBVSGk5bUUyWWJLZldVRGVscmcifQ”
}
2018-12-10 15:18:22,101:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 “POST /acme/authz/2LT7sc2Ks21BpW1mMlYNXcn50UHi9mE2YbKfWUDelrg HTTP/1.1” 200 1162
2018-12-10 15:18:22,102:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Content-Type: application/json
Content-Length: 1162
Boulder-Requester: 9110367
Replay-Nonce: FMiTSwuRRgNHa2ADQ0wFHdxg1ZKhddc7bckZbi7jjiM
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800
Expires: Mon, 10 Dec 2018 20:18:22 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Mon, 10 Dec 2018 20:18:22 GMT
Connection: keep-alive

{
“identifier”: {
“type”: “dns”,
“value”: “blogue.uqtr.ca”
},
“status”: “pending”,
“expires”: “2018-12-17T20:18:21Z”,
“challenges”: [
{
“type”: “http-01”,
“status”: “pending”,
“url”: “https://acme-v02.api.letsencrypt.org/acme/challenge/2LT7sc2Ks21BpW1mMlYNXcn50UHi9mE2YbKfWUDelrg/10182356395”,
“token”: “oZ6D50NHPPYzgTaLpJRHLtrhKLRKbutQt2wYnPuDcd4”
},
{
“type”: “dns-01”,
“status”: “pending”,
“url”: “https://acme-v02.api.letsencrypt.org/acme/challenge/2LT7sc2Ks21BpW1mMlYNXcn50UHi9mE2YbKfWUDelrg/10182356396”,
“token”: “bhl2UthF-h8s4NZNElyu6t8569NNUu0lB3Mfyqk46xI”
},
{
“type”: “tls-alpn-01”,
“status”: “pending”,
“url”: “https://acme-v02.api.letsencrypt.org/acme/challenge/2LT7sc2Ks21BpW1mMlYNXcn50UHi9mE2YbKfWUDelrg/10182356397”,
“token”: “4ksHlkQzI-lPne1Qn7lZkR6bFRmbgW1GaVOmgKPnOYI”
},
{
“type”: “tls-sni-01”,
“status”: “pending”,
“url”: “https://acme-v02.api.letsencrypt.org/acme/challenge/2LT7sc2Ks21BpW1mMlYNXcn50UHi9mE2YbKfWUDelrg/10182356398”,
“token”: “Y79JjiJquiQ3APmCmGXhXig5o3zVYl01Cu6pmlwABUs”
}
]
}
2018-12-10 15:18:22,102:DEBUG:acme.client:Storing nonce: FMiTSwuRRgNHa2ADQ0wFHdxg1ZKhddc7bckZbi7jjiM
2018-12-10 15:18:22,103:INFO:certbot.auth_handler:Performing the following challenges:
2018-12-10 15:18:22,104:INFO:certbot.auth_handler:http-01 challenge for blogue.uqtr.ca
2018-12-10 15:18:22,162:DEBUG:certbot_apache.http_01:Adding a temporary challenge validation Include for name: blogue.uqtr.ca in: /etc/httpd/conf/httpd.conf
2018-12-10 15:18:22,163:DEBUG:certbot_apache.http_01:writing a pre config file with text:
RewriteEngine on
RewriteRule ^/.well-known/acme-challenge/([A-Za-z0-9-_=]+)$ /var/lib/letsencrypt/http_challenges/$1 [L]

2018-12-10 15:18:22,163:DEBUG:certbot_apache.http_01:writing a post config file with text:
<Directory /var/lib/letsencrypt/http_challenges>
Order Allow,Deny
Allow from all

<Location /.well-known/acme-challenge>
Order Allow,Deny
Allow from all

2018-12-10 15:18:22,226:DEBUG:certbot.reverter:Creating backup of /etc/httpd/conf/httpd.conf
2018-12-10 15:18:25,448:INFO:certbot.auth_handler:Waiting for verification…
2018-12-10 15:18:25,450:DEBUG:acme.client:JWS payload:
b’{\n “type”: “http-01”,\n “resource”: “challenge”,\n “keyAuthorization”: “oZ6D50NHPPYzgTaLpJRHLtrhKLRKbutQt2wYnPuDcd4.ghteechRGBwLMJ9iPaHq2gtDiO5EHGQT_rM4Hc0As-w”\n}’
2018-12-10 15:18:25,454:DEBUG:acme.client:Sending POST request to https://acme-v02.api.letsencrypt.org/acme/challenge/2LT7sc2Ks21BpW1mMlYNXcn50UHi9mE2YbKfWUDelrg/10182356395:
{
“payload”: “ewogICJ0eXBlIjogImh0dHAtMDEiLAogICJyZXNvdXJjZSI6ICJjaGFsbGVuZ2UiLAogICJrZXlBdXRob3JpemF0aW9uIjogIm9aNkQ1ME5IUFBZemdUYUxwSlJITHRyaEtMUktidXRRdDJ3WW5QdURjZDQuZ2h0ZWVjaFJHQndMTUo5aVBhSHEyZ3REaU81RUhHUVRfck00SGMwQXMtdyIKfQ”,
“signature”: “gbFQhJLVunFcGNJUL5yAGZb01E87UB6I5XK-JE-L3hWvXawSa3ZKtYXKjeNHnGtirsPP0ThrCO89F-mCNF4yiK1qjBIdvQfSWYzyrMZ7d7MzjMRLdr0wbTKMTRj3bj9zcYZXQDi4PLPdCHcmzapz5KrF-mId3S8i_ZykK8mYj_8ZtKIls97B0KgqHDMiC4DApX8M6q58_mKKRvlZMlIuA0f106RLL1IVdpjgkzX6AQoLMjF73NIJzTiYOhInQ7ciY1WcVhDBjphsPao3NDGgl01klOrrdXLgIr58DAr1gCjMnwE5kiKNasVV98Ll8zl8vVxMZ7BQEF5l1cCQQ2Cfyw”,
“protected”: “eyJraWQiOiAiaHR0cHM6Ly9hY21lLXYwMS5hcGkubGV0c2VuY3J5cHQub3JnL2FjbWUvcmVnLzkxMTAzNjciLCAiYWxnIjogIlJTMjU2IiwgIm5vbmNlIjogIkZNaVRTd3VSUmdOSGEyQURRMHdGSGR4ZzFaS2hkZGM3YmNrWmJpN2pqaU0iLCAidXJsIjogImh0dHBzOi8vYWNtZS12MDIuYXBpLmxldHNlbmNyeXB0Lm9yZy9hY21lL2NoYWxsZW5nZS8yTFQ3c2MyS3MyMUJwVzFtTWxZTlhjbjUwVUhpOW1FMlliS2ZXVURlbHJnLzEwMTgyMzU2Mzk1In0”
}
2018-12-10 15:18:25,575:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 “POST /acme/challenge/2LT7sc2Ks21BpW1mMlYNXcn50UHi9mE2YbKfWUDelrg/10182356395 HTTP/1.1” 200 224
2018-12-10 15:18:25,577:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Content-Type: application/json
Content-Length: 224
Boulder-Requester: 9110367
Link: https://acme-v02.api.letsencrypt.org/acme/authz/2LT7sc2Ks21BpW1mMlYNXcn50UHi9mE2YbKfWUDelrg;rel=“up”
Location: https://acme-v02.api.letsencrypt.org/acme/challenge/2LT7sc2Ks21BpW1mMlYNXcn50UHi9mE2YbKfWUDelrg/10182356395
Replay-Nonce: ZAfhMuVXae_6ssqhesELWLOBQw9aTHjWXUh5sxNXZWo
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800
Expires: Mon, 10 Dec 2018 20:18:25 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Mon, 10 Dec 2018 20:18:25 GMT
Connection: keep-alive

{
“type”: “http-01”,
“status”: “pending”,
“url”: “https://acme-v02.api.letsencrypt.org/acme/challenge/2LT7sc2Ks21BpW1mMlYNXcn50UHi9mE2YbKfWUDelrg/10182356395”,
“token”: “oZ6D50NHPPYzgTaLpJRHLtrhKLRKbutQt2wYnPuDcd4”
}
2018-12-10 15:18:25,577:DEBUG:acme.client:Storing nonce: ZAfhMuVXae_6ssqhesELWLOBQw9aTHjWXUh5sxNXZWo
2018-12-10 15:18:28,581:DEBUG:acme.client:JWS payload:
b’’
2018-12-10 15:18:28,584:DEBUG:acme.client:Sending POST request to https://acme-v02.api.letsencrypt.org/acme/authz/2LT7sc2Ks21BpW1mMlYNXcn50UHi9mE2YbKfWUDelrg:
{
“payload”: “”,
“signature”: “s-ZVZ4OaBZRuglpoT-LBZnd92kmBSCeHmAHyOi1G3xc3-xKLXd8JfbBNzJH98iy4dr-zxPOUJs7HYxF0oMzonpj89ykuyYKLxoXEjYGq9gowAi9tFQG0zZ1KGUJnbQpRecuHEsdwClMJfdI-UA52o7SjXbG-wQ9iUt1bw0qFxJ0kBDI1dATdxjRt5hUOCTXib1mEAhT3gmg1CcTcPE7lFAcHdNtI6y4s6VC3YzsUgumvRbjJ5O1HJfKty8vTCq2AG91BKxQXyBwol8k7w1zBCaWg0cigfIKyBmtiFZf3ndz5N9sBZrz5wB9NQvLA-Ki-6uMhvnB2vGg4sEaZGVnTDw”,
“protected”: “eyJraWQiOiAiaHR0cHM6Ly9hY21lLXYwMS5hcGkubGV0c2VuY3J5cHQub3JnL2FjbWUvcmVnLzkxMTAzNjciLCAiYWxnIjogIlJTMjU2IiwgIm5vbmNlIjogIlpBZmhNdVZYYWVfNnNzcWhlc0VMV0xPQlF3OWFUSGpXWFVoNXN4TlhaV28iLCAidXJsIjogImh0dHBzOi8vYWNtZS12MDIuYXBpLmxldHNlbmNyeXB0Lm9yZy9hY21lL2F1dGh6LzJMVDdzYzJLczIxQnBXMW1NbFlOWGNuNTBVSGk5bUUyWWJLZldVRGVscmcifQ”
}
2018-12-10 15:18:28,663:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 “POST /acme/authz/2LT7sc2Ks21BpW1mMlYNXcn50UHi9mE2YbKfWUDelrg HTTP/1.1” 200 2001
2018-12-10 15:18:28,664:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Content-Type: application/json
Content-Length: 2001
Boulder-Requester: 9110367
Replay-Nonce: 1RZ6_ZRdzJozdiEs0-mwT8bF2zEWoND0QqQLRND9cEA
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800
Expires: Mon, 10 Dec 2018 20:18:28 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Mon, 10 Dec 2018 20:18:28 GMT
Connection: keep-alive

{
“identifier”: {
“type”: “dns”,
“value”: “blogue.uqtr.ca”
},
“status”: “invalid”,
“expires”: “2018-12-17T20:18:21Z”,
“challenges”: [
{
“type”: “http-01”,
“status”: “invalid”,
“error”: {
“type”: “urn:ietf:params:acme:error:unauthorized”,
“detail”: “Invalid response from http://blogue.uqtr.ca/.well-known/acme-challenge/oZ6D50NHPPYzgTaLpJRHLtrhKLRKbutQt2wYnPuDcd4: “\u003c!DOCTYPE HTML PUBLIC \”-//IETF//DTD HTML 2.0//EN\”\u003e\n\u003chtml\u003e\u003chead\u003e\n\u003ctitle\u003e403 Forbidden\u003c/title\u003e\n\u003c/head\u003e\u003cbody\u003e\n\u003ch1\u003eForbidden\u003c/h1\u003e\n\u003cp"",
“status”: 403
},
“url”: “https://acme-v02.api.letsencrypt.org/acme/challenge/2LT7sc2Ks21BpW1mMlYNXcn50UHi9mE2YbKfWUDelrg/10182356395”,
“token”: “oZ6D50NHPPYzgTaLpJRHLtrhKLRKbutQt2wYnPuDcd4”,
“validationRecord”: [
{
“url”: “http://blogue.uqtr.ca/.well-known/acme-challenge/oZ6D50NHPPYzgTaLpJRHLtrhKLRKbutQt2wYnPuDcd4”,
“hostname”: “blogue.uqtr.ca”,
“port”: “80”,
“addressesResolved”: [
“132.209.200.66”
],
“addressUsed”: “132.209.200.66”
}
]
},
{
“type”: “dns-01”,
“status”: “invalid”,
“url”: “https://acme-v02.api.letsencrypt.org/acme/challenge/2LT7sc2Ks21BpW1mMlYNXcn50UHi9mE2YbKfWUDelrg/10182356396”,
“token”: “bhl2UthF-h8s4NZNElyu6t8569NNUu0lB3Mfyqk46xI”
},
{
“type”: “tls-alpn-01”,
“status”: “invalid”,
“url”: “https://acme-v02.api.letsencrypt.org/acme/challenge/2LT7sc2Ks21BpW1mMlYNXcn50UHi9mE2YbKfWUDelrg/10182356397”,
“token”: “4ksHlkQzI-lPne1Qn7lZkR6bFRmbgW1GaVOmgKPnOYI”
},
{
“type”: “tls-sni-01”,
“status”: “invalid”,
“url”: “https://acme-v02.api.letsencrypt.org/acme/challenge/2LT7sc2Ks21BpW1mMlYNXcn50UHi9mE2YbKfWUDelrg/10182356398”,
“token”: “Y79JjiJquiQ3APmCmGXhXig5o3zVYl01Cu6pmlwABUs”
}
]
}
2018-12-10 15:18:28,664:DEBUG:acme.client:Storing nonce: 1RZ6_ZRdzJozdiEs0-mwT8bF2zEWoND0QqQLRND9cEA
2018-12-10 15:18:28,666:DEBUG:certbot.reporter:Reporting to user: The following errors were reported by the server:

Domain: blogue.uqtr.ca
Type: unauthorized
Detail: Invalid response from http://blogue.uqtr.ca/.well-known/acme-challenge/oZ6D50NHPPYzgTaLpJRHLtrhKLRKbutQt2wYnPuDcd4: “\n\n403 Forbidden\n\n

Forbidden

\n<p”

To fix these errors, please make sure that your domain name was entered correctly and the DNS A/AAAA record(s) for that domain contain(s) the right IP address.
2018-12-10 15:18:28,667:DEBUG:certbot.error_handler:Encountered exception:
Traceback (most recent call last):
File “/opt/eff.org/certbot/venv/lib64/python3.4/site-packages/certbot/auth_handler.py”, line 82, in handle_authorizations
self._respond(aauthzrs, resp, best_effort)
File “/opt/eff.org/certbot/venv/lib64/python3.4/site-packages/certbot/auth_handler.py”, line 161, in _respond
self._poll_challenges(aauthzrs, chall_update, best_effort)
File “/opt/eff.org/certbot/venv/lib64/python3.4/site-packages/certbot/auth_handler.py”, line 232, in _poll_challenges
raise errors.FailedChallenges(all_failed_achalls)
certbot.errors.FailedChallenges: Failed authorization procedure. blogue.uqtr.ca (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://blogue.uqtr.ca/.well-known/acme-challenge/oZ6D50NHPPYzgTaLpJRHLtrhKLRKbutQt2wYnPuDcd4: “\n\n403 Forbidden\n\n

Forbidden

\n<p”

2018-12-10 15:18:28,667:DEBUG:certbot.error_handler:Calling registered functions
2018-12-10 15:18:28,667:INFO:certbot.auth_handler:Cleaning up challenges
2018-12-10 15:18:28,913:WARNING:certbot.renewal:Attempting to renew cert (blogue.uqtr.ca) from /etc/letsencrypt/renewal/blogue.uqtr.ca.conf produced an unexpected error: Failed authorization procedure. blogue.uqtr.ca (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://blogue.uqtr.ca/.well-known/acme-challenge/oZ6D50NHPPYzgTaLpJRHLtrhKLRKbutQt2wYnPuDcd4: “\n\n403 Forbidden\n\n

Forbidden

\n<p”. Skipping.
2018-12-10 15:18:28,915:DEBUG:certbot.renewal:Traceback was:
Traceback (most recent call last):
File “/opt/eff.org/certbot/venv/lib64/python3.4/site-packages/certbot/renewal.py”, line 432, in handle_renewal_request
main.renew_cert(lineage_config, plugins, renewal_candidate)
File “/opt/eff.org/certbot/venv/lib64/python3.4/site-packages/certbot/main.py”, line 1170, in renew_cert
renewed_lineage = _get_and_save_cert(le_client, config, lineage=lineage)
File “/opt/eff.org/certbot/venv/lib64/python3.4/site-packages/certbot/main.py”, line 118, in _get_and_save_cert
renewal.renew_cert(config, domains, le_client, lineage)
File “/opt/eff.org/certbot/venv/lib64/python3.4/site-packages/certbot/renewal.py”, line 307, in renew_cert
new_cert, new_chain, new_key, _ = le_client.obtain_certificate(domains, new_key)
File “/opt/eff.org/certbot/venv/lib64/python3.4/site-packages/certbot/client.py”, line 353, in obtain_certificate
orderr = self._get_order_and_authorizations(csr.data, self.config.allow_subset_of_names)
File “/opt/eff.org/certbot/venv/lib64/python3.4/site-packages/certbot/client.py”, line 389, in _get_order_and_authorizations
authzr = self.auth_handler.handle_authorizations(orderr, best_effort)
File “/opt/eff.org/certbot/venv/lib64/python3.4/site-packages/certbot/auth_handler.py”, line 82, in handle_authorizations
self._respond(aauthzrs, resp, best_effort)
File “/opt/eff.org/certbot/venv/lib64/python3.4/site-packages/certbot/auth_handler.py”, line 161, in _respond
self._poll_challenges(aauthzrs, chall_update, best_effort)
File “/opt/eff.org/certbot/venv/lib64/python3.4/site-packages/certbot/auth_handler.py”, line 232, in _poll_challenges
raise errors.FailedChallenges(all_failed_achalls)
certbot.errors.FailedChallenges: Failed authorization procedure. blogue.uqtr.ca (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://blogue.uqtr.ca/.well-known/acme-challenge/oZ6D50NHPPYzgTaLpJRHLtrhKLRKbutQt2wYnPuDcd4: “\n\n403 Forbidden\n\n

Forbidden

\n<p”

2018-12-10 15:18:28,918:ERROR:certbot.renewal:All renewal attempts failed. The following certs could not be renewed:
2018-12-10 15:18:28,918:ERROR:certbot.renewal: /etc/letsencrypt/live/blogue.uqtr.ca/fullchain.pem (failure)
2018-12-10 15:18:28,919:DEBUG:certbot.log:Exiting abnormally:
Traceback (most recent call last):
File “/opt/eff.org/certbot/venv/bin/letsencrypt”, line 11, in
load_entry_point(‘letsencrypt==0.7.0’, ‘console_scripts’, ‘letsencrypt’)()
File “/opt/eff.org/certbot/venv/lib64/python3.4/site-packages/certbot/main.py”, line 1352, in main
return config.func(config, plugins)
File “/opt/eff.org/certbot/venv/lib64/python3.4/site-packages/certbot/main.py”, line 1259, in renew
renewal.handle_renewal_request(config)
File “/opt/eff.org/certbot/venv/lib64/python3.4/site-packages/certbot/renewal.py”, line 457, in handle_renewal_request
len(renew_failures), len(parse_failures)))
certbot.errors.Error: 1 renew failure(s), 0 parse failure(s)


#11

Thanks! Could you also post the Apache configuration from /etc/httpd/conf/httpd.conf?


#12

I don’t see the file.
Please don’t delete it (until we are done).


#13

Nothing has changed (still 403 forbidden):

{
“type”: “http-01”,
“status”: “invalid”,
“error”: {
“type”: “urn:ietf:params:acme:error:unauthorized”,
“detail”: “Invalid response from http://blogue.uqtr.ca/.well-known/acme-challenge/oZ6D50NHPPYzgTaLpJRHLtrhKLRKbutQt2wYnPuDcd4: “\u003c!DOCTYPE HTML PUBLIC \”-//IETF//DTD HTML 2.0//EN\”\u003e\n\u003chtml\u003e\u003chead\u003e\n\u003ctitle\u003e403 Forbidden\u003c/title\u003e\n\u003c/head\u003e\u003cbody\u003e\n\u003ch1\u003eForbidden\u003c/h1\u003e\n\u003cp"",
“status”: 403
},

#14

@rg305, I saw the file earlier. But also, this isn’t --webroot, it’s --apache, so test files manually created in .well-known/acme-challenge are of somewhat limited usefulness. (They do confirm that we have the right IP address and that there’s no firewall blocking connections, but they don’t really mirror what Certbot is going to do when run with --apache, which is different from the --webroot method.)


#15

Agreed, whenever things go “unexpected”, I always prefer the --webroot method.


#16

httpd.conf without remarks

ServerTokens OS
ServerRoot “/etc/httpd”
PidFile run/httpd.pid
Timeout 60
KeepAlive Off
MaxKeepAliveRequests 100
KeepAliveTimeout 15

StartServers 8
MinSpareServers 5
MaxSpareServers 20
ServerLimit 256
MaxClients 256
MaxRequestsPerChild 8000

Listen 80
LoadModule auth_basic_module modules/mod_auth_basic.so
LoadModule auth_digest_module modules/mod_auth_digest.so
LoadModule authn_file_module modules/mod_authn_file.so
LoadModule authn_alias_module modules/mod_authn_alias.so
LoadModule authn_anon_module modules/mod_authn_anon.so
LoadModule authn_dbm_module modules/mod_authn_dbm.so
LoadModule authn_default_module modules/mod_authn_default.so
LoadModule authz_host_module modules/mod_authz_host.so
LoadModule authz_user_module modules/mod_authz_user.so
LoadModule authz_owner_module modules/mod_authz_owner.so
LoadModule authz_groupfile_module modules/mod_authz_groupfile.so
LoadModule authz_dbm_module modules/mod_authz_dbm.so
LoadModule authz_default_module modules/mod_authz_default.so
LoadModule ldap_module modules/mod_ldap.so
LoadModule authnz_ldap_module modules/mod_authnz_ldap.so
LoadModule include_module modules/mod_include.so
LoadModule log_config_module modules/mod_log_config.so
LoadModule logio_module modules/mod_logio.so
LoadModule env_module modules/mod_env.so
LoadModule ext_filter_module modules/mod_ext_filter.so
LoadModule mime_magic_module modules/mod_mime_magic.so
LoadModule expires_module modules/mod_expires.so
LoadModule deflate_module modules/mod_deflate.so
LoadModule headers_module modules/mod_headers.so
LoadModule usertrack_module modules/mod_usertrack.so
LoadModule setenvif_module modules/mod_setenvif.so
LoadModule mime_module modules/mod_mime.so
LoadModule dav_module modules/mod_dav.so
LoadModule status_module modules/mod_status.so
LoadModule autoindex_module modules/mod_autoindex.so
LoadModule info_module modules/mod_info.so
LoadModule dav_fs_module modules/mod_dav_fs.so
LoadModule vhost_alias_module modules/mod_vhost_alias.so
LoadModule negotiation_module modules/mod_negotiation.so
LoadModule dir_module modules/mod_dir.so
LoadModule actions_module modules/mod_actions.so
LoadModule speling_module modules/mod_speling.so
LoadModule userdir_module modules/mod_userdir.so
LoadModule alias_module modules/mod_alias.so
LoadModule substitute_module modules/mod_substitute.so
LoadModule rewrite_module modules/mod_rewrite.so
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_balancer_module modules/mod_proxy_balancer.so
LoadModule proxy_ftp_module modules/mod_proxy_ftp.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule proxy_ajp_module modules/mod_proxy_ajp.so
LoadModule proxy_connect_module modules/mod_proxy_connect.so
LoadModule cache_module modules/mod_cache.so
LoadModule suexec_module modules/mod_suexec.so
LoadModule disk_cache_module modules/mod_disk_cache.so
LoadModule cgi_module modules/mod_cgi.so
LoadModule version_module modules/mod_version.so
Include conf.d/*.conf
User apache
Group apache
ServerAdmin root@localhost
ServerName blogue.uqtr.ca
HostNameLookups off
UseCanonicalName Off
DocumentRoot “/var/www/html”

Options FollowSymLinks

SetEnvIf Origin “http(s)?://(www.)?(uqtr.ca|oradvlnt.uqtr.uquebec.ca|oraprdnt.uqtr.uquebec.ca)$” AccessControlAllowOrigin=$0
Header add Access-Control-Allow-Origin %{AccessControlAllowOrigin}e env=AccessControlAllowOrigin
Header set Access-Control-Allow-Credentials true

Header set Access-Control-Allow-Origin “https://oraprdnt.uqtr.uquebec.ca

AllowOverride All
Options Indexes FollowSymLinks AllowOverride All Order allow,deny Allow from all UserDir disabled

DirectoryIndex index.html index.html.var
AccessFileName .htaccess
<Files ~ “^.ht”>
Order allow,deny
Deny from all
Satisfy All

TypesConfig /etc/mime.types
DefaultType text/plain

MIMEMagicFile /usr/share/magic.mime

MIMEMagicFile conf/magic
HostnameLookups Off ErrorLog "/var/log/httpd/errors_log" LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined LogFormat "%h %l %u %t \"%r\" %>s %b" common LogFormat "%{Referer}i -> %U" referer LogFormat "%{User-agent}i" agent CustomLog "/var/log/httpd/access_log" combined env=!nolog-request LogLevel warn ServerSignature On Alias /icons/ "/var/www/icons/"

<Directory “/var/www/icons”>
Options Indexes MultiViews FollowSymLinks
AllowOverride All
Order allow,deny
Allow from all

# Location of the WebDAV lock database. DAVLockDB /var/lib/dav/lockdb

ScriptAlias /cgi-bin/ “/var/www/cgi-bin/”
<Directory “/var/www/cgi-bin”>
AllowOverride All
Options None
Order allow,deny
Allow from all

IndexOptions FancyIndexing VersionSort NameWidth=* HTMLTable Charset=UTF-8
AddIconByEncoding (CMP,/icons/compressed.gif) x-compress x-gzip

AddIconByType (TXT,/icons/text.gif) text/*
AddIconByType (IMG,/icons/image2.gif) image/*
AddIconByType (SND,/icons/sound2.gif) audio/*
AddIconByType (VID,/icons/movie.gif) video/*

AddIcon /icons/binary.gif .bin .exe
AddIcon /icons/binhex.gif .hqx
AddIcon /icons/tar.gif .tar
AddIcon /icons/world2.gif .wrl .wrl.gz .vrml .vrm .iv
AddIcon /icons/compressed.gif .Z .z .tgz .gz .zip
AddIcon /icons/a.gif .ps .ai .eps
AddIcon /icons/layout.gif .html .shtml .htm .pdf
AddIcon /icons/text.gif .txt
AddIcon /icons/c.gif .c
AddIcon /icons/p.gif .pl .py
AddIcon /icons/f.gif .for
AddIcon /icons/dvi.gif .dvi
AddIcon /icons/uuencoded.gif .uu
AddIcon /icons/script.gif .conf .sh .shar .csh .ksh .tcl
AddIcon /icons/tex.gif .tex
AddIcon /icons/bomb.gif core

AddIcon /icons/back.gif …
AddIcon /icons/hand.right.gif README
AddIcon /icons/folder.gif ^^DIRECTORY^^
AddIcon /icons/blank.gif ^^BLANKICON^^

DefaultIcon /icons/unknown.gif

ReadmeName README.html
HeaderName HEADER.html

IndexIgnore .??* ~ # HEADER README RCS CVS *,v *,t

AddLanguage ca .ca
AddLanguage cs .cz .cs
AddLanguage da .dk
AddLanguage de .de
AddLanguage el .el
AddLanguage en .en
AddLanguage eo .eo
AddLanguage es .es
AddLanguage et .et
AddLanguage fr .fr
AddLanguage he .he
AddLanguage hr .hr
AddLanguage it .it
AddLanguage ja .ja
AddLanguage ko .ko
AddLanguage ltz .ltz
AddLanguage nl .nl
AddLanguage nn .nn
AddLanguage no .no
AddLanguage pl .po
AddLanguage pt .pt
AddLanguage pt-BR .pt-br
AddLanguage ru .ru
AddLanguage sv .sv
AddLanguage zh-CN .zh-cn
AddLanguage zh-TW .zh-tw

LanguagePriority en ca cs da de el eo es et fr he hr it ja ko ltz nl nn no pl pt pt-BR ru sv zh-CN zh-TW

ForceLanguagePriority Prefer Fallback

AddDefaultCharset UTF-8

AddType application/x-compress .Z
AddType application/x-gzip .gz .tgz
AddType application/x-x509-ca-cert .crt
AddType application/x-pkcs7-crl .crl
AddHandler type-map var
AddType text/html .shtml
AddOutputFilter INCLUDES .shtml
Alias /error/ “/var/www/error/”


<Directory “/var/www/error”>
AllowOverride All
Options IncludesNoExec
AddOutputFilter Includes html
AddHandler type-map var
Order allow,deny
Allow from all
LanguagePriority en es de fr
ForceLanguagePriority Prefer Fallback


BrowserMatch “Mozilla/2” nokeepalive
BrowserMatch “MSIE 4.0b2;” nokeepalive downgrade-1.0 force-response-1.0
BrowserMatch “RealPlayer 4.0” force-response-1.0
BrowserMatch “Java/1.0” force-response-1.0
BrowserMatch “JDK/1.0” force-response-1.0

BrowserMatch “Microsoft Data Access Internet Publishing Provider” redirect-carefully
BrowserMatch “MS FrontPage” redirect-carefully
BrowserMatch “^WebDrive” redirect-carefully
BrowserMatch “^WebDAVFS/1.[0123]” redirect-carefully
BrowserMatch “^gnome-vfs/1.0” redirect-carefully
BrowserMatch “^XML Spy” redirect-carefully
BrowserMatch “^Dreamweaver-WebDAV-SCM1” redirect-carefully

<VirtualHost *:80>
ServerAdmin blogue.uqtr.ca
DocumentRoot “/var/www/html”
ServerName blogue.uqtr.ca
ErrorLog “/var/log/httpd/errors_log”
LogFormat “%h %l %u %t “%r” %>s %b “%{Referer}i” “%{User-Agent}i”” combined
LogFormat “%h %l %u %t “%r” %>s %b” common
LogFormat “%{Referer}i -> %U” referer
LogFormat “%{User-agent}i” agent
CustomLog “/var/log/httpd/access_log” combined env=!nolog-request
LogLevel warn

Alias /awstatsclasses “/usr/local/awstats/wwwroot/classes/”
Alias /awstatscss “/usr/local/awstats/wwwroot/css/”
Alias /awstatsicons “/usr/local/awstats/wwwroot/icon/”
ScriptAlias /awstats/ “/usr/local/awstats/wwwroot/cgi-bin/”

<Directory “/usr/local/awstats/wwwroot”>
Options None
AllowOverride All
Order allow,deny
Allow from all

RewriteEngine On

RewriteRule ^/entete($|/.*) $1 [R=301,L]


#17

Please place the 1234 test file as:
/var/www/html/.well-known/acme-challenge/1234


#18

That’s not going to mimic what certbot --apache does to complete the HTTP-01 challenge. What certbot --apache does is

2018-12-10 15:18:22,162:DEBUG:certbot_apache.http_01:Adding a temporary challenge validation Include for name: blogue.uqtr.ca in: /etc/httpd/conf/httpd.conf
2018-12-10 15:18:22,163:DEBUG:certbot_apache.http_01:writing a pre config file with text:
RewriteEngine on
RewriteRule ^/.well-known/acme-challenge/([A-Za-z0-9-_=]+)$ /var/lib/letsencrypt/http_challenges/$1 [L]

2018-12-10 15:18:22,163:DEBUG:certbot_apache.http_01:writing a post config file with text:
<Directory /var/lib/letsencrypt/http_challenges>
Order Allow,Deny
Allow from all

<Location /.well-known/acme-challenge>
Order Allow,Deny
Allow from all

which doesn’t use the existing webroot at all!


#19

Agreed, but it should work for --webroot method:
certbot-auto renew --webroot -w /var/www/html

I think his config may be too confusing for certbot to mange properly.
If that works, then you can go back to just:
certbot-auto renew --quiet
The renewal config should update with the webroot info.


#20

Yes, @Stayfans, you could try using the --webroot method instead in the way that @rg305 suggested (although I haven’t yet understood why your existing approach isn’t working!).