Pebble ACME server fails at the attempt to validate http01 challenges

[vizual1]

Hello,
I'm new to ACME and I'm implementing an ACME client from ground up in python to learn about it. Right now, I'm stuck at the challenge validation process.

To test the keyAuthorization, I created a HTTP server that responds to "GET /.well-known/acme-challenge/" requests from the ACME server to validate the challenge.

The jwk file used to generate the keyAuthorization is:

{'e': 'AQAB', 'kty': 'RSA', 'n': 'xdE3j-LP3uLdiEchiwSkskme1lyhxjlPcx2gwGZF1f0ECWhL-rlqQAtllQTGMJDxlA5_aahPejDj_9Roc0C1h9rPJz1LmmzqOPLVmQo9BrDBuj-cNx6o7kK1doS2eUiDK1GUHDHrZmzpurYyBlL0SSSkobIUpbO1q850ckjOwdvN7f8YCq6ifuJFgkYA3nRzuHWjkm4EMxkJjU5ROpJuDmebbKLbgzGJKYDhzYSzFV62hAFLRQ0iBOdBPpPwuMViitvLFH678RIc_qQ1hKwl_52fPg7GY0pxr6M_riqQUkU9ou_O3gwcv4NWTZS43nPoUn6dYgSTEq6w7P74eUPAVQ'}

The keyAuthorization file contains:

J744SMZOoyezbUmo9WQcg1b_qWLhvPbhyCerBYg80PA.OBuXVbrhksHr03JEgn1eMjZeUpdbnRJlFzQsoHWraFo

The HTTP server that responds to the ACME server validation request is implemented with the flask library:

with open("./.well-known/acme-challenge/" + token, 'rb') as f:
        keyAuthorization = f.read().decode("utf-8")

response = app.make_response(str(keyAuthorization))
response.headers['Content-Type'] = 'application/octet-stream'

return response

Then commandline shows:

pebble-main-pebble-1  | Pebble 2023/10/22 18:14:52 Starting 3 validations.
pebble-main-pebble-1  | Pebble 2023/10/22 18:14:52 Attempting to validate w/ HTTP: http://localhost:5002/.well-known/acme-challenge/J744SMZOoyezbUmo9WQcg1b_qWLhvPbhyCerBYg80PA
pebble-main-pebble-1  | Pebble 2023/10/22 18:14:52 Attempting to validate w/ HTTP: http://localhost:5002/.well-known/acme-challenge/J744SMZOoyezbUmo9WQcg1b_qWLhvPbhyCerBYg80PA
pebble-main-pebble-1  | Pebble 2023/10/22 18:14:52 Attempting to validate w/ HTTP: http://localhost:5002/.well-known/acme-challenge/J744SMZOoyezbUmo9WQcg1b_qWLhvPbhyCerBYg80PA
pebble-main-pebble-1  | Pebble 2023/10/22 18:14:52 authz t4r2ZubnD0fm5TKlUCGfssMM7w7Zg8OCp9lPogTN6PA set INVALID by completed challenge 8r9E3gXey7v9YPV9rWW4fC6OyeRz-RmgGlPmAtdNX24
pebble-main-pebble-1  | Pebble 2023/10/22 18:14:52 order _GRjIjP1zcop3WU78Vd8N6WLTrLL0AJkc5v5bcbMmR0 set INVALID by invalid authz t4r2ZubnD0fm5TKlUCGfssMM7w7Zg8OCp9lPogTN6PA

I'm assuming the keyAuthorization is incorrect or the HTTP server response is wrongly configured, but I cannot figure it out. Any help is appreciated.

[rg305]

Hi @vizual1, and welcome to the LE community forum

I'm not an expert in this area and I might be mistaken about anything I say here.
And surely someone more knowledgeable will come around to help.
But until then...
While glancing at your post I noticed there is no mention of base64 encoding.
I thought that was required in there somewhere.

[orangepizza]

as acme http challange file is reflective : first part before . of answer file should be same with what url acme server visit with so you failed to get right auth path
keyAuthorization = token || '.' || base64url(Thumbprint(accountKey))
and acme server visits well-known/acme-challange/token
I think you processed token wrong
from pebble log token it set for auth was

ZR_D9XQw5FocCDWsgEL20W2H6dGnd3jMbWbgrn1fv38

but token part of your keyauth file is

uKzGhvg2SkOMboZE5K8xSWNkeaNcAHOuSDXtKFNHwI8

[vizual1]

I'm sorry, I forgot to mention it. Yes, I did do base64 encoding.

For more details, the jwk file is also used for the newAccount request (which works fine) too:

jwk = {
      "e": e_base64url,
      "kty": "RSA",
      "n": n_base64url
}

The keyAuthorization is created from jwk and the token:

public_json = json.dumps(jwk, sort_keys=True, separators=(',', ':'))
thumbprint = b64(hashlib.sha256(public_json.encode('utf-8')).digest())
keyAuthorization = "{0}.{1}".format(token, thumbprint)

I also double checked with the ACME client, and also did a request to the HTTP server. The HTTP server returns a response at response.text, which is the keyAuthorization.

[vizual1]

I'm sorry, I think I just copied it wrong. I edited it.

[orangepizza]

than I kinda guess whitespace is somewhat different from jose expects and Jason.dump does, which will make hash different

there is entire rfc about just it

RFC 7638: JSON Web Key (JWK) Thumbprint

tldr: remove whitespace and line change from jwk

[vizual1]

@vizual1

public_json = json.dumps(jwk, sort_keys=True, separators=(',', ':'))

In the json.dumps the separators=(',', ':') already eliminate the whitespace and line change from jwk.

[Osiris]

@vizual1

I'm assuming the keyAuthorization is incorrect or the HTTP server response is wrongly configured, but I cannot figure it out.

Pebble should return an error message to the ACME client, which would be very helpful. It seems the Pebble log itself doesn't provide much information as to WHY the challenge fails.

You're running your challenge response HTTP server on port 5002, right?

[vizual1]

Yes, the HTTP server runs on port 5002.

In the ACME client, I use a while loop to check if the status of the request is still "pending", and continued to send requests after every 2 seconds of wait time. The first request, returns:

{
   "type": "http-01",
   "url": "https://localhost:14000/chalZ/G49OFdBN3cTEmXoGIInwHFfwqOlqInzx0KZGhiM6elQ",
   "token": "jjrgvOamn_hrWygxIeUOHKou3_gEQrvN8q5qAhPVrhk",
   "status": "pending"
}

The next request are already invalid,

{
   "type": "urn:ietf:params:acme:error:malformed",
   "detail": "Cannot update challenge with status invalid, only status pending",
   "status": 400
}

The pebble logs:

pebble-main-pebble-1  | Pebble 2023/10/22 20:33:50 GET /dir -> calling handler()
pebble-main-pebble-1  | Pebble 2023/10/22 20:33:50 HEAD /nonce-plz -> calling handler()
pebble-main-pebble-1  | Pebble 2023/10/22 20:33:51 POST /sign-me-up -> calling handler()
pebble-main-pebble-1  | Pebble 2023/10/22 20:33:51 POST /order-plz -> calling handler()
pebble-main-pebble-1  | Pebble 2023/10/22 20:33:51 There are now 5 authorizations in the db
pebble-main-pebble-1  | Pebble 2023/10/22 20:33:51 Added order "v58MSzX0Hz8iYRjKc8rQnsrwBVCQKcQoNFKY8ZdgCGg" to the db
pebble-main-pebble-1  | Pebble 2023/10/22 20:33:51 There are now 5 orders in the db
pebble-main-pebble-1  | Pebble 2023/10/22 20:33:51 POST /authZ/ -> calling handler()
pebble-main-pebble-1  | Pebble 2023/10/22 20:33:53 POST /chalZ/ -> calling handler()
pebble-main-pebble-1  | Pebble 2023/10/22 20:33:53 Pulled a task from the Tasks queue: &va.vaTask{Identifier:acme.Identifier{Type:"dns", Value:"localhost"}, Challenge:(*core.Challenge)(0xc0000b1720), Account:(*core.Account)(0xc00006cba0)}
pebble-main-pebble-1  | Pebble 2023/10/22 20:33:53 Starting 3 validations.
pebble-main-pebble-1  | Pebble 2023/10/22 20:33:53 Attempting to validate w/ HTTP: http://localhost:5002/.well-known/acme-challenge/jjrgvOamn_hrWygxIeUOHKou3_gEQrvN8q5qAhPVrhk
pebble-main-pebble-1  | Pebble 2023/10/22 20:33:53 Attempting to validate w/ HTTP: http://localhost:5002/.well-known/acme-challenge/jjrgvOamn_hrWygxIeUOHKou3_gEQrvN8q5qAhPVrhk
pebble-main-pebble-1  | Pebble 2023/10/22 20:33:53 Attempting to validate w/ HTTP: http://localhost:5002/.well-known/acme-challenge/jjrgvOamn_hrWygxIeUOHKou3_gEQrvN8q5qAhPVrhk
pebble-main-pebble-1  | Pebble 2023/10/22 20:33:53 authz hl2n-ItedZq94vZ89_DB6XXYctmXIMadSeLVPuWzGV8 set INVALID by completed challenge G49OFdBN3cTEmXoGIInwHFfwqOlqInzx0KZGhiM6elQ
pebble-main-pebble-1  | Pebble 2023/10/22 20:33:53 order v58MSzX0Hz8iYRjKc8rQnsrwBVCQKcQoNFKY8ZdgCGg set INVALID by invalid authz hl2n-ItedZq94vZ89_DB6XXYctmXIMadSeLVPuWzGV8

[Osiris]

@vizual1

The next request are already invalid,

I'm not sure what your intentions with that "next request" was, but it isn't the correct one for polling the challenge.

The Pebble log isn't very helpful, we'd need the output of the actual invalidated challenge object.

[orangepizza]

and as pebble is local run you can add more prints as much as you want

[Osiris]

@orangepizza Pebble is written in Go, so it would require compiling the source code. It's unfortunately not written in an interpreted language such as Python.

[vizual1]

The "next request" is just a retry until the status of the response isn't "pending" anymore. Between each "pending" request, I wait for 2 seconds.

I tried to get the output, but is seems like the problem is different than I thought. For some reason the validation request of the ACME server doesn't reach the HTTP server, even though the ACME client can create GET request to http://localhost:5002/.well-known/acme-challenge/_token_ without problems, and get the keyAuthorization back.

[orangepizza]

til it had binary releases (2years ago latest)
i thought everyone using that are compiling pebble thenselves

[rg305]

@vizual1

For some reason the validation request of the ACME server doesn't reach the HTTP server

What is the URL/FQDN of the HTTP server?
[I suspect it will match the GET you included, but I just want to be sure of it]

[orangepizza]

iirc unless configed differently pebble will resolve everything to a 127.0.0.1 and connect localhost
but looking at domain OP used dont think this is problem

[orangepizza]

wait are you using docker to run pebble? than the container have different localhost domain from host

[vizual1]

Yes, I use docker to run pebble. I think that is the problem, so I have to either change the configuration of pebble docker-compose.yml or to configure the HTTP server to another IP address to test it.

[orangepizza]

just use binary version from github: yout want it run on same vm or bare metal with your client

[vizual1]

That might be a stupid question, but how do I set pebble in the $PATH? Where is the binary entry point? I'm using Windows 10 and I cannot find any bin folder where I can set the path.