Unauthorized/invalid challenge response

[bfenton]

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. https://crt.sh/?q=example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: ssltest22.smugtest.net

I ran this command: AcmePHP’s challengeAuthorization method

It produced this output: (from a logged response) Couldn’t challenge Domain=“ssltest22.smugtest.net”
Message="Challenge failed (response: {“type”:“http-01”,“status”:“invalid”,“error”:{“type”:“urn:ietf:params:acme:error:unauthorized”,“detail”:“Invalid response from https://bfenton.inside.smugmug.net/.well-known/acme-challenge/dCdpx7Wi-e7A_94T8iY0_PVH3pl2pHdUHmMX_BnUm-I [13.225.69.78]: 404”,“status”:403},“url”:"https://acme-staging-v02.api.letsencrypt.org/acme/chall-v3/57623733/nrYQ7A",“token”:“dCdpx7Wi-e7A_94T8iY0_PVH3pl2pHdUHmMX_BnUm-I”,“validationRecord”:[{“url”:“http://ssltest22.smugtest.net/.well-known/acme-challenge/dCdpx7Wi-e7A_94T8iY0_PVH3pl2pHdUHmMX_BnUm-I”,“hostname”:“ssltest22.smugtest.net”,“port”:“80”,“addressesResolved”:[“34.232.160.130”,“52.200.131.149”],“addressUsed”:“34.232.160.130”},{“url”:“https://bfenton.inside.smugmug.net/.well-known/acme-challenge/dCdpx7Wi-e7A_94T8iY0_PVH3pl2pHdUHmMX_BnUm-I”,“hostname”:“bfenton.inside.smugmug.net”,“port”:“443”,“addressesResolved”:[“13.225.69.78”,“13.225.69.64”,“13.225.69.116”,“13.225.69.60”],“addressUsed”:“13.225.69.78”}]})."

My web server is (include version): nginx

The operating system my web server runs on is (include version): Ubuntu 18.04.2 LTS

My hosting provider, if applicable, is: AWS

I can login to a root shell on my machine (yes or no, or I don’t know): no-ish. I can on a different environment but not this exact box

I’m using a control panel to manage my site (no, or provide the name and version of the control panel): no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you’re using Certbot): 1.2.0

The 404/403 seems suspicious but if I navigate to http://ssltest22.smugtest.net/.well-known/acme-challenge/dCdpx7Wi-e7A_94T8iY0_PVH3pl2pHdUHmMX_BnUm-I in a fresh browser I’m redirected to the bfenton.inside version (on https) and the challenge token I saved from the initial challenge creation is displayed.

“check your webserver” output permalink is here: https://check-your-website.server-daten.de/?i=4e5941df-e241-40cc-8bc4-66662ce2ae47

nslookup ssltest22.smugtest.net
Server:		1.1.1.1
Address:	1.1.1.1#53

Non-authoritative answer:
ssltest22.smugtest.net	canonical name = domains.inside.smugmug.net. 
Name:	domains.inside.smugmug.net
Address: 34.232.160.130
Name:	domains.inside.smugmug.net
Address: 52.200.131.149

So it’s either something to do with the http->https redirect or the discrepancy between the bfenton. and domains. hostnames (domains. is a load balancer pointing to servers with all our custom cert bundles installed)… or it could be something unusual about the staging-v2 env or a bug in AcmePHP. Since I didn’t find much about the latter two I’m assuming that I’m where the fault lies but I’m right up against the edge of my knowledge of this unfortunately. Any help would be appreciated. Thanks.

Brian

[_az]

How is the /.well-known/acme-challenge/ request handler implemented? Is it looking something up from a database?

The validation record looks normal to me (https://acme-staging-v02.api.letsencrypt.org/get/chall-v3/57623733/nrYQ7A).

The server follows the redirect, and then reports that your server responded to the second request with an HTTP 404 with no response body. Which is indeed what happens when we request a non-existent challenge token:

$ curl -i https://bfenton.inside.smugmug.net/.well-known/acme-challenge/12345
HTTP/2 404
content-type: text/html; charset=utf-8
content-length: 0

Do you have logs from this request handler?

Are you doing anything like removing the challenge token after it is accessed?

[bfenton]

It’s handled by the AcmePHP client, it writes to the filesystem at /.well-known/acme-challenge/$token when the challenge payload is received. We also save that in a database… and dang there’s a special controller set up just to handle these routes that I didn’t know about. Will document that and trace through the logic here. Hopefully that’s all I needed, but if not I’ll report back. Thanks!