DNS validation remains in pending status

Hello,
I had preciously used the Letencrypt client which has worked good with http validation ,
while HTTP validation is not preferred(not all servers run http) in the cluster env and it is time to move to the DNS validation.

My domain is:
thither.direct

I ran this command:
python:

# account key is a new or reused.
client = acme_client.Client(directory='https://acme-v01.api.letsencrypt.org/directory', key=key)

new_authz = messages.NewAuthorization(identifier=messages.Identifier( typ=messages.IDENTIFIER_FQDN, value=str(fqdn)))

response = client.net.post(client.directory.new_authz, new_authz)
rsp_challs = response.json()

authz = messages.Authorization.from_json(rsp_challs)
# DNS record created with the values of (zero key is DNS chall):
# - domain: authz.challenges[0].validation_domain_name(fqdn)
# - TXT value: authz.challenges[0].validation(key)

after the DNS record set,
the process waits for the status of valid to proceed to cert-request with the URI of the challenge (uri from rsp_challs['challenges'] type 'dns-01')

response = client.net.get(token_uri) # _https://acme-v01.api.letsencrypt.org/acme/challenge/kmRLowREeyMHas8yXzak85RuGlXclOZLyc1QGW8Scb0/1596100859_
rsp = response.json()
if rsp[u'status'] == u'valid':

the wait process goes 30 sec wait between checkups, which had no results.

------- client.net.get(token_uri)
{'Server': 'nginx', 'Content-Type': 'application/json', 'Content-Length': '222', 'Boulder-Request-Id': 'P_mMapTMs-
GpC7JNvZqSpn_MFeWlczOpRvI09enx9XM', 'Link': 'https://acme-v01.api.letsencrypt.org/acme/authz/kmRLowREeyMHas8yXzak85RuGlXclOZLyc1QGW8Scb0;rel="up"', 'Location': 'https://acme-v01.api.letsencrypt.org/acme/challenge/kmRLowREeyMHas8yXzak85RuGlXclOZLyc1QGW8Scb0/1596100859', 'Replay-Nonce': '0Ql1iELEPpSp821s0tjeNoS4txGhUFwpRDPfIbhVNF8', 'Expires': 'Sat, 22 Jul 2017 19:53:02 GMT', 'Cache-Control': 'max-age=0, no-cache, no-store', 'Pragma': 'no-cache', 'Date': 'Sat, 22 Jul 2017 19:53:02 GMT', 'Connection': 'keep-alive'}

{u'type': u'dns-01', u'status': u'pending', u'uri': u'https://acme-v01.api.letsencrypt.org/acme/challenge/kmRLowREeyMHas8yXzak85RuGlXclOZLyc1QGW8Scb0/1596100859', u'token': u'ASkkIgmAgJOq4kbEnnD8uJPG-OkSueMXxBLKm1OSBOI'}

status:pending

The DNS record is set instantly and It is responded at:
https://dns.google.com/query?name=_acme-challenge.thither.direct&type=TXT&dnssec=true
and so as at,
https://unboundtest.com/m/TXT/_acme-challenge.thither.direct/HKXKKLJN

I hope it is enough information to determine the cause of status not changed, It would be clear if status has been invalid/expired, while I'm unable to determine why status remains Pending.

Thank You,
Kashirin Alex

Ah, I went through this surprise at some point :slight_smile: Basically what happens is that as long as there is some “other” challenge in the ‘valid’ state, the one you requested will remain ‘pending’. Say you have validated the domain successfully via HTTP and then trying to validate it again via DNS (with HTTP validation result still being there). You will see that pending state not changing. The client needs to account for this case (and for example just re-use ‘valid’ result), otherwise you may get an infinite loop.

If that is a case, that could be a reason for some of the domains, while there are several which would have failed(stay in pending) from the first time.
Is the valid preservation state depends on the account?

  • and Would that be the same if there was a valid state for DNS (and it was missed) and state remain pending as there is already a validation with valid state for a given domain ?
    – and another thing, last time I used the HTTP validation was 3 month ago.

Thanks,
Kashirin Alex

The current new_authz challenge ,
which would be checking up for the state on interval of 1 minutes for 8 hours.

Here is a log, if that can help:

2017-07-23 17:43:00.351075: ---- NewAuthorization----- START
2017-07-23 17:43:00.451340: NewAuthorization(identifier=Identifier(typ=IdentifierType(dns), value='ns2.thither.direct'), challenges=None, combinations=None, status=None, expires=None, resource='new-authz')
2017-07-23 17:43:00.551758: <Response [201]>
2017-07-23 17:43:00.652040: {'Server': 'nginx', 'Content-Type': 'application/json', 'Content-Length': '1006', 'Boulder-Request-Id': 'UUo2DT8IssRspMbBiwR1uJaOjpCzPU7EYJtEhjNbC2M', 'Boulder-Requester': '19042459', 'Link': 'https://acme-v01.api.letsencrypt.org/acme/new-cert;rel="next"', 'Location': 'https://acme-v01.api.letsencrypt.org/acme/authz/SMNEeR0KCdqFQLuJBU9GfagXh107uKlUG3BfBMnXrfo', 'Replay-Nonce': 'qdAk_-1tuTUzgUHCY4YfsUyZuXVrrdw-iL-xNJQKTKU', 'X-Frame-Options': 'DENY', 'Strict-Transport-Security': 'max-age=604800', 'Expires': 'Sun, 23 Jul 2017 15:43:00 GMT', 'Cache-Control': 'max-age=0, no-cache, no-store', 'Pragma': 'no-cache', 'Date': 'Sun, 23 Jul 2017 15:43:00 GMT', 'Connection': 'keep-alive'}
2017-07-23 17:43:00.752790: {u'identifier': {u'type': u'dns', u'value': u'ns2.thither.direct'}, u'status': u'pending', u'expires': u'2017-07-30T15:43:00.246154769Z', u'challenges': [{u'type': u'http-01', u'status': u'pending', u'uri': u'https://acme-v01.api.letsencrypt.org/acme/challenge/SMNEeR0KCdqFQLuJBU9GfagXh107uKlUG3BfBMnXrfo/1601585146', u'token': u'Ok3pGfR26tSdswABtu6RSOMqe7-T5IQ6l8kGkATFGy0'}, {u'type': u'dns-01', u'status': u'pending', u'uri': u'https://acme-v01.api.letsencrypt.org/acme/challenge/SMNEeR0KCdqFQLuJBU9GfagXh107uKlUG3BfBMnXrfo/1601585147', u'token': u'MGCIxfB_PgJUNU5UXo9ep4Q87e_9x4TcXlxkxWysCyA'}, {u'type': u'tls-sni-01', u'status': u'pending', u'uri': u'https://acme-v01.api.letsencrypt.org/acme/challenge/SMNEeR0KCdqFQLuJBU9GfagXh107uKlUG3BfBMnXrfo/1601585148', u'token': u'cA7kHMXSKIZH5BmFEd2hXpC8G187hZY265dVp051FFo'}], u'combinations': [[0], [2], [1]]}
2017-07-23 17:43:00.853210: ---- NewAuthorization----- END
2017-07-23 17:43:05.005241: try: 1
2017-07-23 17:43:05.105511: client.net.get(token_uri): https://acme-v01.api.letsencrypt.org/acme/challenge/SMNEeR0KCdqFQLuJBU9GfagXh107uKlUG3BfBMnXrfo/1601585147
2017-07-23 17:43:05.205842: {'Server': 'nginx', 'Content-Type': 'application/json', 'Content-Length': '222', 'Boulder-Request-Id': '_fzfb2rD2qtXtUp-MBfQcwO6TicdzuFTG1QL2UI6xUI', 'Link': 'https://acme-v01.api.letsencrypt.org/acme/authz/SMNEeR0KCdqFQLuJBU9GfagXh107uKlUG3BfBMnXrfo;rel="up"', 'Location': 'https://acme-v01.api.letsencrypt.org/acme/challenge/SMNEeR0KCdqFQLuJBU9GfagXh107uKlUG3BfBMnXrfo/1601585147', 'Replay-Nonce': 'IkQXZyfSzZMe55dCKB4_TuzMUdzQ54nOFnobPl4BUnQ', 'Expires': 'Sun, 23 Jul 2017 15:43:04 GMT', 'Cache-Control': 'max-age=0, no-cache, no-store', 'Pragma': 'no-cache', 'Date': 'Sun, 23 Jul 2017 15:43:04 GMT', 'Connection': 'keep-alive'}
2017-07-23 17:43:05.306250: {u'type': u'dns-01', u'status': u'pending', u'uri': u'https://acme-v01.api.letsencrypt.org/acme/challenge/SMNEeR0KCdqFQLuJBU9GfagXh107uKlUG3BfBMnXrfo/1601585147', u'token': u'MGCIxfB_PgJUNU5UXo9ep4Q87e_9x4TcXlxkxWysCyA'}
2017-07-23 17:43:05.406585: status:pending
2017-07-23 17:44:05.191213: try: 2
2017-07-23 17:44:05.291465: client.net.get(token_uri): https://acme-v01.api.letsencrypt.org/acme/challenge/SMNEeR0KCdqFQLuJBU9GfagXh107uKlUG3BfBMnXrfo/1601585147
2017-07-23 17:44:05.391754: {'Server': 'nginx', 'Content-Type': 'application/json', 'Content-Length': '222', 'Boulder-Request-Id': '0WHu8FoobO3JjqCmNOkS8CGC9kvGEGKMICJA6J3Fu2U', 'Link': 'https://acme-v01.api.letsencrypt.org/acme/authz/SMNEeR0KCdqFQLuJBU9GfagXh107uKlUG3BfBMnXrfo;rel="up"', 'Location': 'https://acme-v01.api.letsencrypt.org/acme/challenge/SMNEeR0KCdqFQLuJBU9GfagXh107uKlUG3BfBMnXrfo/1601585147', 'Replay-Nonce': 'MvfjiUR7yNZBAhryFNULoXC-jYoZw0XKiFIhERgspL8', 'Expires': 'Sun, 23 Jul 2017 15:44:05 GMT', 'Cache-Control': 'max-age=0, no-cache, no-store', 'Pragma': 'no-cache', 'Date': 'Sun, 23 Jul 2017 15:44:05 GMT', 'Connection': 'keep-alive'}
2017-07-23 17:44:05.492143: {u'type': u'dns-01', u'status': u'pending', u'uri': u'https://acme-v01.api.letsencrypt.org/acme/challenge/SMNEeR0KCdqFQLuJBU9GfagXh107uKlUG3BfBMnXrfo/1601585147', u'token': u'MGCIxfB_PgJUNU5UXo9ep4Q87e_9x4TcXlxkxWysCyA'}
2017-07-23 17:44:05.592446: status:pending
2017-07-23 17:45:05.386532: try: 3

The dns record would have been set before the 1st state checkup,
https://dns.google.com/query?name=_acme-challenge.ns2.thither.direct&type=TXT&dnssec=true

I’m not particularly familiar with that client library, so I might be missing something, but I think you’re missing a step here. This is the typical request flow of requests for ACME (from the acme-01 spec):

          +--------------------+----------------+--------------+
          | Action             | Request        | Response     |
          +--------------------+----------------+--------------+
          | Register           | POST new-reg   | 201 -> reg   |
          |                    |                |              |
          | Request challenges | POST new-authz | 201 -> authz |
          |                    |                |              |
          | Answer challenges  | POST challenge | 200          |
          |                    |                |              |
          | Poll for status    | GET  authz     | 200          |
          |                    |                |              |
          | Request issuance   | POST new-cert  | 201 -> cert  |
          |                    |                |              |
          | Check for new cert | GET  cert      | 200          |
          +--------------------+----------------+--------------+

In your code, I see 1-2 and 4-6, but the step where you answer the challenge (POST challenge) is missing. In ACME, the challenges aren’t continuously checked, but rather the ACME server waits for you to tell it that you’re ready, and only then is the check performed. Without this step, the authorization remains in a pending state.

I think the answer_challenge method is what you’re looking for in acme-python.

Thanks for the info, my client is based on http://letsencrypt.readthedocs.io/projects/acme/en/latest/api/client.html
My process is missing answer_challenge

now I'm doing :

auth_pkey = crypto.load_privatekey(crypto.FILETYPE_PEM, key_data)
key = acme_jose.JWKRSA(key=auth_pkey.to_cryptography_key())

client = acme_client.Client(directory='https://acme-v01.api.letsencrypt.org/directory', key=key) 
# new_reg if a new key   

authzr = client.request_domain_challenges(str(fqdn))
for authzr_challb in authzr.body.challenges:
    if authzr_challb.typ == 'dns-01':
        chall = authzr_challb
        break
client.answer_challenge(chall, challenges.DNS(token=chall.token).gen_response(key))

and that brings to an error:

/acme/client.py", line 577, in _check_response
raise messages.Error.from_json(jobj)
Error: urn:acme:error:malformed :: The request message was malformed :: Unable to update challenge :: provided key authorization was incorrect

response data for post of challenges.DNS(token=chall.token).gen_response(key)
It is:

DNSResponse(resource='challenge', validation=JWS(payload='{"token": "tk_aQzB7PMNSNaBS0XdoC85v81BRASVDAbCA6zl2Ij0", "type": "dns"}', signatures=(Signature(combined=Header(alg=RS256, jku=None, jwk=JWKRSA(key=<ComparableRSAKey(<cryptography.hazmat.backends.openssl.rsa._RSAPublicKey object at 0x000055b443ab91a0>)>), kid=None, x5u=None, x5c=(), x5t=None, x5tS256=None, typ=None, cty=None, crit=()), protected='', header=Header(alg=RS256, jku=None, jwk=JWKRSA(key=<ComparableRSAKey(<cryptography.hazmat.backends.openssl.rsa._RSAPublicKey object at 0x000055b443ab91a0>)>), kid=None, x5u=None, x5c=(), x5t=None, x5tS256=None, typ=None, cty=None, crit=()), signature='>\xbf\x1fb\xd3\x1a\xfc\xb6\xa1\xd0\xb5\xa5"\x89\xb1N\x96\x1dR/\x00[9\xc1\xfe\x7f\xd2[L\t\xff\x86\xff\x88<\xdbH\xd5\xd5P\xbb\\\x14\xa6\x0f1(\x94\xa4\x93\x17\xefw\xf0\x98\x93P\xc7\xa9\xb1\xe0dF\x80Z\xcfh\x90\x1e)\xac\x1fq\xe3J\xf5\xa4\xb1\xe06y\xa2\xc5h\xad:!\xe2L\x9a\xfc\xc4\xb1\xfc\x98\xb2\x9d\xf3\x89P\xda\xe1\xe0\x8e\xaf~\xc5\xc4\x13\xef\xdc\xac/\xfaM\x1d\xdc\x06y\xd9i\xfe\x9c\x13\xd5\xa3\xf6\xff\x8b\xe4\xcdZ|Q\x93\xf4[\xe1zCDRsMS\x07x\xbby\xb0\xa3\xf7<\xf6]\xbbU\xc1SQl\xa9\xf6\xa6\xecc\xe9\x84\x1e\xf1\\[\xb8\xeb"\xdd\xdc\x86Y1Uv\xe9\xa4cm\xe8\xb9\xf6\xcb\x97\x9c\xe9_v\x03A\xb7\xdf\x84\xf0.Se\r \x03I\xc95\xe84\x89w\xb1\xf2\xb3\x88\x86\xfc\xe3\xd1\xc11b\x05A<C\x19\nq&\xf3\xb1\xf67\x8a\xe1g\xca\xafh\x0ed\xf8V\xb7Y\xa4\xf9\x13\x8f\x9c^\x96\xb0Xc\x87\xb5\xe3\x9a\x1b\xc4`\x82+\x18\xec1\x1c\xba\xe9\xe9\x8f\xa7\x19]Q\xaa\xefV\xa1\x1c\x86\xb0\xe0\xb8\xb5Y\x00p\xa6\xbcj\xae\xdd\x12\x8c\xe9)%\xfb\xb3\xe8\xe3\x89\x9e\xea\xab\xe2\xfd\xfa\xd7G\xdc\xe9<G\xe3\x0cs\xeb%\x90\x1fQ\xa4\xfc\xc4)#\xb7B\xb7\xf5o?\xcd\xfe\xc5Zi.\x0b\x18\xe5\xddoS-z\xc4\xec@\xaa\x0b\x8b\x8a9~\x95\xe6L\xf6\x1dA|\xb7\xe4\x10y \xbf\xed\x0cF\xe2,F\x7f\xc8\x8f\xd4\t-%\xd3H5g\xee\xf3\x02\x99\xfbnf\xea\xd6," \xaf\xb482\x06\xe7T\x8b2\x8a+C\x969\xfeA\xaa]\x05BI\x08;\t\xae\xffo\xa1\xb2\x1a\x9c\xb3\x9eElO\xb0\x16\xf7\xf3#\x9d\x99]|\xab\xfc\x026\xf2\x1d~>\x17v\x96\xf5\x98\x0b\xb4\x8c\x06\xca\xd5\x98\\xefZ!\x19r\xba\x97\xe4\xa7\x9d\xccv\xba<\xb2\xc5\x88\x90\xa2\x8e\xcc\xfb\xbdA\x9e\x0b~\x1a\xa5\xaf\x17d\x83g-\x0f\xe7\x0b\x92\xdd'),)))

Trying as well: (error is the same)

ans = acme_jose.JWS.sign(
            key=key, alg=acme_jose.RS256,
            payload=str(
                '{"resource": "challenge", "type": "'+chall.typ+'", "token": "' +
                acme_jose.b64.b64encode(chall.token)+'"}').encode('utf-8')) 
response = client.net.post(chall.uri,  challenges.DNSResponse(validation=ans))

Will be great to know the proper way to debug the format of the data to post.

Thanks,
Kashirin Alex

Followed the data for post at :

where the payload is "resource": "challenge", "keyAuthorization": 'https://github.com/certbot/certbot/blob/8011fb2879659008528e8b6ba5c04ac3de1e63a0/acme/acme/challenges.py#L163'

equivalent is:

ans = acme_jose.JWS.sign(
            key=key, alg=acme_jose.RS256,
            payload=str(
                '{"resource": "challenge", "keyAuthorization": "'+acme_jose.b64encode(
                    chall.key_authorization(key).encode("utf-8"))+'"}'
            )) 
challr = client.answer_challenge(chall, challenges.DNSResponse(validation=ans))

-- keyAuthorization, being b64 or not, did not make the difference.

and the response posted is:

JWS(payload='{"resource": "challenge", "keyAuthorization": "Sm9EVGlJUUhxNTBNa2JSU0NXMjZJWVN0ZWZFdWpwN1F6TzhiVzJrbjlndy5VNnRqVjZuRVlwbmhTQlFSakttUnV2NlNrcFQ2N3poUjdDTnRRRUI3SFpR"}', signatures=(Signature(combined=Header(alg=RS256, jku=None, jwk=JWKRSA(key=<ComparableRSAKey(<cryptography.hazmat.backends.openssl.rsa._RSAPublicKey object at 0x0000562cb2895c20>)>), kid=None, x5u=None, x5c=(), x5t=None, x5tS256=None, typ=None, cty=None, crit=()), protected='', header=Header(alg=RS256, jku=None, jwk=JWKRSA(key=<ComparableRSAKey(<cryptography.hazmat.backends.openssl.rsa._RSAPublicKey object at 0x0000562cb2895c20>)>), kid=None, x5u=None, x5c=(), x5t=None, x5tS256=None, typ=None, cty=None, crit=()), signature="\x0f\x81+{\xea\xd6@v\x0f\xa4\xd3(m\xefm\xda\xbc]\xdcZ\x89W\x11\x8c9\x02\xbdy\x82\x9c\xf29&T\xb2\xa4#4\xff\xeb3\xe1S>\x98\x9b\xaf\xe5\xe2-A\xfd\xa4\x9c2\x1bLg\xf2a\x80\x13\xa3x\xc4zB\xcbW$\xdc\xad\x04\xb2M\xe7\x00\xd7/\xfe\x06\x83\x01t\xe0\x88\x0f\x10,\xd9\x9d\x0b\xde@\xc9o\xda\xaf\xed\xf1D\x0e\xaf\xac\x1a\x9a\xb9y\x14\xd36\xf3+\x98-\xd2\xfeb\x8e\xa7HId\x0f\xbfI\xa6@\xd4J:\x0c\x9c\x94O\x889\x1c^\r\x00\xd0\x99\xbf/^\xc6\x9d )\xfe\xf2\xa5{\x19W\xad\x13\xe4\x88\x10m\xd8#\xcd\x8d\r\x9e\xb6\xa0+^\xa3\xbaes3K\x9b%\xcdkBX\xa7\xf6B\x93<*29o\xb6\xb1)\x1c{\xc6d3\xf3\xf6Qq\xfa\xe3\xd9\x1e\x06\x16\x19\x0cE\x98\x8a\x15\xd0\xd2\xb3\x97>\xbd\xd4\x9b+\xc7\xb3\x82W>\x96\xa2\xa9\x0e'\xa5\x90\xd4[\x04r\xcb\xa3\xa1D;\xde\x8b\xadX\x9e\xa3\x14\xac\x16\xca\x8d\xd0K%\x9aL\xf9vG\xe8\x8b^\x93\x88\x01~\xe0K5\xb6Z\xa1&\xd2\x9d\x19b8\xd58~\xa0fs\xdc\xff\xe7\x8c\x93\xbd\xffA3buc\x07p\xb7\x97\xd0N\xa9\x8a'\x10\xa1\x05\xdf{\x1a\xb0l\xc3U\xc8\x0fO\xb4\xe5\xaf\xb9\xd2\xf6\x9ft\x9aP\xe6TZ\xde\xe1\xb0w\x1d`\xe1\x83I\x10\xf1$\xd0\xfaC\xf8F0\x15\xa5\x85cp\x11\xf2\xbd7#}}\xfb\xe0\xcd\xcd\xf6xt\xc4\x13\x1c\xfc\xf5\xf4oh\x18\x9d\xb3\x0f\xc7#\xe8\x89\xbaD\xc5\xfc{\xed`\xeb07\x82v\x96h\xb7\xc87\x1e\xb2r-\xafoz\x10\xdeC\xfb\x1a\xec\xb8@\^\x05\x0eY\x7f\xc2\x99\x95\x91\x94\x8e\n\x82&-\xd4\x97X{\xb9\x10G\xb0Vg{m$\xaf0\x12\xce#\xecH\xbe\x13o\x9c\xd5\xbc\xac\xfc\xd1Z\xf2\x80\xd40A\x82\xb4\x1aN\x93k-\x04\x83Y\xe5\x96\xf5\xf0\x86\xfcw\xf6\x03\xfcF'\x1a\xe0\xe3\xc2\x11\xcd\x9c\xf8\x0c\xce\xacm\xca\x8f"),))`

-- still no success.

and it is Working,

client = acme_client.Client(directory='https://acme-v01.api.letsencrypt.org/directory', key=key)
authzr = client.request_domain_challenges(str(fqdn))
chg = False
        for authzr_challb in authzr.body.challenges:
            if authzr_challb.typ == 'dns-01':
                chg = authzr_challb
                break
chg = client.answer_challenge(chg,  chg.chall.response(key))
response = client.net.get(chg.uri)
rsp = response.json()
if rsp[u'status'] == u'valid':
        response = client.request_issuance(csr, [authzr])
        crt = response.body.wrapped

Thanks,
Kashirin Alex

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.