Generate dns-01 TXT challenge with ACME-python

[Nagendra]

I am new to SSL certificate installation. I am trying to do the dns-01 validation using acme-python library. Below is the code I am using. I have added the TXT record also in domain, but in answer challenge i am getting error as "acme.messages.Error: urn:ietf:params:acme:error:malformed :: The request message was malformed :: Unable to update challenge :: authorization must be pending"

Can anyone please help what I am missing.

My domain is:grid1234.fanvase.com

I ran below commands in same order:

Register User.

1. user_key = josepy.JWKRSA(key=rsa.generate_private_key(public_exponent=65537,key_size=2048,backend=default_backend()))
2. net = client.ClientNetwork(user_key, user_agent=USER_AGENT)
   3)directory = messages.Directory.from_json(net.get(DIRECTORY_URL).json())
   4)client_acme = client.ClientV2(directory, net=net)
   5)account_created = messages.NewRegistration.from_data(email=(EMAIL), terms_of_service_agreed=True)
   6)regr = client_acme.new_account(account_created)

Generate CSR

7)pkey = OpenSSL.crypto.PKey()
8)pkey.generate_key(OpenSSL.crypto.TYPE_RSA, CERT_PKEY_BITS)
9)pkey_pem = OpenSSL.crypto.dump_privatekey(OpenSSL.crypto.FILETYPE_PEM,pkey)
10)csr_pem = crypto_util.make_csr(pkey_pem, [domain])

Create New Order

11. orderr = client_acme.new_order(csr_pem)
12. challenge = None
13. authz_list = orderr.authorizations
    14)for authz in authz_list:
    for i in authz.body.challenges:
    if isinstance(i.chall, challenges.DNS01):
    challenge = i
14. response, validation = challenge.response_and_validation(client_acme.net.key)
15. After adding the generated TXT record in my DNS and waited till propagated. I ran below command
    client_acme.answer_challenge(challenge, response)

Then I got an error saying "acme.messages.Error: urn:ietf:params:acme:error:malformed :: The request message was malformed :: Unable to update challenge :: authorization must be pending"

My web server is (include version):

The operating system my web server runs on is (include version):

My hosting provider, if applicable, is:

I can login to a root shell on my machine (yes or no, or I don't know):

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):

[Bruce5051]

Hello @Nagendra, welcome to the Let's Encrypt community.

@Nagendra

grid1234.fanvase.com

Yes using this online tool https://unboundtest.com/ I see your DNS TXT record
https://unboundtest.com/m/TXT/_acme-challenge.grid1234.fanvase.com/ZQKNWTIK

That DNS TXT record contents need to change base off of the challenge; it is not constant.

@Nagendra

"acme.messages.Error: urn:ietf:params:acme:error:malformed :: The request message was malformed :: Unable to update challenge :: authorization must be pending"

Can you show the "malformed URI"?

[rmbolger]

@Nagendra

Then I got an error saying "acme.messages.Error: urn:ietf:params:acme:error:malformed :: The request message was malformed :: Unable to update challenge :: authorization must be pending"

Your code does not appear to be checking the status of the authorizations returned. The error message is basically telling you that the authorization isn't in the right state to answer the challenge. If you had previously successfully validated a challenge, it is now probably cached and the status of those authorizations is valid when you create a new order for the same names.

[Nagendra]

@Bruce5051 & @rmbolger Thanks for the info. I am able to issue a certificate with your update.
I have a couple of doubts, it would be great if you will be able to give some heads up.

1. Do I need to register the customer each time (with same Email) to form client_acme (in above code)
2. When should I do the client_acme.answer_challenge(challenge, response). Apparently, I am able to do only one time and getting error if I execute the same command second time (if not successful in the first time).
3. Is there a way if I can customise "_acme-challenge" to a custom value.
4. I am generating SSL for my client domains. So, I want to implement this (Onboarding Your Customers with Let's Encrypt and ACME - Let's Encrypt). Can you point me in the right direction to achieve this. Below is what I am planning to do as per the documentation.
   1. Once customer gives his domain, generate a unique UUID for the domain and store it in my DB and ask the customer to add a CNAME record with name ".<client_domain>" and value "<my_domain>". (if possible to give custom key else will ask to add "_acme-challenge.<client_domain>")
   2. Once customer confirms he added the CNAME, check whether the CNAME is added or not using some python packages. (Not sure how to do this. would be great if you have any inputs)
   3. Request for challenge from ACME.
   4. Add the TXT record in my DNS as ".<client_domain>" and value generate by the challenge

As I am beginner with the process, you may find some of the questions are basic. But, would really appreciate your help in this regard.

[rmbolger]

It sounds like you need to review the actual ACME spec, RFC 8555, to get a better understanding of how the protocol works and how you should be using it in your code. It answers a lot of your questions in more detail than I'll try to give below.

@Nagendra

Do I need to register the customer each time (with same Email) to form client_acme (in above code)

No, you should be saving the account key you generated and subsequent account metadata so you can re-use it with future orders.

@Nagendra

When should I do the client_acme.answer_challenge(challenge, response)

Whenever an authorization object associated with an order has status pending

@Nagendra

Is there a way if I can customise "_acme-challenge" to a custom value

No, that's a fundamental part of the spec. Though there is a draft spec that is still in development that could offer an account specific FQDN alternative.

@Nagendra

Below is what I am planning to do as per the documentation.

Yes, having your customers create a CNAME to a DNS record you control will allow you to authorize certificates on their behalf.

@Nagendra

Add the TXT record in my DNS as ".<client_domain>" and value generate by the challenge

The TXT record goes wherever they have their CNAME pointing to. You won't want to have all of your customers point their CNAME to the same place in your DNS zone.

[Nagendra]

Everything is in place except one thing, Answering the challenge for my client domain. Ideally I want to implement the CNAME process mentioned in this article (Onboarding Your Customers with Let's Encrypt and ACME - Let's Encrypt).

1. Let's say I have a domain as help.xyz.com
2. My client has a Domain as help.abc.com
3. My client wants to map help.abc.com to help.xyz.com, so I am asking to add a CNAME record as below.
   Name : help.abc.com ,Value : help.xyz.com
4. Now , as part of ACME challenge verification, I am asking my client to add one more CNAME with below details.
   Name : _acme-challenge.help.abc.com , Value : help.xyz.com
5. I am adding a TXT record in my DNS settings as below
   Name: _acme-challenge.help.abc.com , Value : "Digest Value Given by ACME"

After following the above process, Once I answer the challenge then I am getting as "No TXT record found at _acme-challenge.help.abc.com"

Am i doing anything wrong?

[rg305]

@Nagendra

Name : _acme-challenge.abc.com , Value : help.xyz.com

Unless you intend on getting a wildcard cert, that should always be _acme-challenge.&{FQDN}
So: _acme-challenge.help.abc.com
NOT: _acme-challenge.abc.com

But, if you already have a CNAME for help.abc.com that might not be possible [nor necessary].

[Nagendra]

I am sorry, it was a typo. I have corrected it now.

But, if you already have a CNAME for help.abc.com that might not be possible [nor necessary].

I didn't get it, Can you please give some more info.
Below is what I am assuming.

3. My client wants to map help.abc.com to help.xyz.com, so I am asking to add a CNAME record as below.
   Name : help.abc.com ,Value : help.xyz.com

This record is to redirect the client traffic to my domain (created on behalf of client).

4. Now , as part of ACME challenge verification, I am asking my client to add one more CNAME with below details.
   Name : _acme-challenge.help.abc.com , Value : help.xyz.com

This record is for ACME Challenge.

[rg305]

@Nagendra

But, if you already have a CNAME for help.abc.com that might not be possible [nor necessary].

I didn't get it, Can you please give some more info.

Once you have CNAMEd an FQDN, you may no longer be able to add anything to the left of it.
Thus:
CNAME help.abc.com is effectively doing:
CNAME of all help.abc.com entries.
[ i.e. *.help.abc.com ]

You simply need to look for:
_acme-chalenge.help.abc.com

at:
_acme-challeng.help.xyz.com
[without any extra CNAME]

[rg305]

Actually after some testing, that is not always the case.

[rg305]

I would keep things as simple as possible.

If you will be hosting their content, then they will need the CNAME:
help.their-site CNAME help.your-site

If you can, use that to obtain a cert via HTTP-01 authentication.
[it's much simpler than DNS-01 authentication]

If you must use DNS-01 authentication, then (for simplicity) I'd keep the entire FQDN similar:
_acme-challenge.help.their-site CNAME _acme-challenge.help.your-site
OR maybe more like
_acme-challenge.help.their-site CNAME _acme-challenge.customers.your-site
[so as not to be confused with your own help sites' acme challenge TXT records]