DNS01: How is the challenge supposed to be formatted?


#1

Hello there,

This is the continuation of the problem I mentionned at Letsencrypt doesn't verify dns-01 and leaves challenge in status: pending state.

My question is which of those 2 implementation are correct ? If None, then which one is correct ?

I firstly implemented the following:

DEBUG: JWS payload:
b''
DEBUG: Sending POST request to https://acme-staging-v02.api.letsencrypt.org/acme/authz/tLAyXS_a2-ph1Gm0I2V-b_hOOsbNZrnyQUfNfhCxf-A:
{
  "signature": "LiSsgSD0QflQ-s2XSV4qIyJy1xJbLln_3LtYERbfZ7wh2d8ICOTWWSEvvhVpHW5uo-Bz-NXXvg3NnJVJbAsvQ6MZiQzW2pj-0p8dKivz-R4SV4NhUSEq_F9lfDAqQxmT8JK1w8-9WlaYrAOLuA1G-RYJSONNvWTAkgSExdNBZJYJNAfUvpkGd3txhy3R8-kLOqfhg9QF5NaChsnpKG0wfi44bSSsNLSgdU-WYfifdVEbDm7qX0-QiZ55zBoofiLXMO26AryIpKIoU-CmNXwaNGqRhOOWu9yQgigGcCQIzUaPpsAuq2QByMcEA7yyW23B3vSX6MOIhsojXyEbByKhPQ",
  "protected": "eyJhbGciOiAiUlMyNTYiLCAibm9uY2UiOiAiVXNTYVVfUkxNNzVxdHMwYWExNmpub2lySVg1MXh1QnZMTWZVUDctR1h2MCIsICJ1cmwiOiAiaHR0cHM6Ly9hY21lLXN0YWdpbmctdjAyLmFwaS5sZXRzZW5jcnlwdC5vcmcvYWNtZS9hdXRoei90TEF5WFNfYTItcGgxR20wSTJWLWJfaE9Pc2JOWnJueVFVZk5maEN4Zi1BIiwgImtpZCI6ICJodHRwczovL2FjbWUtc3RhZ2luZy12MDIuYXBpLmxldHNlbmNyeXB0Lm9yZy9hY21lL2FjY3QvODY1MjQ4OSJ9",
  "payload": ""
}
DEBUG: https://acme-staging-v02.api.letsencrypt.org:443 "POST /acme/authz/tLAyXS_a2-ph1Gm0I2V-b_hOOsbNZrnyQUfNfhCxf-A HTTP/1.1" 200 929
DEBUG: Received response:
HTTP 200
Server: nginx
Content-Type: application/json
Content-Length: 929
Boulder-Requester: 8652489
Link: <https://acme-staging-v02.api.letsencrypt.org/directory>;rel="index"
Replay-Nonce: 0nsVTLEmI-zowO3uk4yoRZLry8tAd5W1jbmfraizoXc
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800
Expires: Fri, 22 Mar 2019 12:26:34 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Fri, 22 Mar 2019 12:26:34 GMT
Connection: keep-alive

{
  "identifier": {
    "type": "dns",
    "value": "test.juris-sb.de"
  },
  "status": "pending",
  "expires": "2019-03-29T09:12:09Z",
  "challenges": [
    {
      "type": "http-01",
      "status": "pending",
      "url": "https://acme-staging-v02.api.letsencrypt.org/acme/challenge/tLAyXS_a2-ph1Gm0I2V-b_hOOsbNZrnyQUfNfhCxf-A/278379717",
      "token": "G6h6aagnc_J4fKWZlSRRwQDFJOW529gtyugodWBxMcc"
    },
    {
      "type": "dns-01",
      "status": "pending",
      "url": "https://acme-staging-v02.api.letsencrypt.org/acme/challenge/tLAyXS_a2-ph1Gm0I2V-b_hOOsbNZrnyQUfNfhCxf-A/278379718",
      "token": "UJICsxiz22UlY8z6KXPR5Xph61ApwN36A10JaF0nVuE"
    },
    {
      "type": "tls-alpn-01",
      "status": "pending",
      "url": "https://acme-staging-v02.api.letsencrypt.org/acme/challenge/tLAyXS_a2-ph1Gm0I2V-b_hOOsbNZrnyQUfNfhCxf-A/278379719",
      "token": "vNN5b7yxtEkiNoNg6rVgTiniUJsAckB6lOQa75wGmeE"
    }
  ]
}
DEBUG: Storing nonce: 0nsVTLEmI-zowO3uk4yoRZLry8tAd5W1jbmfraizoXc

With the DNS entry:

$ dig +short -t TXT _acme-challenge.test.juris-sb.de
"UJICsxiz22UlY8z6KXPR5Xph61ApwN36A10JaF0nVuE"

Then I have been told by @rmbolger at Letsencrypt doesn't verify dns-01 and leaves challenge in status: pending state

The token value supplied in the dns-01 authz is not the only thing that needs to be in the TXT record. You need a “key authorization” value which is a combination of that token and the ACME account’s thumbprint as described in section 8.1 of the ACME spec.

I may have misunderstood something but then, I implemented:

DEBUG: Storing nonce: oLqYdonM4ARA0ib3ZiZs1B11KKW9IweXDbMVSnYzPPg
DEBUG: JWS payload:
b''
DEBUG: Sending POST request to https://acme-staging-v02.api.letsencrypt.org/acme/authz/tLAyXS_a2-ph1Gm0I2V-b_hOOsbNZrnyQUfNfhCxf-A:
{
  "protected": "eyJub25jZSI6ICJvTHFZZG9uTTRBUkEwaWIzWmlaczFCMTFLS1c5SXdlWERiTVZTbll6UFBnIiwgInVybCI6ICJodHRwczovL2FjbWUtc3RhZ2luZy12MDIuYXBpLmxldHNlbmNyeXB0Lm9yZy9hY21lL2F1dGh6L3RMQXlYU19hMi1waDFHbTBJMlYtYl9oT09zYk5acm55UVVmTmZoQ3hmLUEiLCAiYWxnIjogIlJTMjU2IiwgImtpZCI6ICJodHRwczovL2FjbWUtc3RhZ2luZy12MDIuYXBpLmxldHNlbmNyeXB0Lm9yZy9hY21lL2FjY3QvODY1MjQ4OSJ9",
  "signature": "MhVjwKn1btMLeamwBnvIbZhNCZrfNE-rBnn7EdII8jVgY4nsl7WPNeoxiKmy2Cq01MZhIeRqitZnmOJMYeKKiBWCpy88Kb7B_KwgNt04i6_0yLO_2Z-dMmjPg-BWBFO-SM9HKka2Q-dZVAvS5t0n_-rvr1yiEqe1O__6SJocz9hztU0fumxgR7ba_xpXIbpAlcjW6gjmspbvX8UmkHAClt9ocN-X2DKIRyfe3-ndeSBIg4wwQObi-B9AVgrg_kfURdKI3Ue1pS8GbNoXiehzu1JRMqu3WxV0h3PKp1di8tQUvJNAH2ZK_VDnFQwg6tumpMX3gQSxego_hrwgEkc0bQ",
  "payload": ""
}
DEBUG: https://acme-staging-v02.api.letsencrypt.org:443 "POST /acme/authz/tLAyXS_a2-ph1Gm0I2V-b_hOOsbNZrnyQUfNfhCxf-A HTTP/1.1" 200 929
DEBUG: Received response:
HTTP 200
Server: nginx
Content-Type: application/json
Content-Length: 929
Boulder-Requester: 8652489
Link: <https://acme-staging-v02.api.letsencrypt.org/directory>;rel="index"
Replay-Nonce: LDMytKc-W_VSYYRvcdkJut2llCGu_OlLQcRX4xZ9pUc
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800
Expires: Mon, 25 Mar 2019 08:59:09 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Mon, 25 Mar 2019 08:59:09 GMT
Connection: keep-alive

{
  "identifier": {
    "type": "dns",
    "value": "test.juris-sb.de"
  },
  "status": "pending",
  "expires": "2019-03-29T09:12:09Z",
  "challenges": [
    {
      "type": "http-01",
      "status": "pending",
      "url": "https://acme-staging-v02.api.letsencrypt.org/acme/challenge/tLAyXS_a2-ph1Gm0I2V-b_hOOsbNZrnyQUfNfhCxf-A/278379717",
      "token": "G6h6aagnc_J4fKWZlSRRwQDFJOW529gtyugodWBxMcc"
    },
    {
      "type": "dns-01",
      "status": "pending",
      "url": "https://acme-staging-v02.api.letsencrypt.org/acme/challenge/tLAyXS_a2-ph1Gm0I2V-b_hOOsbNZrnyQUfNfhCxf-A/278379718",
      "token": "UJICsxiz22UlY8z6KXPR5Xph61ApwN36A10JaF0nVuE"
    },
    {
      "type": "tls-alpn-01",
      "status": "pending",
      "url": "https://acme-staging-v02.api.letsencrypt.org/acme/challenge/tLAyXS_a2-ph1Gm0I2V-b_hOOsbNZrnyQUfNfhCxf-A/278379719",
      "token": "vNN5b7yxtEkiNoNg6rVgTiniUJsAckB6lOQa75wGmeE"
    }
  ]
}
DEBUG: Storing nonce: LDMytKc-W_VSYYRvcdkJut2llCGu_OlLQcRX4xZ9pUc

With the record:

$ dig -t TXT +short _acme-challenge.test.juris-sb.de
"UJICsxiz22UlY8z6KXPR5Xph61ApwN36A10JaF0nVuE.Y-ETMP1lasSBSfpx-W8r15Vj4H1psxxfij91tOWeQkw"

Now which one is correct ? What am I missing ?

Side Notes:

  • I use our own client designed for our needed based on https://github.com/certbot/certbot/tree/master/acme/ (python-acme) v0.32.0
  • I used challenges.KeyAuthorizationChallengeResponse() to verify UJICsxiz22UlY8z6KXPR5Xph61ApwN36A10JaF0nVuE.Y-ETMP1lasSBSfpx-W8r15Vj4H1psxxfij91tOWeQkw (last record)
  • UJICsxiz22UlY8z6KXPR5Xph61ApwN36A10JaF0nVuE.Y-ETMP1lasSBSfpx-W8r15Vj4H1psxxfij91tOWeQkw was generated with key_authorization()

Thanks the help.


Letsencrypt doesn't verify dns-01 and leaves challenge in status: pending state
#2

Hello @jsha and thanks for your awesome work!

May I ask if this is still actual ?

I also implemented python-acme for our infrastructure and I get the following when running ClientV2().poll_and_finalize() on staging.

Here is the last logs for about 3-5 minutes:

DEBUG: JWS payload:
b''
DEBUG: Sending POST request to https://acme-staging-v02.api.letsencrypt.org/acme/authz/tLAyXS_a2-ph1Gm0I2V-b_hOOsbNZrnyQUfNfhCxf-A:
{
  "signature": "LiSsgSD0QflQ-s2XSV4qIyJy1xJbLln_3LtYERbfZ7wh2d8ICOTWWSEvvhVpHW5uo-Bz-NXXvg3NnJVJbAsvQ6MZiQzW2pj-0p8dKivz-R4SV4NhUSEq_F9lfDAqQxmT8JK1w8-9WlaYrAOLuA1G-RYJSONNvWTAkgSExdNBZJYJNAfUvpkGd3txhy3R8-kLOqfhg9QF5NaChsnpKG0wfi44bSSsNLSgdU-WYfifdVEbDm7qX0-QiZ55zBoofiLXMO26AryIpKIoU-CmNXwaNGqRhOOWu9yQgigGcCQIzUaPpsAuq2QByMcEA7yyW23B3vSX6MOIhsojXyEbByKhPQ",
  "protected": "eyJhbGciOiAiUlMyNTYiLCAibm9uY2UiOiAiVXNTYVVfUkxNNzVxdHMwYWExNmpub2lySVg1MXh1QnZMTWZVUDctR1h2MCIsICJ1cmwiOiAiaHR0cHM6Ly9hY21lLXN0YWdpbmctdjAyLmFwaS5sZXRzZW5jcnlwdC5vcmcvYWNtZS9hdXRoei90TEF5WFNfYTItcGgxR20wSTJWLWJfaE9Pc2JOWnJueVFVZk5maEN4Zi1BIiwgImtpZCI6ICJodHRwczovL2FjbWUtc3RhZ2luZy12MDIuYXBpLmxldHNlbmNyeXB0Lm9yZy9hY21lL2FjY3QvODY1MjQ4OSJ9",
  "payload": ""
}
DEBUG: https://acme-staging-v02.api.letsencrypt.org:443 "POST /acme/authz/tLAyXS_a2-ph1Gm0I2V-b_hOOsbNZrnyQUfNfhCxf-A HTTP/1.1" 200 929
DEBUG: Received response:
HTTP 200
Server: nginx
Content-Type: application/json
Content-Length: 929
Boulder-Requester: 8652489
Link: <https://acme-staging-v02.api.letsencrypt.org/directory>;rel="index"
Replay-Nonce: 0nsVTLEmI-zowO3uk4yoRZLry8tAd5W1jbmfraizoXc
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800
Expires: Fri, 22 Mar 2019 12:26:34 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Fri, 22 Mar 2019 12:26:34 GMT
Connection: keep-alive

{
  "identifier": {
    "type": "dns",
    "value": "test.juris-sb.de"
  },
  "status": "pending",
  "expires": "2019-03-29T09:12:09Z",
  "challenges": [
    {
      "type": "http-01",
      "status": "pending",
      "url": "https://acme-staging-v02.api.letsencrypt.org/acme/challenge/tLAyXS_a2-ph1Gm0I2V-b_hOOsbNZrnyQUfNfhCxf-A/278379717",
      "token": "G6h6aagnc_J4fKWZlSRRwQDFJOW529gtyugodWBxMcc"
    },
    {
      "type": "dns-01",
      "status": "pending",
      "url": "https://acme-staging-v02.api.letsencrypt.org/acme/challenge/tLAyXS_a2-ph1Gm0I2V-b_hOOsbNZrnyQUfNfhCxf-A/278379718",
      "token": "UJICsxiz22UlY8z6KXPR5Xph61ApwN36A10JaF0nVuE"
    },
    {
      "type": "tls-alpn-01",
      "status": "pending",
      "url": "https://acme-staging-v02.api.letsencrypt.org/acme/challenge/tLAyXS_a2-ph1Gm0I2V-b_hOOsbNZrnyQUfNfhCxf-A/278379719",
      "token": "vNN5b7yxtEkiNoNg6rVgTiniUJsAckB6lOQa75wGmeE"
    }
  ]
}
DEBUG: Storing nonce: 0nsVTLEmI-zowO3uk4yoRZLry8tAd5W1jbmfraizoXc

I was (and I’m still) searching for a bug somewhere before getting here.

If it can help save some time while reading the logs, here is the dig output from outside our infrastructure/network.

$ dig +short -t TXT _acme-challenge.test.juris-sb.de
"UJICsxiz22UlY8z6KXPR5Xph61ApwN36A10JaF0nVuE"

Thanks again for everything!


Letsencrypt doesn't verify dns-01 and leaves challenge in status: pending state
#3

The token value supplied in the dns-01 authz is not the only thing that needs to be in the TXT record. You need a “key authorization” value which is a combination of that token and the ACME account’s thumbprint as described in section 8.1 of the ACME spec.


#4

Thanks for the info @rmbolger! Will keep you updated on Monday…

Did someone wrote all of those little rules nobody though of somewhere? Like a “How to use ACME for Dummies” ?

I know the RFC are there for that but sometime it’s just too complicated when everything is simple :man_facepalming::man_shrugging:

Thanks again.


#5

Hi @rmbolger, it did not changed anything…

My latest log (about 3-5 minutes ago):

DEBUG: Storing nonce: oLqYdonM4ARA0ib3ZiZs1B11KKW9IweXDbMVSnYzPPg
DEBUG: JWS payload:
b''
DEBUG: Sending POST request to https://acme-staging-v02.api.letsencrypt.org/acme/authz/tLAyXS_a2-ph1Gm0I2V-b_hOOsbNZrnyQUfNfhCxf-A:
{
  "protected": "eyJub25jZSI6ICJvTHFZZG9uTTRBUkEwaWIzWmlaczFCMTFLS1c5SXdlWERiTVZTbll6UFBnIiwgInVybCI6ICJodHRwczovL2FjbWUtc3RhZ2luZy12MDIuYXBpLmxldHNlbmNyeXB0Lm9yZy9hY21lL2F1dGh6L3RMQXlYU19hMi1waDFHbTBJMlYtYl9oT09zYk5acm55UVVmTmZoQ3hmLUEiLCAiYWxnIjogIlJTMjU2IiwgImtpZCI6ICJodHRwczovL2FjbWUtc3RhZ2luZy12MDIuYXBpLmxldHNlbmNyeXB0Lm9yZy9hY21lL2FjY3QvODY1MjQ4OSJ9",
  "signature": "MhVjwKn1btMLeamwBnvIbZhNCZrfNE-rBnn7EdII8jVgY4nsl7WPNeoxiKmy2Cq01MZhIeRqitZnmOJMYeKKiBWCpy88Kb7B_KwgNt04i6_0yLO_2Z-dMmjPg-BWBFO-SM9HKka2Q-dZVAvS5t0n_-rvr1yiEqe1O__6SJocz9hztU0fumxgR7ba_xpXIbpAlcjW6gjmspbvX8UmkHAClt9ocN-X2DKIRyfe3-ndeSBIg4wwQObi-B9AVgrg_kfURdKI3Ue1pS8GbNoXiehzu1JRMqu3WxV0h3PKp1di8tQUvJNAH2ZK_VDnFQwg6tumpMX3gQSxego_hrwgEkc0bQ",
  "payload": ""
}
DEBUG: https://acme-staging-v02.api.letsencrypt.org:443 "POST /acme/authz/tLAyXS_a2-ph1Gm0I2V-b_hOOsbNZrnyQUfNfhCxf-A HTTP/1.1" 200 929
DEBUG: Received response:
HTTP 200
Server: nginx
Content-Type: application/json
Content-Length: 929
Boulder-Requester: 8652489
Link: <https://acme-staging-v02.api.letsencrypt.org/directory>;rel="index"
Replay-Nonce: LDMytKc-W_VSYYRvcdkJut2llCGu_OlLQcRX4xZ9pUc
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800
Expires: Mon, 25 Mar 2019 08:59:09 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Mon, 25 Mar 2019 08:59:09 GMT
Connection: keep-alive

{
  "identifier": {
    "type": "dns",
    "value": "test.juris-sb.de"
  },
  "status": "pending",
  "expires": "2019-03-29T09:12:09Z",
  "challenges": [
    {
      "type": "http-01",
      "status": "pending",
      "url": "https://acme-staging-v02.api.letsencrypt.org/acme/challenge/tLAyXS_a2-ph1Gm0I2V-b_hOOsbNZrnyQUfNfhCxf-A/278379717",
      "token": "G6h6aagnc_J4fKWZlSRRwQDFJOW529gtyugodWBxMcc"
    },
    {
      "type": "dns-01",
      "status": "pending",
      "url": "https://acme-staging-v02.api.letsencrypt.org/acme/challenge/tLAyXS_a2-ph1Gm0I2V-b_hOOsbNZrnyQUfNfhCxf-A/278379718",
      "token": "UJICsxiz22UlY8z6KXPR5Xph61ApwN36A10JaF0nVuE"
    },
    {
      "type": "tls-alpn-01",
      "status": "pending",
      "url": "https://acme-staging-v02.api.letsencrypt.org/acme/challenge/tLAyXS_a2-ph1Gm0I2V-b_hOOsbNZrnyQUfNfhCxf-A/278379719",
      "token": "vNN5b7yxtEkiNoNg6rVgTiniUJsAckB6lOQa75wGmeE"
    }
  ]
}
DEBUG: Storing nonce: LDMytKc-W_VSYYRvcdkJut2llCGu_OlLQcRX4xZ9pUc

And my last DIG:

$ dig -t TXT +short _acme-challenge.test.juris-sb.de
"UJICsxiz22UlY8z6KXPR5Xph61ApwN36A10JaF0nVuE.Y-ETMP1lasSBSfpx-W8r15Vj4H1psxxfij91tOWeQkw"

#6

Hi @funilrys

that’s wrong. Checked your domain ( https://check-your-website.server-daten.de/?q=test.juris-sb.de ), the result:

Domainname TXT Entry Status ∑ Queries ∑ Timeout
test.juris-sb.de ok 1 0
_acme-challenge.test.juris-sb.de UJICsxiz22UlY8z6KXPR5Xph61ApwN36A10JaF0nVuE.Y-ETMP1lasSBSfpx-W8r15Vj4H1psxxfij91tOWeQkw missing entry or wrong length 1 0

See my own domain:

Domainname TXT Entry Status ∑ Queries ∑ Timeout
server-daten.de v=spf1 a mx ?all ok 1 0
www.server-daten.de ok 1 0
_acme-challenge.server-daten.de LJWtQAv_5lFCjGdssk4B2pyYoan8xRT7P8t1Bhvy4VU looks good 1 0

These entries have always 43 characters.

Key authorization -> SHA256 Hash -> Base64 -> Base64Url

Not the raw key authorization.


#7

@funilrys, your problem is unrelated to this thread, please don’t hijack it. Thanks


#8

@JuergenAuer, I’m lost …

What is correct then ?

My previous implementation Letsencrypt doesn't verify dns-01 and leaves challenge in status: pending state or my current implementation Letsencrypt doesn't verify dns-01 and leaves challenge in status: pending state ?

Thanks for the help.


#9

Sorry, I created a new one DNS01: How is the challenge supposed to be formatted?.

Sorry for the noise… :frowning_face:


#10

Hi @funilrys

I’ve splitted and merged the posts.

If you want to develop your own client, the specification

RFC 8555

Automatic Certificate Management Environment (ACME)

https://tools.ietf.org/html/rfc8555

is your source.


#11

@JuergenAuer,

I started from

+---------------------+--------------------------------+--------------+
| Action              | Request                        | Response     |
+---------------------+--------------------------------+--------------+
| 1 Get directory     | GET  directory                 | 200          |
|                     |                                |              |
| 2 Get nonce         | HEAD newNonce                  | 200          |
|                     |                                |              |
| 3 Create account    | POST newAccount                | 201 ->       |
|                     |                                | account      |
|                     |                                |              |
| 4 Submit order      | POST newOrder                  | 201 -> order |
|                     |                                |              |
| 5 Fetch challenges  | POST-as-GET order's            | 200          |
|                     | authorization urls             |              |
|                     |                                |              |
| 6 Respond to        | POST authorization challenge   | 200          |
| challenges          | urls                           |              |
|                     |                                |              |
| 7 Poll for status   | POST-as-GET order              | 200          |
|                     |                                |              |
| 8 Finalize order    | POST order's finalize url      | 200          |
|                     |                                |              |
| 9 Poll for status   | POST-as-GET order              | 200          |
|                     |                                |              |
| 10 Download         | POST-as-GET order's            | 200          |
| certificate         | certificate url                |              |
+---------------------+--------------------------------+--------------+
  1. ==> python-acme does it for me.
  2. ==> account key saved/loaded into/from a safe place
  3. ==> I got UJICsxiz22UlY8z6KXPR5Xph61ApwN36A10JaF0nVuE
  4. ==> I’m stuck here.

What I did is:

  1. Read Section 8.1

  2. Update the TXT record with the Key Authorizations which was generated with the help of challenge_body.challenge.key_authorization(account_key)

  3. I also checked it with challenges.KeyAuthorizationChallengeResponse ans it is correct.

What do you mean with the following ?

Key authorization -> SHA256 Hash -> Base64 -> Base64Url


#12

Please read the specification:

https://tools.ietf.org/html/rfc8555#section-8.4

There you will find the same.


#13

Oh ! Thanks I missed that part. It is now working.

Vielen Dank!
Einen schönen Tag noch!