Agreeing to subscriber agreement

Issuance Tech
1

Hi there,

I’m trying to get an in-depth understanding of how ACME and LetsEncrypt works and therefore writing my own ACME client that I’m trying to get running with LetsEncrypt.

The ACME RFC is rather horrible, in my opinion. All examples just show half of the actual work and no actual message is included in the examples, which makes implementation from scratch extremely frustrating and annoying. For example, in https://letsencrypt.github.io/acme-spec/ in 5.2 “Registration Objects”, there’s a description and an example JSON. The description mentions a “required” key field. That key field is MISSING in the exmaple, however. The example specifies a “resource” field, but that “resource” field is not included in the description. It’s just inconsistent. Further down, in the signed examples, there’s JSON examples provided and then it says “Signed as JWS”. That is just plain HORRIBLE. I want to see what I actually need to send to the server and can decode the JWS by myself. It would be awesome if this could be fixed and actual examples could be provided which include everything. For example, “consider this private/public rsa keypair: [blob] with with this message [json] is signed to yield the following signed message which is transmitted to the server [msg]”.

Anyways. Finally by digging through tons of debug logs and grabbing some JSONs from there, I was able to successfully execute a new-reg. However, now I’m trying to get a new-authz from the staging LetsEncrypt server and that is refused:

{“type”:“urn:acme:error:unauthorized”,“detail”:“Must agree to subscriber agreement before any further actions”,“status”:403}

True, I never agreed to anything and in my registration also didn’t include any “agreement” field (wouldn’t have known what to write there in the first place). Now with that account key I cannot reregister. How is this agreement supposed to happen and where can I find the appropriate documentation?

Thanks in advance,
Johannes

2

Okok, I got it, I think.

First new-reg gives “link” terms-of-service header field, which points to the TOS URI. Then I can submit a second post to the URL that is given as Location in the 409 new-reg response in which I re-post the subscription information (with the correct URL). It’s in the docs, I just needed to RTFM more closely.

I still think the RFC examples are horrible, though :smiley:

3

What language are you developing your client in ? it may be worth looking at a similar working example in the alternate clients

4

Note that the ACME spec moved to https://ietf-wg-acme.github.io/acme/. This doesn’t address the inconsistency you mentioned, but other things might have changed.

To add to what @serverco said, I would specifically look at the “Libraries” section in that thread, which contains low-level libraries for many languages that could be useful if you’re writing your own integration.

5

Hi Johannes,

I ran into the same problem. Your post saved my night.

When trying to submit a new-authz, I got this response:

RESPONSE STATUS: 403
HEADERS: [('Server', 'nginx'), ('Content-Type', 'application/problem+json'), ('Content-Length', '137'), ('Boulder-Request-Id', 'nUDq2tctipap93zUx8nFvw1yZP7Ikt6Oi2wMt1s2xsg'), ('Boulder-Requester', '******'), ('Replay-Nonce', 'iSUqMyN_SGDmzTwQ7mUSBaDMdsq2sfwbZ73-mUl95v0'), ('Expires', 'Thu, 22 Dec 2016 11:21:19 GMT'), ('Cache-Control', 'max-age=0, no-cache, no-store'), ('Pragma', 'no-cache'), ('Date', 'Thu, 22 Dec 2016 11:21:19 GMT'), ('Connection', 'close')]
BODY: {'type': 'urn:acme:error:unauthorized', 'detail': 'Must agree to subscriber agreement before any further actions', 'status': 403}

I scrolled up to see the response of the successful new-reg:

RESPONSE STATUS: 201
HEADERS: [('Server', 'nginx'), ('Content-Type', 'application/json'), ('Content-Length', '583'), ('Boulder-Request-Id', 'TjlSrRKr6KEO2sXM4MD31j7oNWm_dfqcMp7vjeUY_Pk'), ('Boulder-Requester', ''), ('Link', 'https://acme-staging.api.letsencrypt.org/acme/new-authz;rel="next"'), ('Link', 'https://letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf;rel="terms-of-service"'), ('Location', 'https://acme-staging.api.letsencrypt.org/acme/reg/'), ('Replay-Nonce', 'knDvHPr8FrvLCyPJ9Vd5oouApFmlFbOkwOuD_xehH3Y'), ('X-Frame-Options', 'DENY'), ('Strict-Transport-Security', 'max-age=604800'), ('Expires', 'Thu, 22 Dec 2016 09:12:15 GMT'), ('Cache-Control', 'max-age=0, no-cache, no-store'), ('Pragma', 'no-cache'), ('Date', 'Thu, 22 Dec 2016 09:12:15 GMT'), ('Connection', 'keep-alive')]
SAVING NONCE: knDvHPr8FrvLCyPJ9Vd5oouApFmlFbOkwOuD_xehH3Y
{'key': {'n': 'p4XHQxSqTD_mkFWfdFZOr05fj8Z2JuYqgDtb-RAiuNIyalyzVmE-maxISF9jeugYrnJdeF788XgvDgjRjJdBz0ELUHNzBjGN6tXNdpue5Ck_lynv32RmjdYjOhH_bxCYV_hLugwvgo5nDkWEvj4l1bxPKU8VIQSH63ZHmQkHRsYAGD49PucZeCWbe76zOJ6wo47GkfGSXhh6mUeyPh4PPa8o2LXBnPGNNAg5F334wxuQfaoBwnA1Zj_4_BJR-ZdOYCkIKcgxbuDFbW5nKkVeA_X75sCVb9F-qAFWEzgZ-keTvJkafb8WTOZCrjxdU27q4qSWxXWfkFd6TBffOI2s6Q', 'kty': 'RSA', 'e': 'AQAB'}, 'Status': 'valid', 'initialIp': '...', 'createdAt': '2016-12-22T09:12:15.668933577Z', 'id': **, 'contact': ['@gmail.com']}

I searched for subscriber agreement on all versions of the ACME protocol but couldn't figure out how I was supposed to explicitly agree to https://letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf. So I went to the source code of letsencrypt/acme/client.py, and found this function:

def agree_to_tos(self, regr):
    """Agree to the terms-of-service.

    Agree to the terms-of-service in a Registration Resource.

    :param regr: Registration Resource.
    :type regr: `.RegistrationResource`

    :returns: Updated Registration Resource.
    :rtype: `.RegistrationResource`

    """
    return self.update_registration(
           regr.update(body=regr.body.update(agreement=regr.terms_of_service)))

With a bit of luck, I created a variant of my new-reg function, by adding this field into the payload:

'agreement': 'https://letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf'

, setting

'resource': 'reg',

and posted the request to /acme/reg/******

Then submitted a new-authz, it worked!!!!!!!!!

RESPONSE STATUS: 201
HEADERS: [('Server', 'nginx'), ('Content-Type', 'application/json'), ('Content-Length', '1008'), ('Boulder-Request-Id', 'AT3XDuUw80-o4u6Z6Ia6mt_gGtNBk1yA_SCuyx5OTGM'), ('Boulder-Requester', ''), ('Link', 'https://acme-staging.api.letsencrypt.org/acme/new-cert;rel="next"'), ('Location', 'https://acme-staging.api.letsencrypt.org/acme/authz/'), ('Replay-Nonce', '58aiWi1WiVgScay8DtDqogFgkQIkflrgR00sGO3V3Jc'), ('X-Frame-Options', 'DENY'), ('Strict-Transport-Security', 'max-age=604800'), ('Expires', 'Thu, 22 Dec 2016 11:51:42 GMT'), ('Cache-Control', 'max-age=0, no-cache, no-store'), ('Pragma', 'no-cache'), ('Date', 'Thu, 22 Dec 2016 11:51:42 GMT'), ('Connection', 'keep-alive')]
BODY: {'expires': '2016-12-29T11:51:42.222343123Z', 'combinations': [[0], [1], [2]], 'challenges': [{'uri': 'https://acme-staging.api.letsencrypt.org/acme/challenge/******************************/', 'token': '', 'type': 'http-01', 'status': 'pending'}, {'uri': 'https://acme-staging.api.letsencrypt.org/acme/challenge/****************/', 'token': '***', 'type': 'tls-sni-01', 'status': 'pending'}, {'uri': 'https://acme-staging.api.letsencrypt.org/acme/challenge/*************/', 'token': '-hB0', 'type': 'dns-01', 'status': 'pending'}], 'status': 'pending', 'identifier': {'type': 'dns', 'value': '.co.nz'}}

I'm getting one step closer to getting my certificate without using certbot!!!

The project LetsEncrypt is great, as it lowers the cost of obtaining an SSL Certificate to zero dollars. However, I believe there's room for the documentations to improve:
a) SEO of the latest version of ACME protocol. It should rank higher than the first edition on Google Search result;
b) Detailed and up-to-date documentation on the difference between the implementation of Boulder and the ACME protocol;

Cheers,
G

6
7

Hi Andy,

Thank you for the message.

Yes I went through this page last night. It's just... incomplete...

It says in the version 04 of the protocol (draft-ietf-acme-acme-04) that:

5.2. Request Authentication

All ACME requests with a non-empty body MUST encapsulate the body in
a JWS object, signed using the account key pair. The server MUST
verify the JWS before processing the request. (For readability,
however, the examples below omit this encapsulation.) Encapsulating
request bodies in JWS provides a simple authentication of requests by
way of key continuity.

JWS objects sent in ACME requests MUST meet the following additional
criteria:

o The JWS MUST be encoded using UTF-8

o The JWS MUST NOT have the value "none" in its "alg" field

o The JWS Protected Header MUST include the following fields:

  *  "alg"

  *  "jwk" (only for requests to new-reg and revoke-cert resources)

  *  "kid" (for all other requests).

  *  "nonce" (defined below)

  *  "url" (defined below)

The "jwk" and "kid" fields are mutually exclusive. Servers MUST
reject requests that contain both.

Barnes, et al. Expires May 4, 2017 [Page 9]

Internet-Draft ACME October 2016

For new-reg requests, and for revoke-cert requests authenticated by
certificate key, there MUST be a "jwk" field.

For all other requests, there MUST be a "kid" field. This field must
contain the account URI received by POSTing to the new-reg resource.

The divergences document didn't mention kid or jwk. In practice, Boulder complains about the absence of jwk, where according to the protocol kid should be used.

The paragraph below may imply this divergence:

Section 6.3.2.

Boulder implements draft-04 style key roll-over with a few divergences. Since Boulder doesn't currently use the registration URL to identify users we do not check for that field in the JWS protected headers but do check for it in the inner payload. Boulder also requires the outer JWS payload contains the "resource": "key-change" field.

Cheers,
G

8

I must admit I’d missed all the “kid” stuff that came in v4. I wrote the bulk of my client before v4 came out. perhaps @cpu can comment on that.

For general help in developing a client you may be better posting in the “client dev” forum. What language are you developing a client in ?

9

Hi @serverco, @guti,

I think you’ve identified a place where the divergences document isn’t capturing a Boulder behaviour that doesn’t match the latest draft. I’ll look at fixing that. In the future filing an issue on the Boulder repo about the divergence is the best path for this sort of issue.

Thanks!

2 Likes
10

@serverco I wrote in python. If you ask why I write another python implementation when there’s already an official one, then the answer will be:

Certbot aims at providing a command line console which still requires human interventions to acquire a certificate, I.e. pressing ENTER after copying and pasting the challenge string (authentication key) into the corresponding place where it will be served by the webserver. Unfortunately I’m only using nginx for load balancing but not request routing, and I’d like my server to respond to my get the certificate done command fully automatically. I tried to call Certbot through subprocess but couldn’t deal with waiting for stdout/reading/writing ENTER back nicely enough. Furthermore I have a personal preference that my pip dependencies must be countable, which ruled out introducing packages like expect…

It would be nice if someday the Certbot can be programmatically integrated into the user’s codebase. Ripping out Acme package from Certbot is not a good idea because as far as I know it’s currently a submodule of Certbot, which is subject to changes without sticking to any specific standard API.

While, I’m just saying. Not sure if there are many people who make use of Boulder this way.

Merry Christmas.

Cheers,
G

11

Hi @cpu,

Thank you for the reply.

I’ll post stuff at the designated area if I discover another mismatch.

You guys are doing a great job. I’d been comparing the pricing of the CAs for a while until I accidentally discovered the existence of LetsEncrypt when browsing the docs of BeautifulSoup4. I like the idea of automated certificate management.

Thanks again and have a great holiday.

Cheers,
G

1 Like
12

I thought, but may be wrong, that you could use the -n option on certbot for non-interactive mode.

I understand fully writing your own code, I wrote my own bash one ( GetSSL ) because certbot (letsencrypt as it was called then) wouldn’t run on many of the servers (because I didn’t want all the dependency packages). I also wanted something that would automatically update remote servers without needing to be installed on the server. The Let’s Encrypt folk are very supportive of alternate clients :slight_smile:

There are 8 other python clients listed on https://letsencrypt.org/docs/client-options/ The only reason I was asking was in case one of them might provide a useful base for you ( I didn’t know your requirements well enough, or those clients well enough to know if one would though ) or even provide some functions you could utilise.

Merry Christmas and have a great New Year :slight_smile:

Andy

1 Like