ACME Account Registration in Python

I have learned quite a bit about ACME, but still need some help. I have successfully implemented an ACME account registration, and requesting a certificate with EAB and "no-challenge" all using Ansible.

Now, b/c I have a whole slew of Python scripts and tools that are also used in certificate renewal automation, I need to figure out how to try to do this in Python. If it is done in Ansible.. and Ansible is mostly Python... ..and I have scanned this GitHub - ansible-collections/community.crypto: The community.crypto collection for Ansible..

So, I was able to find the section where this is happening, and can follow along, but unfortunately .. there is SO MUCH code supporting Ansible and inclusion of other files, it is really hard to follow and on top of that.. trying to figure out exactly what python modules I need to have in my script.

So looking for some help. Have no reason to re-invent the wheel, but I also do not want a bunch of code around that I would never use. My goals in my Python script are:

  1. Register a private key with EAB and my CA (Sectigo/InCommon) using the "no challenge" challenge.
  2. Use that registration to submit a CSR for a certificate.
  3. Fetch the certificate.
  4. Put the newly generated certificate and private-key in the appropriate location or save locally to the same path as the script.

The documentation for the acme module (Python)[Welcome to acme-python’s documentation! — acme-python 0 documentation] is truly technical, with very few if any examples of using the calls, so I am lost there.

I have reviewed code for numerous other clients on this page: ACME Client Implementations - Let's Encrypt and a bit further down: ACME Client Implementations - Let's Encrypt

In Ansible .. it is written as:

- name: "Register private key with Sectigo EAB credentials, get acct URI."
  community.crypto.acme_account:
    account_key_content: "{{ acct_privatekey }}"
    acme_directory: "{{ acme_url }}"
    state: present
    terms_agreed: true
    acme_version: "{{ acme_version }}"
    contact:
      - "mailto:{{ cert_group_email }}"
    external_account_binding:
      alg: HS256
      key: "{{ acme_hmac_key }}"
      kid: "{{ acme_account_id }}"
  register: acct_key_registration

- name: "Perform a 'no challenge' request w/ account private key"
  community.crypto.acme_certificate:
    account_key_content: "{{ acct_privatekey }}"
    csr_content: "{{ cert_csr }}"
    account_email: "{{ acme_account_email }}"
    acme_version: "{{ acme_version }}"
    account_uri: "{{ acct_key_registration.account_uri }}"
    acme_directory: "{{ acme_url }}"
    remaining_days: "{{ cert_term_length }}"
    dest: "/var/tmp/{{ cert_filename }}"
    terms_agreed: true
    validate_certs: true
    challenge: no challenge
  register: acme_no_challenge
  no_log: false

- name: Resubmit the request with the challenge result
  community.crypto.acme_certificate:
    account_key_content: "{{ acct_privatekey }}"
    csr_content: "{{ cert_csr }}"
    account_email: "{{ acme_account_email }}"
    acme_version: "{{ acme_version }}"
    challenge: no challenge
    data: "{{ acme_no_challenge }}"
    account_uri: "{{ acct_key_registration.account_uri }}"
    acme_directory: "{{ acme_url }}"
    remaining_days: "{{ cert_term_length }}"
    dest: "/var/tmp/{{ cert_filename }}"
    chain_dest: "/var/tmp/{{ chain_filename }}"
    fullchain_dest: "/var/tmp/{{ full_chain_filename }}"
  when: acme_no_challenge is changed
  no_log: false

This works!

So, I already have all the supporting code for gathering certificate info (CN, SANs, org, state, country, etc..), generating a private key (for the account) and a private key for the certificate, generate a signed CSR, and supporting parameter CLI, and other stuff.. now I just need to get the above steps in Python.

So can anyone help me figure where I can find good example code .. for starters registering a private key using EAB and my CA. If not the "acme" module, what other would you recommend? If acme module, what imports do I need?

Thank you!

1 Like

I haven't implemented the EAB flow yet, so I can't comment on that.

What I can share though:

  • The acme module does handle a lot of things elegantly, but beware that it is undergoing some breaking changes that haven't been released yet. I was the one who broke them, and the fix got held up Certbot's need to release a deprecation version first.

If you use acme you will need to pin the version, and be prepared for some small upgrades. The next release of acme will use josepy2.0 which drops the it's own ComparableX509 object that wraps a PyOpenSSl x509 object, in favor of a raw cryptography.X509 object; it also drops PyOpenSSL for almost everything, in support of the cryptography.x509 object

If you don't want to use acme, the simplest way to understand how to implement the ACME protocol in python is with acme-tiny: GitHub - diafygi/acme-tiny: A tiny script to issue and renew TLS certs from Let's Encrypt it's a single short script and is probably the easiest way to understand how to construct and send a payload. There is even an EAB PR for it here:
Implement externalAccountBinding by systemcrash · Pull Request #270 · diafygi/acme-tiny · GitHub

I used acme-tiny as a reference for the python client I maintain, and I recommend it strongly as the best python implementation of the protocol for educational purposes. It is very limited, but the code is short and clear.

4 Likes

I am actually not very familiar with using the acme package, so I can't help much. I handled the PR to remove PyOpenSSL from the josepy library because the Cerbot team wants PyOpenSSL completely gone ASAP, and I wanted to remove it fro my code (which uses josepy) too.
I updated the acme code as a courtesy for the Certbot team, because I was the person most familiar with those changes.

2 Likes

@jvanasco - So, the PR#270 is exactly what is preventing me from using acme-tiny. I am using Sectigo/InCommon CA and they require EAB with "no-challenge". I am going to see if I can pull down acme-tiny and see if I can create a version that has the content from PR#270.

@jvanasco -- I have made the changes to the tiny-acme on my machine. I was about to try it out, and realized that the command still refers to /var/www/challeneges/ acme directory.

❯ python acme_tiny.py --account-key ../account.key --csr ../test.csr --acme-dir /var/www/challenges/ --contact 'mailto:jewettg@austin.utexas.edu' --eabkid 'xxx' --eabhmackey 'xxx' > ./signed_chain.crt
Parsing account key...
Parsing CSR...
Found domains: greg-test-2025-delete.its.utexas.edu
Getting directory...
Directory found!
Registering account...
Building externalAccountBinding...
Registered! Account ID: https://acme-v02.api.letsencrypt.org/acme/acct/2293499366
Creating new order...
Order created!
Verifying greg-test-2025-delete.its.utexas.edu...
Traceback (most recent call last):
  File "/Users/jewettg/GitHub/devops_python/cert_tool/acme-tiny/acme_tiny.py", line 203, in <module>
    main(sys.argv[1:])
  File "/Users/jewettg/GitHub/devops_python/cert_tool/acme-tiny/acme_tiny.py", line 199, in main
    signed_crt = get_crt(args.account_key, args.csr, args.acme_dir, log=LOGGER, CA=args.ca, disable_check=args.disable_check, directory_url=args.directory_url, contact=args.contact, check_port=args.check_port, eabkid=args.eabkid, eabhmackey=args.eabhmackey)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jewettg/GitHub/devops_python/cert_tool/acme-tiny/acme_tiny.py", line 143, in get_crt
    with open(wellknown_path, "w") as wellknown_file:
         ^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/var/www/challenges/8ozgExfxualyN4chNLQ8hd42DfUa5IDRGc7fLVZYZ4U'

So all that worked up to I guess the challenge part.
Any idea how to remove this dependency or make this support "no challenge"?

try running mkdir -p /var/www/challenges in a terminal window before invoking the script.

if things work how i hope, acme-tiny will write the challenge file - but your CA won't bother trying to authenticate it.

from what I saw in that PR earlier, it doesn't have a logic gate to guard against writing that file - so it should just need the directory to exist.

3 Likes

kill acme-tiny/acme_tiny.py at master · diafygi/acme-tiny · GitHub (line 142-147) from acme-tiny

3 Likes

@orangepizza and @jvanasco -- Thank you very much for your help!

I had to create the directory and give write permissions to it before it got past that point. I will have added that to my internal documentation.

I also had not updated line 9-10 to my CA.. doh! Once I did that .. it worked!

@orangepizza - now that I have it working, I am going to pull stuff out that I do not need (the /var/www/challenges part of the code), make it so I can incorporate all of the source into my code, remove command-line parameters, etc..

So it included the whole chain in the response. Is there a way to have it not do that, or do I just need to load it and find the right one and extract it? I am digging through the code now that I know it works with my CA, so I might find it!

THANK YOU AGAIN!!

The CA Chain files are sent by the ACME server, as part of the RFC 8555 - Section 7.4.2

You'll have to decompose the chain and pull out the leaf/end-entity certificate. This is what Certbot and most other acme-clients do.

A quick note on this: acme-tiny is barebones. it does not support alternate chains. Your CA might not have any alternate chains now, LetsEncrypt does not currently offer any (they previously did with the DST cross signs. If you need altnernate chains, you'd need to inspect the response during the "download certificate" phase for "links" in the headers.

2 Likes

@jvanasco - THANK YOU!
When you say "alternate chains", you are referring to extended validation, etc.. right? Other types of certificates? If so, acme-tiny will do just fine for now, and when I come to those cross-roads needing something else, I will revisit.

That is not true. There is an alternate ECDSA chain for X2

3 Likes

Under the RFC, a CA can provide alternate chains up to the a different Trust Anchor for the same certificate. This is accomplished by the CA having their intermediate cross-signed by another root or intermediate.

Doh! That slipped my mind. Thank you for the correction.

3 Likes

Thank you all (@jvanasco and @orangepizza ) for all your help!

I was successfully able to incorporate acme-tiny into my script, heavily modified, removing all calls to "openssl" and reading in files, and relying of pre-written code to generate private-keys, CSRs, and storing that in variables and generating a certificate!

I was really struggling before trying to truly battle those windmills .. trying to tackle more necessary! I appreciate you putting me on the path of acme-tiny and the work done by Daniel Roesler (aka @diafygi) to make acme-tiny.

It is not worthy of PR to his project, since I literally blew up his two-hundred lines to do what I needed to do, and make it more readable in my script.

@diagygi -- I did give you credit in my code and made sure to point out where your code has been included, note that it has been heavily modified.

THANKS AGAIN ALL!!!

4 Likes