Guidance Request: Any experience using Certes .Net client from GitHub?

I apologize if this isn't the right place to ask; my understanding is that these forums are for all ACME clients and Certes is listed as one of the options.

I'm using fszlin/certes: A client implementation for the Automated Certificate Management Environment (ACME) protocol (github.com) to write a custom utility designed to update Sophos Firewall certs for our managed clients using DNS validation with validation forwarding.

I've got the DNS updating part working fine, but have run into trouble with the certificate ordering process. There are a couple of different examples of the order process in that GitHub repo, but following order example is giving me an error of "Can not finalize order with status 'Invalid'" around this portion of my code:

Dim order = Await acmeContext.NewOrder(New String() {$"vpn.{domainName}"})
Dim authz = Await order.Authorizations()

There is an additional example here (certes docs - orders) that differs enough from the other one that I'm just not clear on what the actual process is supposed to be. I am hesitant to continue experimenting with attempting to figure it out as I don't want to create a mess on the Let's Encrypt side with a bunch of invalid orders.

Does anyone have any experience with this or can offer guidance on the best way to proceed? Thanks in advance.

1 Like

Pretty sure @webprofusion uses Certes in his Certify The Web client. Though I vaguely recall him saying at one point he switched to a forked version for reasons I can't recall.

11 Likes

I saw that ChrisC has a commit to that repo so I believe you are correct. It will be great if he can reach out here when he gets a chance. Thanks so much for the mention!

2 Likes

Hi,

Firstly it's worth noting that you can achieve what you're trying to do without writing your own client, you could use @rmbolger's Home - Posh-ACME or Certify The Web to achieve much the same thing (see the Tasks feature). However we obviously know it's sometimes nice to write stuff yourself.

Certes is a little bit neglected by it's owner, only receiving very sporadic updates but you could argue it just doesn't currently need them. We do have a fork (GitHub - webprofusion/certes: A client implementation for the Automated Certificate Management Environment (ACME) protocol) also on NuGet, people can use that if they want but I don't intend to provide support for it and most won't need it. The original was a very well put together project, our version is intentionally playing a little more fast and loose and really just exists to immediately solve challenges specific to Certify The Web.

Regarding your specific example, an "invalid' status on your order means your validation failed: RFC 8555 - Automatic Certificate Management Environment (ACME) - with DNS the most common reason is your _acme-challenge TXT record is either wrong or hasn't synced to all your domains nameservers, so it's common to have to wait a minute before proceeding with validation (asking LE to check your answers). Some clients perform a check on the authoritative nameservers before proceeding, some just wait a given amount of time.

You can check the status of an Authorization using certes by fetching the authorization resource again. You can see an example of just repeatedly checking the auth status in the Certify source code: certify/CertesACMEProvider.cs at development · webprofusion/certify · GitHub

Your auth with then either be valid, invalid or still pending. You can't finalize your order until you have a valid auth for all your domains.

Note also that if you have recently performed a valid auth for a domain then Let's Encrypt caches that and your auth will already be valid (even without performing a DNS update etc), so it's possible to start a new order, be given a set of already valid auth then proceed directly to finalize and download.

10 Likes

Thanks so much for the reply.

I looked at both proposed existing solutions but am not sure they are a good fit for this requirement. In both cases, I would still need my own routines for pushing the cert onto the Sophos firewall. With Posh I supposed it would be technically possible to do the firewall part in PowerShell, but my experience in .Net vastly outweighs my experience in PS so I'm much more confident and comfortable building that part of the solution in a .Net application. My end result needs to run completely unattended, either as a scheduled job or perhaps a Windows service. I'm not sure that Certify the Web could be utilized without any UI interaction. I'm far enough along with my own solution that I think the most expedient resolution is to work out the certificate ordering process.

I didn't think it was the DNS authorization process causing the issue because the client domain entry has been in place for a day or two and the dynamic entry is being made on our own cloudflare domain so the changes are pretty much instant (I also tried again the next day). I assume I'm doing something wrong with the request itself or not following the process properly.

Thanks for the link into your own code. I may be able to work through your class implementation and its usage to figure out the order of steps you are taking to place an order. I'll see what I can infer from your code.

3 Likes

What does the Sophos firewall run?

7 Likes

It is recommended to use Pebble for development (GitHub - letsencrypt/pebble: A miniature version of Boulder, Pebble is a small RFC 8555 ACME test server not suited for a production certificate authority.) and the LetsEncrypt staging endpoint for integration.

Since you're pushing the certificates onto client devices, you may not be as constrained as you think. For example, you could run Certbot to handle everything, and use the success hook to deploy your certificates. You might also find a linux based client that already handles most of your expected workload.

I also strongly suggest using acme-dns for this (GitHub - joohoi/acme-dns: Limited DNS server with RESTful HTTP API to handle ACME DNS challenges easily and securely.), as it works by delegating the acme records onto a secondary DNS system (you don't need use acme-dns to leverage that technique). You only enter a DNS record on the primary/main nameservers once, which delegates the challenge to to acme-dns, and the acme-dns system provides real-time answers to the dns query. While I think acme-dns is the best option for DNS challenges, I think it is the only viable option for large-scale and client deployments, as it avoids many of the most-common problems.

9 Likes

It is some flavor of Linux. There is SSH access but we are highly discouraged from modifying anything in the firewall at the Linux level. They don't provide any documentation and won't support anything you do at that level. To avoid a fight with support later down the road, we want to stick to their Web API (and I should throw quotes around "Web API"... it is a, shall we just call it, unique take on a Web API interface). I have to craft an HTTP request (with a potentially invalid HTTP method specified, still waiting for support to verify that it isn't a typo in the documentation) and send three multipart uploads with the request (the PEM cert file, the KEY private key file, and their own special XML action file).

1 Like

Thanks for the lead on Pebble - that could be very helpful for this troubleshooting step.

I started with Certbot but was finding it difficult to understand how to do anything that wasn't connected to a Web Server - I will have to look at it again more closely; the success hook does sound promising and I could potentially just automate the CLI from my own application that is handling the Sophos part.

I'm already doing the delegated DNS through a Cloudflare domain that was created expressly for this purpose - it has no other records. It was worth the tiny price to not have to maintain a fake DNS server on our own equipment (though I did look at that option before deciding). I do appreciate the advice though.

1 Like

Oh. Wow. I honestly can't say any of that surprises me, although it should!

The relevant bits for you:
Certbot CloudFlare DNS Plugin: Welcome to certbot-dns-cloudflare’s documentation! — certbot-dns-cloudflare 0 documentation
Certbot Hooks: Some of the hooks have docs in multiple places on this page, so make sure to search the page - User Guide — Certbot 1.29.0 documentation

  • --pre-hook and --post-hook will wrap the overall certbot execution
  • --manual-auth-hook and --manual-cleanup-hook will wrap the DNS setup on each domain/challenge
  • --deploy-hook is the successful "we got a certificate!" hook

If you can't run a .NET script to handle the hooks, maybe you can write a .NET script to invoke one of the clients (certbot, posh-acme, etc) and then operate on the certs as needed. This may sound backwards, but there are a lot of weird edge cases in the ACME spec and peculiarities in the evolving service so (having written a client myself) I would try to avoid writing a custom client as much as possible.

Sweet! Setting up the delegated system is a real time/energy saver so you're ahead of the curve!
Since you're doing this centrally, you shouldn't need to worry much about API credentials either.

9 Likes

Awesome! Thank you for the links, this looks very promising! What are the actual calls to the hooks? Is it just basically a "process.start("thepath.ext")" kind of call that relies on the system's registered application for ".ext" (e.g. whatever bash interpreter is installed)? Or is there an internal scripting engine that is picking up and executing a bash script inside of Certbot?

maybe you can write a .NET script to invoke one of the clients (certbot, posh-acme, etc) and then operate on the certs as needed.

This is exactly what I'm thinking. The application I have to write to handle the Sophos part can just execute certbot from the command line and then pickup the output.

3 Likes

Have a look at:
https://eff-certbot.readthedocs.io/en/stable/using.html

7 Likes

So, you invoke Certbot with a defined hook, such as:

certbot renew --manual-auth-hook=/path/to/script

Certbot then invokes the hook and populates the environment with a handful of variables that are relevant to it's function (such as the domain, validation string, token, etc). You can pass in a binary, bat or python file for the hook - it just needs to be an executable file registered with the system for CMD operations (see Windows: Renewal deploy hook opens in notepad.exe instead of being executed · Issue #7713 · certbot/certbot · GitHub). So, Powershell scripts don't work, because they will open the editor; but a bat file that executes the Powershell script works. Passing in a .NET may work natively or need the bat trick.

7 Likes

Perfect!

Certbot uses subprocess.call('my_executable param1', shell=True) to run hook scripts.

That answers the question succinctly. I can totally work with that.

Thank you so much for all of the useful links! I can't imagine how long I would have searched to try to find this info. This community rocks! :smiley:

4 Likes

I'm glad you've got some of the guidance you sought. I'm sorry I didn't think of mentioning the subprocess call myself!

5 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.