Client usage data availability

Does anyone know if LE keeps internal metrics about client use based on the mandatory user agent strings that clients have to send. I know they’ve been able to do things like block badly behaving clients based on user agent in the past.

As a client author, it would be a huge benefit to get some sort of access to the metrics for my own client. Things like being able to see how many users are on what version of the client, what platform they’re running it on, how many certs have been generated over a particular time frame.

To be clear, I’m not looking to see the data for anyone’s client but my own. Publishing all the data publicly, while interesting, might lead to petty popularity conversations. I was thinking more along the lines of some way to register with LE as a client author and be able to get like a monthly email report with some sort of breakdown of usage for my client.

It’s probably not a realistic ask as there are far more important things for the LE folks to spend their time on. But I figured I’d put it out there.

7 Likes

A public report should be a better way to handle this. (See: https://telemetry.mozilla.org/)

If you want private data you should probably bake in some (opt-in) analytics in your client :slight_smile:

1 Like

I’d not be too thrilled with a public report, it would be doxxing the size of private integrations (like SaaS).

Yes, as they have shared them with Certbot in the past. There’s some gists floating around with the breakdown of Certbot versions in-use. Though it might be a bit burdensome to extend that privilege to every client author.

Personally I just collect the running client version as part of the autoupdate mechanism and then grep my access logs. More granular metrics than that would be great, but I wouldn’t agree to it if the roles were reversed, so I don’t do it …

1 Like

The blessing and curse of my PowerShell client is that I don’t host any of the infrastructure responsible for providing installs or updates. And while I could theoretically add something to the client that attempts to phone home with basic telemetry, I’d also wouldn’t appreciate that as a user. Making it opt-in would be pointless without some sort of nag asking them to turn it on and I don’t like those as a user either.

It’s not the end of the world if it doesn’t happen. Would just be cool to be able to access data that likely already exists.

4 Likes

Telemetry is a potentially thorny issue. Many users complain about it for popular tools but given that certbot itself collects email addresses for the EFF mailing list I think it’s all relative to what you’re trying to achieve and users as a whole are quite understanding.

I wouldn’t really mind if LE published a report on requests per client user agent but I can imagine that’s not great for everyone.

Certify The Web (for instance) has over 250K (new user) downloads and 80K users who have telemetry enabled (Azure Application Insights). It’s been great to be able to see what distribution of OS/app versions that are in use, and it’s occasionally useful to see exception reporting. If you also track feature usage events then you can start to see whether things like DNS providers are getting used or not - for instance a worrying number of people opt to use Manual DNS instead of having real automation. As yet there have been no complaints from users regarding telemetry reporting.

I’d recommend you try (the free 90-day data) application insights in Posh-ACME, you will get some user push back but it’s in your interest as a maintainer and as long as you can switch it off I don’t see an issue (obviously some users will, but their use of your tool is optional to them and they didn’t pay for it - people are sometime quick to make moral judgments on the software they are using for free, at the expense on the maintainer).

Here is a difficult to read chart of the those DNS provider used in Certify which are based on Posh-ACME providers over the last 90 days (not a huge number - most people are using the other built-in providers like Route53):

The key order is Other, Hetzner, Zonomi, Dynu, ClouDNS, GCloud

Here is the same chart across all DNS provider methods (including manual and custom scripting):


The key order is Manual, Other, acme-dns, Cloudflare, Go Daddy, Route53 - you can see there’s a lot of Manual DNS happening despite our UI warning that Manual DNS should be used only for testing or as a last resort.

2 Likes

Private reports also incentivize developers to participate in application registration and not spoof user-agents.