Do *NOT* remove TLS Client Auth EKU!

Feature Requests
63

Yeah, I didn't post here in case it just confused things more than helped things. I think there are lots of different Microsoft-run email products, so I don't know as they all do this. But there are configurations where the current recommendation is authenticating your mail server to theirs with a publicly-trusted client certificate, yes.

I think there's sometimes a lot of focus on Let's Encrypt just because they're more proactive and transparent than other CAs about their planning. This Chrome policy charge really isn't anything specific to Let's Encrypt, and while people might have found utility in using Let's Encrypt certs for client certificates in the past and present (I know I have in the past, at least), it really isn't part of their core mission.

I think that's actually the ultimate goal here: To have client certificates be a separate hierarchy, and only used if needed. Kind of like how S/MIME and Code Signing and such are separate things for separate purposes. I think the question that may really be on people's minds is around what the typical user cost and experience will be for getting such certificates (in an automatable way such as through ACME), and from what CAs. I'm not aware of other CAs having stated their plans yet, so there's uncertainty. If we could just say "Oh, the such-and-such CA is running a client certificate hierarchy, and you can just point certbot to --server https://such-and-such.example to get free client certificates just like you are now", then I think there'd be a lot less consternation. But we don't yet know what the future will bring.

Well, if you're using Google CA right now for client certificates, please ask them for what their plans are for complying with the Chrome root program requirements to split out server and client roots.

It's a tough problem. Each root program really can use whatever criteria they want for including or excluding roots, which I'm sure makes things tricky for the CAs to try to be included in all the places they want to be included. The state of the public PKI is certainly better than it was in say, 2011. The CA/B Forum (created the next year) has helped with that governance and discussion to make common standards. But certainly root programs have sometimes felt the need to push past that and just implement their own additional requirements to move the industry forward: Google unilaterally required Certificate Transparency logging, and Apple reduced maximum certificate validity to ~one year, both I think (correct me if I'm wrong) without having full buy-in from everyone, though the rest of the industry eventually followed. I don't know as that's really the best approach, and I'm not thrilled about so much power being concentrated in a couple specific big companies either. (And the issue goes well beyond CA root programs: W3C lost control of the HTML specification because browsers wanted to do their own thing, and many web standards are basically "What does Chrome do" now.) I'm not sure forcing users to pick and choose which CAs they really want to trust rather than having browsers ship with their own carefully-chosen defaults would actually be a better world, though.

Exactly. It's just not clear (at least to me) how a public-CA-driven client certificate hierarchy has enough advantages over using private CAs to make it worth non-profit organizations spending effort on it. Maybe it is worth it, and another non-profit, or maybe some big tech company, will make something. I can see how it's worth asking around for who's interested in running such a thing. But I don't think there's a whole lot of interest out there yet. Or maybe there just needs to be clearer guidance and how-tos out there about how people should set up and run their own private CAs, both for client certificates and for server ones.

And while I appreciate your trust in me, I'm not officially a "Community Leader" in the sense of having that "title" in the system. And regardless of title, to be clear I'm just a random guy on the Internet who sometimes has some coherent thoughts, but I have no idea what Google/Apple/Mozilla/Microsoft/CAs/etc. are planning on doing next.

7 Likes
64

Interesting! I take it that the name collision between atProtocol and AT Protocol is purely coincidental, i.e. that the two protocols are unrelated?

There appears to be only a single mention of TLS in the whole of docs.atsign.com, and none at all of EKU, mTLS, or of how web PKI DV certificates are used. Do you mind pointing to an outline if it exists, or briefly describing here if not?

What does "keep the lumps out" mean, in practical terms?

I'd suggest that that's a somewhat unfortunate way of thinking about it. The Chrome team's trust decisions are amongst the highest-stakes trust decisions made by any non-state actor, anywhere in the world, for any purpose at all. Several billion people depend upon their getting it right for protecting the transmission of sensitive personal data. It is in fact somewhat odd that they have only been sporadically ahead of CABF on requirements like this.

(I'm not in any way associated with Google, but am concerned about the uncomfortably large attack surface that web TLS still presents. I view the recent Chrome root policy changes as years overdue.)

The Chrome Team's stated view appears to be roughly that use-case overloading compromises decision-making and therefore security, so it's ending its participation in that overloading so it can make security decisions appropriate to its use case. Much of the response appears to have assumed that LE certificates were already capable of everything that certificates can be used for, apparently unaware that LE decided to exclude S/MIME message signing and encryption, code-signing, and [I think] time-stamping from the outset. Even Google's own Public CA hasn't excluded that much. (That is: people are genuinely unaware that LE is specifically focussed on the web.)

As above, my guess is that the Chrome Team's risk assessment is utterly compelling and so it will not be possible to construct a reasonable basis for them to not do what they're doing. (Which is to say, my contribution to such a conversation would be to research and draft a rational argument, but so far as I can tell there isn't one.)

I would offer a fourth option, and would be interested in your view of whether it's applicable to atProtocol: DANE TLSA (RFC 7671). Stated as a question: why are you looking to third-party CAs for DV certificates at all?

3 Likes
65

It's more than that: the WebTrust criteria are explicitly a minimum standard. That individual participants will apply additional controls is taken for granted. I'd suggest that it would be surprising/concerning for the big guys not to be doing so.

1 Like
66

I was also bit by this. I switched to tlsserver profile, and when my XMPP certificate got renewed today, it failed to make any S2S connections :(.

I'd to revert to classic profile.

Could we please keep TLS client auth EKU ?

Thanks!

67

Please read the entire thread. Thank you.

2 Likes
69

Please refrain from making these kinds of trolling and incorrect statements.

4 Likes
70

Google isn't paying Let's Encrypt to break anything. Google is setting requirements for all CAs, if they want to be trusted in Chrome, to split out server certificates from client certificates.

Now I'm not that familiar with XMPP, but I don't know why connections between XMPP systems would be harder to migrate to another CA than any other system. That "another CA" might be a private CA special-purpose created just for each pair of XMPP systems, or a private CA run by the XMPP community that's more widely trusted, or maybe a public CA (other than Let's Encrypt) that will be choosing to continue to offer a client-certificate hierarchy.

If you want to take the fight to Google, that you don't like forcing changes of having certificates used for public websites be different from certificates used for integration between backend systems, you're more than welcome to. But Google is trying to protect their users the best they can, and generally having single-purpose certificates and roots can help minimize attack surfaces. I don't know as any "community leaders" here are that thrilled with needing to make those kinds of changes, but I think we just recognize that engineering is sometimes (always?) about trying to make the best decisions possible while understanding tradeoffs.

And Let's Encrypt, as a nonprofit with only a few dozen employees, is focused first and foremost on securing https web servers. I'm sure they'd be happy to run a client certificate root if they had the funding and thought it was a core part of their mission. But they're not the only CA in town, and they're not even the only free CA in town.

6 Likes
71

They are. XMPP operates like the web, and therefore with web CAs: the desire is trusted communication between arbitrary units that don’t know each other, other than via the FQDN. Only that it’s s2s and both sides authenticate the other from the certificates.

72

Maybe just continue doing what you're doing without the client EKU? :man_shrugging:

Might not be best practice, but it seems pragmatic. Not like the fundamental nature of certs will change or certs will become unverifiable.

2 Likes
73

That’s not possible unless we patch OpenSSL to remove it checking the EKUs in the first place.

74

It would make more sense to change your software to configure which EKUs are allowable: Server, Client, both.

5 Likes
75

SSL.com just announced:

It's not just Let's Encrypt. Really all CAs will go this road.

More on their blog: Removal of the Client Authentication EKU from TLS Server Certificates – What You Need to Know - SSL.com

9 Likes
76

Unfortunately — as has been hashed out in some detail above — what you're proposing is not a feasible approach for LE to take.

I'd suggest that the best way forward for XMPP administrators is to:

  • Implement and test all three of dialback, DANE, and POSH for XMPP (RFC 7712), both for others to authenticate you and for you to authenticate others.
  • Bring up a staging instance with a TLS certificate lacking the client EKU and attempt connection to each of the other servers that yours frequently connects to.
  • For any that fail connection, reach out to the administrator directly and enquire whether they're willing to implement one or more of these mechanisms for client authentication to resolve the issue with yourself, and ideally to implement all three in both directions to improve the resilience of the XMPP ecosystem.

Given the small size of the XMPP ecosystem, it is likely that a handful of adminstrators doing this diligently will be enough to get the entire XMPP ecosystem weaned off TLS client certificates for S2S connections prior to suitable TLS certificates becoming unavailable from the free-of-charge CAs.

In more detail:

Rather like the SMTP ecosystem, the XMPP ecosystem solved this problem (authenticating a stranger claiming control of a specific domain name) long ago. Free-of-charge dual-use TLS certificates — available to anyone, via an automated mechanism, pre-integrated into many software stacks and services, ... — provided an expedient way to streamline the process as a side effect of EFF et al's strengthening the web, but were never necessary for LE, nor for the SMTP and XMPP ecosystems, and unfortunately have reached the point where they're no longer feasible for LE.

Fortunately, XMPP S2S mutual authentication without TLS client authentication is a long-solved problem:

  • XMPP dialback was published as part of IETF's XML core proposed standard more than two decades ago (RFC 3920), and later separated into a XMPP standard (XEP-0220).
  • The IETF proposed standard for TLS in XMPP (RFC 7590) makes clear in S3.4 that servers should authenticate other servers connecting as clients via TLS, but are not required to, and points to Domain Name Associations (DNA) (RFC 7712) for the approach to take.
  • DNA in turn specifies what methods of authentication to attempt in what sequence, and in particular acknowledges that:

The upcoming changes for LE and SSL.com are simply a current example of this, in that administrators who have been enjoying free-of-charge dual-use certificates are unwilling to switch to a paid service. Notably, not only is the use of DANE to publish an authenticator through DNSSEC for this purpose already specified in a standards track document (this wasn't clear to me earlier in the conversation), so is an additional mechanism called POSH for publishing similar information via HTTPS.

As part of my broader exploration of whether there is some value in LE changing its approach, or in recruiting another sponsor to fund the work (whether within LE or elsewhere), it's now clear that both the SMTP and XMPP ecosystems have had rock-solid solutions in place for a long time that don't rely on TLS client authentication at all, so there's no problem to solve here.

7 Likes
77

Deliberately breaking TLS libraries would appear undesirable... better to solve the actual problem (which, it turns out, the XMPP standards already do).

4 Likes
78

Given the small size of the XMPP ecosystem, it is likely that a handful
of adminstrators doing this diligently

Administrators cannot do much here, given very prominent XMPP servers
are written in… very opinionated and rare languages (Erlang especially,
but others as well), this is something for the developers of the
respectice server softwares.

the entire XMPP ecosystem weaned off TLS client certificates for S2S

Unlikely, as older servers are also still around.

prior to suitable TLS certificates becoming unavailable

Extremely implausible; alone the release cycle of most distributions
encompasses two or more years, and this just missed e.g. the Debian
trixie freeze, so late 2027, at the earliest, will have a chance to
get this onto servers actually ran in real life.

79

Maybe I'm not following, but I'm not sure what this thread could accomplish as Let's Encrypt (or any other CA for that matter) cannot comply with the request to continue including a Client Auth EKU without being distrusted by the Chrome root program. Given how the majority of certificates are issued for webserver, that's probably a non-starter.

The only logical course of action I can see is getting all the XMPP server operators together to implement the RFCs described before the XMPP ecosystem collapses in 2026.

Unfortunately this fight appears to be Google vs XMPP

4 Likes
80

I'm not suggesting modifying any software. Bear in mind that XMPP has been around far longer than LE has, and has long-established s2s mechanisms that don't rely upon TLS client authentication at all.

You've not pointed to empirical data[1], but even if a significant number of museum exhibits are still in production use on the public Internet[2], this is probably an opportune time for those sites to upgrade at least to versions released in the current decade. xmpp.org evaluates XML Core compliance as Advanced for 6 of the 7 servers that they list. There are plenty of options.

It seems wildly improbable that currently-supported versions of Debian and other distributions have 20-year-old versions of XMPP servers in them.

1: but please share it if you have it!
2: And how good is their TLS implementation even if they are?

5 Likes
81

You're not the only one who's confused :slight_smile:

Yes, the literal request to not exclude client EKUs from the LE root certificate would invalidate LE's reason for existence, as Chrome would then exclude LE, at which point LE certificates would not solve the problem that the overwhelming majority of subjects obtain them for. Some alternative approaches that weren't specifically requested when the ticket was opened have nonetheless been discussed:

  • Persuade the Chrome team not to make this change. This is pretty clearly outside of LE's control. The Chrome team hasn't published their detailed risk assessment, but they have made clear that they're removing cruft. Given the risk that it's their hands I'd suggest that this is not only worthwhile, it's years overdue.

  • Spin up a second root which continues to permit client EKUs — or both client and server EKUs, or omits EKUs altogether — and offer it to operating system developers/vendors. This makes some sense as LE has already built the mechanisms, the security program, the backing, and the trust in its name. However, this lies outside of their mission and would cost significant additional resources. My interest has been to work out whether there's an actual problem to solve, where to house a solution, and a basis for recruiting adequate sponsorship. As it's now clear that there isn't a material problem, this line of argument appears moot.

So far as I can tell, this is not a serious risk. There are at least half a dozen servers that implement dialback, if not DANE and POSH. I'd guess that enough XMPP operators are pragmatic enough to get through the required configuration changes soon enough to prevent the system falling apart. Even the less proactive operators are likely to become very interested in doing so once they experience significant disruption. No ecosystem-wide coordination is required in either case.

5 Likes
82

A correction

Somehow I'd confused Mirabilos talking about hundreds of people examining SMTP TLS client authentication information with the size of the XMPP ecosystem. Some quick poking around puts the latter into at least tens of thousands of servers[1].

This is a much larger problem than a handful of diligent administrators can drive a solution to, however the "no ecosystem-wide coordination is required" reasoning above still applies. The self-interested behaviour of operators and the existing DNA framework provides a sufficient attractor for the ecosystem to converge on, and to do so rapidly as operators individually recognise the issue.

1: How Prosody developers spent 2020:

Shodan reports over 85000 federating XMPP servers on port 5269 in total

Doing a current search on Shodan requires creating an account there, so I didn't, but presumably the ecosystem hasn't shrunk 90% in 5 years.

3 Likes
83

Per section 8.4 of the Verisign Registrar Reference Manual v3.09, we are required to hold such a certificate [a certificate with clientAuthentication] chained to a publicly trusted CA of Verisign's choosing.

found a real use case: Domain registrar to domain registry connections.

1 Like