Using Let's Encrypt certificate by Kafka brokers?

Has anyone used Let's Encrypt certificates for Kafka brokers?

I managed to import the wildcard certificate we already have into Kafka's Java Key Storage and run Kafka with it, but connections keep failing on "invalid certificate" type of errors.

Ref for Kafka SSL configuration: Encryption with SSL — Confluent Documentation

This link doesn't cover the new Kafka 2.7.0 feature to support PEM format as input, but that's just a matter of format change, the way that Kafka uses it hasn't changed.

One blog post I found talks about the requirements of the certificate by Kafka and that the Subject must begin with /, but I didn't find a way to do that when requesting a certificate.

Hi @amossc,

I don't know anything about Kafka, but Let's Encrypt certificates' Subjects do begin this way. The CN is the first name that was requested by the ACME client.

(This use of CN is obsolete, but still provided by Let's Encrypt for a limited amount of backwards compatibility.)

Also the problem with using a wildcard certificate is that the CN will be CN=* so if your system actually requires then you need to get a specific cert, not a wildcard.

I admit that I'm still confused after a long time trying to make it work, but I got the impression that Kafka would accept having the explicit hostnames in the Alternative Subject Names list, if this matters at all.

It's just that after a very long time tinkering with it and digging the web, I still haven't found indication of anyone doing this (using Let's Encrypt certificates for Kafka), so I suspect that it can't be done.

You mentioned using a wildcard certificate so that would imply that your subject alternative names are non-specific. If your cert that you got from Let's Encrypt does indeed contain the exact host name (not just a wildcard) as part of the Subject Alternative Names then it appears it should be ok.

I think someone else tried it here: SSL connection not working when using the lets-encrypt public cert as CA · Issue #1814 · edenhill/librdkafka · GitHub

A thing I'd check is that your PEM file(s) - if that's what you're using- contain the full chain of intermediates and if necessary the root cert DST Root CA X3 - as a invalid chain would also presumably result in failures.

If there is a debug mode you can switch on for extended logging that would make sense as I'd expect the exact failure to be logged somewhere, I'm sure it's a pretty common problem. If kafka has it's own support community then I'd check/ask there? (and if it doesn't have one, don't use it!)

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.