We're using 'ssl_tls' mechanism with bitnami/kafka helm chart. We're using Let's Encrypt and cert-manager for issuing the certificate. Created a secret out of the let's Encrypt generated certificates and passed the secret to the
existingSecrets parameter in the helm chart. Now when I'm using KafkaJS library to connect to the Kafka broker, with
ssl: true it is throwing an error:
KafkaJSConnectionError: Connection error: unable to verify the first certificate
Detailed Steps/How to generate:
- Enabled external access to kafka chart so that it gives us an IP at port 9094
externalAccess.enabled: true externalAccess.autoDiscovery.enabled: true externalAccess.service.type: LoadBalancer externalAccess.service.ports.external: 9094 externalAccess.service.domain: ""
- Bound this IP to a domain
- Bound this domain name to Let's Encrypt certificate issuer to issue certificate for this domain
- Renamed these files and used these to create a secret
kubectl create secret generic kafka-tls-0 --from-file=tls.crt=kafka-0.tls.crt --from-file=tls.key=kafka-0.tls.key
- Modified chart value to configure tls part
tls.type: pem tls.pemChainIncluded: true tls.existingSecrets: ["kafka-tls-0"]
- Applied the values of the chart (started broker)
- Now in KafkaJS client setup, tried to pass value to the
brokersparameter in either format
xyz.com:9094, also passed
- Is the flow correct? Or are we going to the wrong direction?
- What is the reason behind the problem? Is this the certificate chain that is being being wrong? (seems like it is!)
- If we can make it work, what will be the next steps for ensuring auto-renewal of the certificates? Is it managed automatically? Or should we have to maintain a script for Lets' Encrypt certificate auto-renewal?