HTTP-01 Challenge Check Fails

My domain is: vaultwarden.home.ulis.family

I ran this command: The ingress for the service has been annotated with cert-manager annotations to automate certificate management.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: vaultwarden
  namespace: ulis-family  
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-stage
    kubernetes.io/ingress.class: nginx
    kubernetes.io/tls-acme: 'true'
    nginx.ingress.kubernetes.io/backend-protocol: HTTP
    nginx.ingress.kubernetes.io/ssl-passthrough: 'false'
    nginx.ingress.kubernetes.io/ssl-redirect: 'false'
spec:
  tls:
    - hosts:
        - vaultwarden.home.ulis.family
      secretName: vaultwarden-tls
  rules:
    - host: vaultwarden.home.ulis.family
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: vaultwarden
                port:
                  name: http
          - path: /notifications/hub
            pathType: Prefix
            backend:
              service:
                name: vaultwarden
                port:
                  name: websocket

It produced this output:

Status:
  Presented:   true
  Processing:  true
  Reason:      Waiting for HTTP-01 challenge propagation: did not get expected response when querying endpoint, expected "_YznpAV6Rk6rcIRHK2h4tFCOaZriRoHzb7Yy_tmnzKA.Ssd4CG0XLfV1t59RndrB1dD4Div33zaYTuBDWOdsfKg" but got: <!doctype html>
<!--[if... (truncated)
  State:  pending

My web server is: Kubernetes Nginx Ingress Controller (Version: rke2-ingress-nginx:4.5.201)

The operating system my web server runs on is: Ubuntu 22.04 LTS

My hosting provider is: I host my services on my personal infrastructure

I can login to a root shell on my machine: Yes

I'm using a control panel to manage my site: No, I'm using Kubernetes and cert-manager to manage my services

The version of my client is: Cert-manager v1.12.1

Additional Details:

The issue started when I tried to create a new certificate. The cert-manager's HTTP-01 challenge is failing, although manual validation from a different network works perfectly.

DNS is set up correctly, and the A record for vaultwarden.home.ulis.family correctly points to my external IP address 80.178.181.10. This has been confirmed using Google's DNS resolver.

However, the Let's Encrypt HTTP-01 challenge doesn't seem to receive the expected token, and instead receives an HTML response.

I've manually checked the token using the following command from a different network and got the expected response:

curl http://vaultwarden.home.ulis.family/.well-known/acme-challenge/_YznpAV6Rk6rcIRHK2h4tFCOaZriRoHzb7Yy_tmnzKA

> _YznpAV6Rk6rcIRHK2h4tFCOaZriRoHzb7Yy_tmnzKA.Ssd4CG0XLfV1t59RndrB1dD4Div33zaYTuBDWOdsfKg

I've been struggling with this for quite some time now and any help would be greatly appreciated!

Your Nginx Ingress configuration might be interfering with the challenge path /.well-known/acme-challenge/....

Check any rules in the Nginx configuration that might be catching this traffic.

That's cert-manager itself trying to make the HTTP request, from inside your cluster, to verify the configuration is correct before requesting a cert. Is it possible it's set up to work properly from outside, but the ingress doesn't work properly from inside the cluster itself?

Try doing the curl from inside the cluster and see if it works there.

cert-manager can be configured to use a different DNS server, which is sometimes required in situations where the cluster-internal DNS doesn't resolve the external names properly:

5 Likes

Thanks, it was my local DNS. Fixed.

3 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.