Issues with getting a certificate in kubernetes

Help
1

Hello there,

I am trying to set up a kube clusterissuer using cert-manager and ingress.
My subdomain is: titiler-dev.undpgeohub.org

kubectl get certificate -n geodemo
NAME READY SECRET AGE
geodemo-cert False geodemo-cert 41m

next

kubectl describe certificate -n geodemo

Status:
Conditions:
Last Transition Time: 2022-10-09T23:09:37Z
Message: Issuing certificate as Secret does not exist
Observed Generation: 1
Reason: DoesNotExist
Status: True
Type: Issuing
Last Transition Time: 2022-10-09T23:09:37Z
Message: Issuing certificate as Secret does not exist
Observed Generation: 1
Reason: DoesNotExist
Status: False
Type: Ready
Next Private Key Secret Name: geodemo-cert-w9zpm
Events:
Type Reason Age From Message

Normal Issuing 42m cert-manager-certificates-trigger Issuing certificate as Secret does not exist
Normal Generated 42m cert-manager-certificates-key-manager Stored new private key in temporary Secret resource "geodemo-cert-w9zpm"
Normal Requested 42m cert-manager-certificates-request-manager Created new CertificateRequest resource "geodemo-cert-b48kq"

and into the request

kubectl describe certificaterequest geodemo-cert-b48kq -n geodemo

Status:
Conditions:
Last Transition Time: 2022-10-09T23:09:37Z
Message: Certificate request has been approved by cert-manager.io
Reason: cert-manager.io
Status: True
Type: Approved
Last Transition Time: 2022-10-09T23:09:38Z
Message: Waiting on certificate issuance from order geodemo/geodemo-cert-b48kq-3670509394: "pending"
Reason: Pending
Status: False
Type: Ready
Events:
Type Reason Age From Message

Normal cert-manager.io 44m cert-manager-certificaterequests-approver Certificate request has been approved by cert-manager.io
Normal OrderCreated 44m cert-manager-certificaterequests-issuer-acme Created Order resource geodemo/geodemo-cert-b48kq-3670509394

next i looked at the order

kubectl describe order geodemo-cert-b48kq-3670509394 -n geodemo

Status:
Authorizations:
Challenges:
Token: EhxmKzrsFKrGgIoGx39VnNjMvXZWosyGX8cOIrCB3Oc
Type: http-01
URL: https://acme-v02.api.letsencrypt.org/acme/chall-v3/162759015656/QIkh8Q
Token: EhxmKzrsFKrGgIoGx39VnNjMvXZWosyGX8cOIrCB3Oc
Type: dns-01
URL: https://acme-v02.api.letsencrypt.org/acme/chall-v3/162759015656/GHr9LA
Token: EhxmKzrsFKrGgIoGx39VnNjMvXZWosyGX8cOIrCB3Oc
Type: tls-alpn-01
URL: https://acme-v02.api.letsencrypt.org/acme/chall-v3/162759015656/mAnf-g
Identifier: titiler-dev.undpgeohub.org
Initial State: pending
URL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/162759015656
Wildcard: false
Finalize URL: https://acme-v02.api.letsencrypt.org/acme/finalize/768432736/133100419896
State: pending
URL: https://acme-v02.api.letsencrypt.org/acme/order/768432736/133100419896
Events:

And this is where i stopped because I can not figure out what is happening.

at the same time the cert-manager pod logs said:

E1009 23:30:54.254553 1 controller.go:166] cert-manager/orders "msg"="re-queuing item due to error processing" "error"="create not allowed while custom resource definition is terminating" "key"="geodemo/geodemo-cert-b48kq-3670509394"
E1009 23:41:29.765609 1 controller.go:166] cert-manager/challenges "msg"="re-queuing item due to error processing" "error"="[error reading (cluster)issuer "letsencrypt-staging": clusterissuer.cert-manager.io "letsencrypt-staging" not found, namespaces "dev" not found]" "key"="dev/tls-secret-lclvt-2994752350-2717540044"
E1009 23:52:14.263448 1 controller.go:166] cert-manager/orders "msg"="re-queuing item due to error processing" "error"="create not allowed while custom resource definition is terminating" "key"="geodemo/geodemo-cert-b48kq-3670509394"

My cluster-issuer is letsencrypt

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: xxx@yyy
privateKeySecretRef:
name: letsencrypt
solvers:
- dns01:
cloudflare:
email: xxx@yyy
apiTokenSecretRef:
name: cloudflare-api-token-secret
key: api-token
selector:
dnsZones:
- 'undpgeohub.org'
- '*.undpgeohub.org'

so it makes no sense for the cert-manegr to complain about letsencrypt-staging

The cluster is deployed in AKS

Any hints are appreciated

2

Welcome to the Let's Encrypt Community! :slightly_smiling_face:

I have been immersed in this type of setup (AKS and ingress-nginx) for several months now at my company. You do not explicitly request a certificate with this setup. It occurs automatically based upon deployment of an ingress manifest. From what I can tell from your outputs, it appears that you might have conflicting installations/configurations. Your best bet is usually to uninstall your cert-manager and ingress controller(s), fix your deployment configurations, then reinstall everything. I highly recommend reviewing the documentation below.

2 Likes
3

Hi @griffin and thanks so much for replying.

I am aware the certificate is requested automatically by the ingress manifest:
this is the one I use

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: geodemo-ssl-tls-ingress
  namespace: geodemo
  annotations:
    kubernetes.io/ingress.class: nginx
    #nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
    cert-manager.io/cluster-issuer: letsencrypt
    
spec:
  tls:
    - hosts:
      - titiler-dev.undpgeohub.org # update IP address here
      secretName: geodemo-cert
  rules:
  - host: titiler-dev.undpgeohub.org # update IP address here
    http:
      paths:
        - path: "/"
          pathType: Prefix
          backend:
            service:
              name: titiler
              port:
                number: 80

I actually used the link you sent. The truth is this is the first time in my life I am dealing with certificates
given that I have a geospatial background and I find it it quite complicated :slight_smile:

I will delete everything and try from scratch.

I have on more question.

The cluster issuer uses cloudflare API token to solve the challenges. I stored the token as a secret.
Does the Token need to be a base64 string?

solvers:
      - dns01:
          cloudflare:
            email: ioan.ferencik@undp.org
            apiTokenSecretRef:
              name: cloudflare-api-token-secret
              key: api-token

and the secret

apiVersion: v1
kind: Secret
metadata:
  name: cloudflare-api-token-secret
  namespace: geodemo
type: Opaque
stringData:
  api-token: API_TOKEN_VALUE

my ingress does not get an address, and I believe it should

kubectl get ingress -n geodemo
NAME                      CLASS    HOSTS                        ADDRESS   PORTS     AGE
geodemo-ssl-tls-ingress   <none>   titiler-dev.undpgeohub.org             80, 443   1s
2 Likes
4

You should get your ingress working (i.e. accessible from wherever you want access) before attempting to acquire a certificate (and before installing cert-manager at all for that matter). Typically a load balancer is provisioned during the ingress-nginx deployment process with an IP address that is connected one step further up in your network toward your public ingress point (e.g. a private IP address with a route from your corporate firewall or an Azure public IP address). That load balancer connects to your ingress-nginx replica instances (usually two, sometimes more depending upon your needs). There are many online examples of how to configure this (though I'm unsure of the exact addresses at the moment). They typically involve passing a yaml file to the ingress-nginx installer (via helm) that contains a configuration specifying the IP address of the load balancer to be deployed.

3 Likes
5

Hello @griffin

thanks again for you guidance. I looked into my ingress-controller and it is an Azure flavor of nginx.

The whole issue was that in my ingress resource I was using examples from internet. The ingress pod logs were complaining about the wrong ingress class. I then found out my AKS cluster uses a different class addon-http-application-routing not nginx

    #kubernetes.io/ingress.class: nginx
    kubernetes.io/ingress.class: addon-http-application-routing

once I updated the annotation I was able to get my ingress running.

I will look now into the cert-manager as the ingress is working.

Thanks for your advice.

3 Likes
6

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.