Certificate pending long time with cert-manager

hello, could you please help to understand whether I'm waiting for the certificate, or the process stuck:
https://acme-v02.api.letsencrypt.org/acme/authz-v3/199210635327

My domain is: em.trood.com

I ran this command: cmctl status certificate em-prod-tls-new -n trood-saas-tools

It produced this output:
Name: em-prod-tls-new
Namespace: trood-saas-tools
Created at: 2023-01-29T17:27:31-05:00
Conditions:
Issuing: True, Reason: DoesNotExist, Message: Issuing certificate as Secret does not exist
Ready: False, Reason: DoesNotExist, Message: Issuing certificate as Secret does not exist
DNS Names:

  • em.trood.com
    Events:
    Issuer:
    Name: letsencrypt-issuer-sz
    Kind: ClusterIssuer
    Conditions:
    Ready: True, Reason: ACMEAccountRegistered, Message: The ACME account was registered with the ACME server
    Events:
    error when finding Secret "em-prod-tls-new": secrets "em-prod-tls-new" not found
    Not Before:
    Not After:
    Renewal Time:
    CertificateRequest:
    Name: em-prod-tls-new-pbrnq
    Namespace: trood-saas-tools
    Conditions:
    Approved: True, Reason: cert-manager.io, Message: Certificate request has been approved by cert-manager.io
    Ready: False, Reason: Pending, Message: Waiting on certificate issuance from order trood-saas-tools/em-prod-tls-new-pbrnq-4212886358: "pending"
    Events:
    Order:
    Name: em-prod-tls-new-pbrnq-4212886358
    State: pending, Reason:
    Authorizations:
    URL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/199210635327, Identifier: em.trood.com, Initial State: pending, Wildcard: false
    Challenges:
  • Name: em-prod-tls-new-pbrnq-4212886358-3037336914, Type: DNS-01, Token: yi8nv3TzteJWum2dGuW2Ex18Nf61XUscn8jxOZtSGnA, Key: FRWf7QDlrNR7n-1fQheUT4uokLqWpX-3KdBEjvL9hXM, State: , Reason: , Processing: false, Presented: false

My web server is (include version): nginx-1.19.9

The operating system my web server runs on is (include version): kubelet 1.24.4

My hosting provider, if applicable, is: DigitalOcean

I can login to a root shell on my machine (yes or no, or I don't know): yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): DO Kubernetes Console

Maybe since you are using the DNS-01 Challenge it is taking a long propagation time to get the DNS TXT record for _acme-challenge.em.trood.com. :man_shrugging:

2 Likes

I do not find any such DNS TXT record with nslookup; but that isn't surprising.

$ nslookup -q=txt _acme-challenge.em.trood.com dns1.yandex.net.
Server:         dns1.yandex.net.
Address:        213.180.204.213#53

** server can't find _acme-challenge.em.trood.com: NXDOMAIN


$ nslookup -q=any _acme-challenge.em.trood.com dns1.yandex.net.
Server:         dns1.yandex.net.
Address:        213.180.204.213#53

** server can't find _acme-challenge.em.trood.com: NXDOMAIN


$ nslookup -q=any em.trood.com dns1.yandex.net.
Server:         dns1.yandex.net.
Address:        213.180.204.213#53

Name:   em.trood.com
Address: 46.101.69.189
2 Likes

The challenge failed (it is not pending):

curl https://acme-v02.api.letsencrypt.org/acme/authz-v3/199210635327

      "type": "dns-01",
      "status": "invalid",
      "error": {
        "type": "urn:ietf:params:acme:error:dns",
        "detail": "DNS problem: NXDOMAIN looking up TXT for _acme-challenge.em.trood.com - check that a DNS record exists for this domain",
        "status": 400

Yandex can take 1 hour or longer to propagate among their own authoritative DNS servers. I don't know how you make cert-manager wait longer.

Also, there may be some DNS config issue. See DNSViz report (link here)

4 Likes

Thank you very much!
Duplicated _acme-challenge to yandex.net, and put cname and ns records to point to digitalocean,- looks like that's not the lean solution, just wanted to double check.
But looks like it's just still wrong.

2 Likes

Thank you!
Looks like it's waiting till the timeout and then fails.
DNSViz is very useful. Trying to find out the configuration which will not fail.

3 Likes

Yes, failed again.
nslookup now works fine, but DNSViz shows "SOA RR with owner name not matches the zone name"

Resolved it, but still not working

Have you given time for the DNS to propagate?
Here is what I presently see

$ nslookup -q=txt _acme-challenge.em.trood.com ns1.digitalocean.com.
Server:         ns1.digitalocean.com.
Address:        173.245.58.51#53

_acme-challenge.em.trood.com    text = "FRWf7QDlrNR7n-1fQheUT4uokLqWpX-3KdBEjvL9hXM"
_acme-challenge.em.trood.com    text = "uobe2Wz4lRR29ERxrk_9R2QHpF8QaJUg0VNiQbb01pk"

2 Likes

Yes.
I believe that looks like a right configuration?
If not explicitly stating 'ns1.digitalocean.com' it says "
Non-authoritative answer:
_acme-challenge.em.trood.com text = "FRWf7QDlrNR7n-1fQheUT4uokLqWpX-3KdBEjvL9hXM"
_acme-challenge.em.trood.com text = "uobe2Wz4lRR29ERxrk_9R2QHpF8QaJUg0VNiQbb01pk"

Authoritative answers can be found from:

can it mean that something is wrong?

I believe with the DNS-01 challenge the token changes each time; lets see if a more knowledgeable Let's Encrypt community volunteer can confirm or deny my belief.

From here DNS-01 challenge
"After Let’s Encrypt gives your ACME client a token, your client will create a TXT record derived from that token and your account key, and put that record at _acme-challenge.<YOUR_DOMAIN> ."

2 Likes

Thank you,
it's ok to have multiple TXT records for the same challenge as they say, however I don't understand why letsecrypt keeps failing the check - so my assumption is they might look for an "authoritative answer", which is hardly though

2 Likes

They always will.
When using DNS-01 authentication, one should check that all authoritative DNS servers are showing the expected TXT entry(ies) before proceeding.

But do you really require the use of DNS-01 authentication?
[it is usually the more complicated choice]

4 Likes

Thank you,
yes, we need wildcards, so looks like DNS-01 fits us best. It even worked before the Kubernetes upgrade on DO, no looks like we have to set it up from scratch.

I have the only NS pointing to digitalocean.com and don't understand how to make it more authoritative.

2 Likes

That is something the ACME client should do for you.
Perhaps your issue is related to the "upgrade":

3 Likes

Honestly hardly with that - I'm just resetting cert-manager from scratch.
Does that mean that I have to sign up to cloudflare, and create CNAME or NS to there on my domain registrator?

Now I got totally frustrated:
szverev$ nslookup -q=soa _acme-challenge.em.trood.com
Server: 209.18.47.62
Address: 209.18.47.62#53
Non-authoritative answer:

*** Can't find _acme-challenge.em.trood.com: No answer

Authoritative answers can be found from:

origin = ns1.digitalocean.com
mail addr = hostmaster.trood.com
serial = 1675050171
refresh = 10800
retry = 3600
expire = 604800
minimum = 1800

szverev$ nslookup -q=txt _acme-challenge.em.trood.com ns1.digitalocean.com
Server: ns1.digitalocean.com
Address: 173.245.58.51#53
_acme-challenge.em.trood.com text = "FRWf7QDlrNR7n-1fQheUT4uokLqWpX-3KdBEjvL9hXM"
_acme-challenge.em.trood.com text = "uobe2Wz4lRR29ERxrk_9R2QHpF8QaJUg0VNiQbb01pk"

You've moved the authoritative nameservers to Yandex - you need to stop using DO DNS:

trood.com nameserver = dns1.yandex.net
trood.com nameserver = dns2.yandex.net
4 Likes

Wow, so NS won't work at all? Does that mean I can not use Digitalocean integration even if I host the system there, while my dns is in another place?

I just removed NS from yandex.net and put TXT there, but it's still "Non-authoritative answer"