# K8s cert-manager duplicate cert ratelimited

**URL:** <https://community.letsencrypt.org/t/k8s-cert-manager-duplicate-cert-ratelimited/115186>\
**Category:** Help\
**Created:** [March 4, 2020, 2:27am UTC](https://community.letsencrypt.org/t/k8s-cert-manager-duplicate-cert-ratelimited/115186 "2020-03-04T02:27:10Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![lucasreed](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/lucasreed/32/37691_2.png) [@lucasreed](https://community.letsencrypt.org/u/lucasreed)\
**Post date:** [March 4, 2020, 2:27am UTC](https://community.letsencrypt.org/t/k8s-cert-manager-duplicate-cert-ratelimited/115186/1 "2020-03-04T02:27:10Z")

</div>

Hello, first off thanks for all the hard work. #HugOps to you all, I’m sure today has been a trying time. I’m using cert-manager v0.9.1 within kubernetes and I am still getting a rate limiting error: “too many certificates already issued for exact set of domains” despite the increase in limits. If I look for the certificate name on [https://crt.sh/](https://crt.sh/) I can see certificate creation happening today multiple times with a `notBefore:2020-03-03`(I assume that’s what this crt.sh tool does), but it never makes it down to the cluster via cert-manager, and I get the rate limiting error. I’m out of ideas for fixing this, any suggestions?

---

<div class="post-metadata">

**Author:** ![Phil](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/phil/32/76801_2.png) [@Phil](https://community.letsencrypt.org/u/Phil)\
**Post date:** [March 4, 2020, 2:30am UTC](https://community.letsencrypt.org/t/k8s-cert-manager-duplicate-cert-ratelimited/115186/2 "2020-03-04T02:30:57Z")

</div>

I split you off of the main topic so that specific triage can happen here. Thanks for the #hugops 🙂

---

<div class="post-metadata">

**Author:** ![lucasreed](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/lucasreed/32/37691_2.png) [@lucasreed](https://community.letsencrypt.org/u/lucasreed)\
**Post date:** [March 4, 2020, 2:37am UTC](https://community.letsencrypt.org/t/k8s-cert-manager-duplicate-cert-ratelimited/115186/3 "2020-03-04T02:37:32Z")

</div>

Thanks @Phil! I’m curious, does the crt.sh tool in fact do what I think? I am reading it as the certificate exists via letsencrypt, but the rate limit is preventing cert-manager from pulling it down. Is that accurate, or am I off with my assumptions?

---

<div class="post-metadata">

**Author:** ![Phil](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/phil/32/76801_2.png) [@Phil](https://community.letsencrypt.org/u/Phil)\
**Post date:** [March 4, 2020, 2:40am UTC](https://community.letsencrypt.org/t/k8s-cert-manager-duplicate-cert-ratelimited/115186/4 "2020-03-04T02:40:34Z")

</div>

@lucasreed,

That’s a good question. We run cert-manager for our certificate transparency logs, but haven’t been affected by our own rate limits there. Let’s check what the cert-manager logs say

```auto
kubectl -n ${NAMESPACE} logs ${POD}

```

---

<div class="post-metadata">

**Author:** ![lucasreed](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/lucasreed/32/37691_2.png) [@lucasreed](https://community.letsencrypt.org/u/lucasreed)\
**Post date:** [March 4, 2020, 2:48am UTC](https://community.letsencrypt.org/t/k8s-cert-manager-duplicate-cert-ratelimited/115186/5 "2020-03-04T02:48:55Z")

</div>

Ok, I’ll lay out the steps I’ve done so far. I patch the Certificate object to add a `.spec.renewBefore` of `2159h00m00s`. As expected, cert-manager starts to go through the process of validating the challenge (http01) which it does successfully. It then throws out the error below:

`E0304 02:05:37.297236 1 base_controller.go:189] cert-manager/controller/orders "msg"="re-queuing item due to error processing" "error"="error creating new order: acme: urn:ietf:params:acme:error:rateLimited: Error creating new order :: too many certificates already issued for exact set of domains: REDACTED,REDACTED: see https://letsencrypt.org/docs/rate-limits/" "key"="REDACTED"`

I then gave up and removed the `.spec.renewBefore` property of my Certificate object.

I redacted client-identifying information.

---

<div class="post-metadata">

**Author:** ![Phil](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/phil/32/76801_2.png) [@Phil](https://community.letsencrypt.org/u/Phil)\
**Post date:** [March 4, 2020, 2:56am UTC](https://community.letsencrypt.org/t/k8s-cert-manager-duplicate-cert-ratelimited/115186/6 "2020-03-04T02:56:30Z")

</div>

Gotcha, ok. We increased the duplicate certificate rate limit from 5/3d to 10/3d earlier today. For some reason cert-manager kept on issuing instead of applying the updated certificate. To sidestep this issue, you can add or remove of the SANs on the certificate and reissue the cert.

---

<div class="post-metadata">

**Author:** ![lucasreed](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/lucasreed/32/37691_2.png) [@lucasreed](https://community.letsencrypt.org/u/lucasreed)\
**Post date:** [March 4, 2020, 2:39pm UTC](https://community.letsencrypt.org/t/k8s-cert-manager-duplicate-cert-ratelimited/115186/7 "2020-03-04T14:39:57Z")

</div>

Unfortunately I don’t think that will work for our case. Because of the way the validation happens over an ingress via http-01. We can’t use dns-01 validation because it’s a domain we don’t own or manage. If I add a random domain I don’t believe validation will ever pass, but I can try.

---

<div class="post-metadata">

**Author:** ![Phil](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/phil/32/76801_2.png) [@Phil](https://community.letsencrypt.org/u/Phil)\
**Post date:** [March 4, 2020, 2:50pm UTC](https://community.letsencrypt.org/t/k8s-cert-manager-duplicate-cert-ratelimited/115186/8 "2020-03-04T14:50:11Z")

</div>

@lucasreed,

For extra eyes on this, have you tried posting in #cert-manager on [kubernetes.slack.com](http://kubernetes.slack.com)?

---

<div class="post-metadata">

**Author:** ![lucasreed](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/lucasreed/32/37691_2.png) [@lucasreed](https://community.letsencrypt.org/u/lucasreed)\
**Post date:** [March 4, 2020, 2:51pm UTC](https://community.letsencrypt.org/t/k8s-cert-manager-duplicate-cert-ratelimited/115186/9 "2020-03-04T14:51:23Z")

</div>

Good thought, I’ve been in there following along, but thought I’d start here because of the rate limiting message. I can try the extra domain and then go chat with them if that fails.

Thanks for your responses Phil, I appreciate it!

---

<div class="post-metadata">

**Author:** ![Phil](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/phil/32/76801_2.png) [@Phil](https://community.letsencrypt.org/u/Phil)\
**Post date:** [March 4, 2020, 2:53pm UTC](https://community.letsencrypt.org/t/k8s-cert-manager-duplicate-cert-ratelimited/115186/10 "2020-03-04T14:53:09Z")

</div>

Regarding the extra domains, our [rate limiting documentation](https://letsencrypt.org/docs/rate-limits/) states,

> A certificate is considered a renewal (or a duplicate) of an earlier certificate if it contains the exact same set of hostnames, ignoring capitalization and ordering of hostnames. For instance, if you requested a certificate for the names [`www.example.com` , `example.com`], you could request four more certificates for [`www.example.com` , `example.com`] during the week. If you changed the set of hostnames by adding [`blog.example.com`], you would be able to request additional certificates.

---

<div class="post-metadata">

**Author:** ![lucasreed](https://sea3.discourse-cdn.com/letsencrypt/user_avatar/community.letsencrypt.org/lucasreed/32/37691_2.png) [@lucasreed](https://community.letsencrypt.org/u/lucasreed)\
**Post date:** [March 4, 2020, 5:01pm UTC](https://community.letsencrypt.org/t/k8s-cert-manager-duplicate-cert-ratelimited/115186/11 "2020-03-04T17:01:55Z")

</div>

So, interesting thing today. I was working with another colleague on a different cluster where the setup is more conducive of adding a fake domain. It appeared to work, as the domain was validated. Right after validation it gave the same rate-limiting errors even though it was a new certificate with the new domain added. We are unfortunately at a loss here. I imagine we’ll need to check with the cert-manager folks.

---

<div class="post-metadata">

**Author:** ![system](https://global.discourse-cdn.com/letsencrypt/original/3X/c/a/ca6c06ea1ea201324bba7048c6841ce60236468d.png) [@system](https://community.letsencrypt.org/u/system)\
**Post date:** [April 3, 2020, 5:01pm UTC](https://community.letsencrypt.org/t/k8s-cert-manager-duplicate-cert-ratelimited/115186/12 "2020-04-03T17:01:57Z")

</div>

Closed on April 3, 2020, 5:01pm UTC

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.
