Rate Limits - Let's Encrypt - Free SSL/TLS Certificates

[Stephen]

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. https://crt.sh/?q=example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: https://psgshared.northeurope.cloudapp.azure.com

I ran this command:

1. I created ingress route to my services on the azure cloud.

It produced this output:

As I could not connect to my services, I tried regeneration of the certificates. on checking the logs from the cert manager pods. I see an error saying below

E0814 13:04:25.230774 1 controller.go:200] cert-manager/controller/orders “msg”=“re-queuing item due to error processing” “error”=“error creating new order: acme: urn:ietf:params:acme:error:rateLimited: Error creating new order :: too many certificates already issued for exact set of domains: psgshared.northeurope.cloudapp.azure.com: see https://letsencrypt.org/docs/rate-limits/” “key”=“tls-secret-3766360619”

Please note this is a production environment and we need to have cert renew so our applications can work.

My web server is (include version): microsoft azure cloud

The operating system my web server runs on is (include version): linux

My hosting provider, if applicable, is: azure cloud

I can login to a root shell on my machine (yes or no, or I don’t know): yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel):
cloud shell from microsoft azure
The version of my client is (e.g. output of certbot --version or certbot-auto --version if you’re using Certbot):

[cpu]

Hi @Stephen, welcome to the community forum

@Stephen

“error creating new order: acme: urn:ietf:params:acme:error:rateLimited: Error creating new order :: too many certificates already issued for exact set of domains: psgshared.northeurope.cloudapp.azure.com:

This error indicates that the ACME client your Azure environment is using has re-created the exact same certificate 5 times. This is causing you to hit the duplicate certificate rate limit.

Checking certificate transparency shows that four certificates for that name were issued on Aug 13th and another today on Aug 14th.

You should contact the Azure Cloud support to find out why this is happening. The ACME client should only issue the certificate once, and then use it. Instead it appears you're repeatedly issuing the certificate and not using it, instead a certificate issued by "Kubernetes Ingress Controller Fake Certificate" is being served.

@Stephen

Please note this is a production environment and we need to have cert renew so our applications can work.

If you aren't able to recover one of the five certificates you've already issued (and the corresponding private key) then I recommend you add another name to the certificate so it isn't an exact match to the previous 5. Alternatively this rate limit will expire on the 17th.

[Stephen]

thank cpu from you prompt response.
But we can wait for 17th and how do i add another name in the certificate. do you mean a DNS name? and do I have to remove the old "psgshared.northeurope.cloudapp.azure.com " DNS from this cert?
here is my cert creation

apiVersion: certmanager.k8s.io/v1alpha1
kind: Certificate
metadata:
name: tls-secret
spec:
secretName: tls-secret
dnsNames:

• psgshared.northeurope.cloudapp.azure.com
  acme:
  config:
  • http01:
    ingressClass: nginx
    domains:
    • psgshared.northeurope.cloudapp.azure.com
      issuerRef:
      name: letsencrypt-prod
      kind: ClusterIssuer

[cpu]

@Stephen

apiVersion: certmanager.k8s.io/v1alpha1

It looks like you're using the Cert-manager ACME client for k8s. Unfortunately I'm not familiar with k8s or cert-manager and can't suggest the best way to add another name or to debug the repeated duplicate certificate issuance.

I recommend you follow cert-manager's troubleshooting documentation. There's also a Slack channel where you might be able to find more immediate cert-manager specific help.

As a note in case you're unaware, the cert-manager client has lots of problems related to hitting rate limits and we are in the process of blocking old versions. You should also confirm you're using the latest version.

[Stephen]

can you recommend another client instead of Cert-manager

[cpu]

We maintain a large list of ACME client options on the Let’s Encrypt website: https://letsencrypt.org/docs/client-options/

[Stephen]

thank for the list. but back to the issue. is the a way to reset this rate limit so I can get the cert. We are having a production issue and all our services are not working. we are now recreated the ingress on the default namespace and removed all other to get this solution working after certificate expired last July

your assistance is appreciated

[cpu]

There is no way to reset the rate limit. The only options are:

1. Wait for the rate limit period to expire.
2. Add additional domain names to the certificate so it isn’t a duplicate of the previously issued certs.
3. Recover one of the previously issued certs and corresponding private key.

I’m sorry to hear you’re in a bind but those are the only options at hand. Pursuing the second option with the help of the cert-manager community is likely the fastest way to remediate your problem.

Once the dust has settled I’d also recommend you pursue monitoring your certificates for expiration and attempting renewal earlier. We recommend trying to renew certificates when they are 30d from expiry so that any problems like this can be caught with breathing room for a fix.