Renewal failure


#1

Hello,

We are a hosting company managing thousands of certificates.
Over the last year we tried to renew around 1000 certificates and 800 of them failed with error 403. Most of the certificates were 2 months old when we tried to renew them. We worked around this by simply creating new certificates instead of renewing. Renewal of newly created certificates always succeeded.
Now we upgraded from acme-v01 to acme-v02 and all acme-v01 certificate renewals fail. Instead of creating new certs, this time we would like to figure out what the cause of the issue is.

Questions:

  • can acme-v01 certificates be renewed with acme-v02?
  • why do so many renewals fail? Are the 2 months old certificates too old for renewals?
  • We try to create a certificate with urls: *.domain.be and domain.be using dns-01 validation. TXT records are properly created but validation fails for the second url. Can we use dns-01 validation when TXT records are created on the same domain?

Let me know if you need any further information.


#2

Can you provide more information?


Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. https://crt.sh/?q=example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is:

I ran this command:

It produced this output:

My web server is (include version):

The operating system my web server runs on is (include version):

My hosting provider, if applicable, is:

I can login to a root shell on my machine (yes or no, or I don’t know):

I’m using a control panel to manage my site (no, or provide the name and version of the control panel):


On a technical level, there is very little difference between “renewing” a certificate and “creating a new” certificate. You’re using the API to issue a certificate in the same way. The only difference is how Let’s Encrypt internally applies certain rules, primarily the rate limits.

Some of your questions are hard to answer, because it doesn’t work that way.

With what error messages do they fail?

It’s recommended to renew certificates after about 60 days.

With what error message is it failing? That’s fine, but depending on how your ACME client works, you probably have to ensure that both records exist simultaneously, and one of them isn’t deleted too quickly. With some DNS software, that might be more difficult.


#3

Hi @aleksandra.perunska

a certificate doesn’t know if it is created via acme-01 or 02. So this is a question / problem of your client.

Perhaps your client uses tls-sni-01 - validation, which is deprecated. So a renew-command using older parameters fails, but creating a new certificate works.

If you want one certificate with two names *.domain.be + domain.be, you have to create two dns text entries with the same name

_acme-challenge.domain.be

and different values.

The data sent from a client to Letsencrypt to create a new order are the same - if you start new or if you use a renew option of your client. So if your client has trouble with old parameters, you can create a new certificate. It’s only a client problem.


#4

My domain is: https://vandevaenwinning.be/ (just an example, there are hundreds)

I ran this command: we tried to renew a cert using acmephp client (getCertificate() method)

It produced this output: [unauthorized] The client lacks sufficient authorization: Error finalizing order:: authorizations for these names not found or expired

I can login to a root shell on my machine (yes or no, or I don’t know): yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel): no

When we create a new certificate and try to renew it immediately afterwards, it works. But if we wait ~2 months, the renewal fails with the error above.

We manage DNS records on our own DNS servers and the TXT records do both exist simultaneously which we verified with dig command.


#5

I don’t know a thing about that client, so I can only speak generally.

Authorizations are valid for a certain amount of time, and then you have to validate the hostnames again. Currently, they expire after 30 days.

Can you examine the client’s logs, and how it works?

Is it trying to validate again? If so, is validation failing? If so, what’s the error?

Does it have some sort of bug, like validating with one account and then trying to issue a certificate with a different account?


#6

Authorizations are valid for a certain amount of time, and then you have to validate the hostnames again. Currently, they expire after 30 days.

“Authorization” refers to the challenge received during cert creation? Does that mean we have to re-validate the cert to do a renewal beyond 30 days?


#7

Pretty much. In the ACME protocol, authorization and challenge have different but related meanings. (An authorization object includes one or more challenge objects.) Speaking informally, people might use them more interchangeably.

Yes.


#8

Thanks for the clarification, that’s probably the root cause there. We’ll look into it.

Regarding the DNS validation with two TXT records on the same domain:
One example is that we tried to create a cert for *.firecontrol.be and firecontrol.be
The validation worked for *.firecontrol.be but for firecontrol.be we got
Challenge failed (response: {“type”:“dns-01”,“status”:“pending”,“url”:“https://acme-v02.api.letsencrypt.org/acme/challenge/9yD{truncated}",“token”:"x_D5Vrg{truncated}”})

We verified with dig command before and after attempting validation that both DNS records existed. Do you have any ideas how we could debug this?


#9

Do the logs have more details? If it’s still “pending”, that means the client hasn’t tried to validate it yet.

Edit: Probably.


#10

The two txt entries are looking good:

D:\temp>nslookup -type=txt _acme-challenge.firecontrol.be.
_acme-challenge.firecontrol.be text =

    "1EkCCoSgcJ6q2frEbKmiOfgj08eDXzo1N1M56YUfDcQ"

_acme-challenge.firecontrol.be text =

    "h0j9aa1BPx6t9UHZkEIAYBgvCI05SRXqCHIxhSGrJvM"

Has the log an order url? Something like

https://acme-v02.api.letsencrypt.org/acme/order/yourAccountId/OrderId

If yes, check both authorization urls, if they are valide or pending.


#11

The response is “Challenge failed” - so the client must have tried to validation, no?
For logs, see https://pastebin.com/CGbH4zyU
I’m not sure if the logs help. At the end you can see that the first domain validated and the second one generated an exception.


#12

I don’t know. Not necessarily. “Challenge failed” is an error message that comes from the client. I don’t know the circumstances in which it can be thrown.


#13

Authorizations are valid for a certain amount of time, and then you have to validate the hostnames again. Currently, they expire after 30 days.

I’m confused about one thing - the documentation here https://letsencrypt.org/docs/integration-guide/ says

“We recommend renewing certificates automatically when they have a third of their total lifetime left. For Let’s Encrypt’s current 90-day certificates, that means renewing 30 days before expiration.”

If we want to do a renewal after 60 days, we have to go through validation again. How is this “renewal” different from simply creating the certificate from scratch? It’s possible to extend the expiration of the cert without re-validation within the first 30 days - that’s what what I would consider a “renewal”. Is there something I’m missing?

BTW thank you for the answers so far, this is probably the most responsive support I’ve seen.


#14

On a protocol level, it isn’t different. The difference between “creating” and “renewing” certificates is mostly an administrative or management notion. At a low level, you’re just making certificates either way.


#15

We investigated more and right now we have one question - if http validation goes through, does that mean that DNS validation is forever pending? We seem to have that situation with bedrijfskat.be.


#16

If you validate using HTTP-01, the authorization’s status will be “valid”, the HTTP-01 challenge’s status will be “valid”, and the other challenges in the authorization will remain “pending”.