Renewing certs, too many currently pending authorizations

Help
1

Hi!

My domain is: doetinchemslotenmaker.nl (one of many)

I ran this command: the letsencrypt plesk extension does the work

It produced this output: see below

My web server is (include version): linux

The operating system my web server runs on is (include version): deamon 8

My hosting provider, if applicable, is: duocast

I can login to a root shell on my machine (yes or no, or I don’t know): yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel): plesk

Im running the letscenrypt plesk extension on 1 of our servers. Ik have 2 subscriptions with each 230 domains. Last week 350 domains got renewed by the cron, which runs daily, but the last 150 have the “pending authorization” error.

When i look in /var/log/plesk/panel.log i see this error:

[2018-05-30 07:29:49.693] INFO [extension/letsencrypt] Renew certificate of domain ‘doetinchemslotenmaker.nl’: the certificate will expire in less than 30 days at 2018-06-25…
[2018-05-30 07:29:49.696] INFO [extension/letsencrypt] Register to ACME server ‘https://acme-v01.api.letsencrypt.org/directory’ using e-mail ‘ict@allfree.nl’
[2018-05-30 07:29:49.696] INFO [extension/letsencrypt] Validate ACME server using custom CA bundle: ‘/opt/psa/admin/plib/modules/letsencrypt/resources/ca/cacert.pem’.
[2018-05-30 07:29:49.696] DEBUG [extension/letsencrypt] Use existing registration from /opt/psa/var/modules/letsencrypt/registrations/195e649f02aed31a83540d908567b6ec99b5e443.json
[2018-05-30 07:29:49.696] INFO [extension/letsencrypt] Begin validation for domains: doetinchemslotenmaker.nl, www.doetinchemslotenmaker.nl, webmail.doetinchemslotenmaker.nl…
[2018-05-30 07:29:50.117] ERR [extension/letsencrypt] Domain validation failed for doetinchemslotenmaker.nl: Invalid response from https://acme-v01.api.letsencrypt.org/acme/new-authz.
Details:
Type: urn:acme:error:rateLimited
Status: 429
Detail: Error creating new authz :: too many currently pending authorizations: see https://letsencrypt.org/docs/rate-limits/
[2018-05-30 07:29:50.117] DEBUG [extension/letsencrypt] PleskExt\Letsencrypt\Acme\Exception\BadResponseException: Invalid response from https://acme-v01.api.letsencrypt.org/acme/new-authz.
Details:
Type: urn:acme:error:rateLimited
Status: 429
Detail: Error creating new authz :: too many currently pending authorizations: see https://letsencrypt.org/docs/rate-limits/
file: /opt/psa/admin/plib/modules/letsencrypt/library/Acme/Exception/BadResponseException.php
line: 38
code: 0
trace: #0 /opt/psa/admin/plib/modules/letsencrypt/library/Acme/Challenge.php(140): PleskExt\Letsencrypt\Acme\Exception\BadResponseException::create(object of type GuzzleHttp\Psr7\Response)
#1 /opt/psa/admin/plib/modules/letsencrypt/library/Acme/Challenge.php(35): PleskExt\Letsencrypt\Acme\Challenge->requestChallenges(string ‘doetinchemslotenmaker.nl’)
#2 /opt/psa/admin/plib/modules/letsencrypt/library/DomainValidation/AcmeDomainValidator.php(65): PleskExt\Letsencrypt\Acme\Challenge->solve(object of type PleskExt\Letsencrypt\ChallengeSolver\DomainDocRootHttpSolver, boolean false)
#3 /opt/psa/admin/plib/modules/letsencrypt/library/DomainValidation/AcmeDomainValidationTask.php(96): PleskExt\Letsencrypt\DomainValidation\AcmeDomainValidator->validateDomain(string ‘doetinchemslotenmaker.nl’)
#4 /opt/psa/admin/plib/modules/letsencrypt/library/Acme.php(226): PleskExt\Letsencrypt\DomainValidation\AcmeDomainValidationTask->run()
#5 /opt/psa/admin/plib/modules/letsencrypt/library/Acme.php(386): PleskExt\Letsencrypt\Acme->provideCertificate(array, object of type PleskExt\Letsencrypt\AcmeCertOrderContext, object of type PleskExt\Letsencrypt\ChallengeFailed\SkipChallengeFailedStrategy, object of type PleskExt\Letsencrypt\CertificateIssuance\CertSubjectsValidatorRequireNothing, array)
#6 /opt/psa/admin/plib/modules/letsencrypt/library/KeepSecured/KeepSecuredService.php(396): PleskExt\Letsencrypt\Acme->secureDomainAutomatically(string ‘ict@allfree.nl’, object of type PleskExt\Letsencrypt\Bridge\Domain, array, object of type PleskExt\Letsencrypt\CertificateIssuance\CertSubjectsValidatorRequireNothing, boolean true, boolean true, boolean false, boolean false)
#7 /opt/psa/admin/plib/modules/letsencrypt/library/KeepSecured/KeepSecuredService.php(255): PleskExt\Letsencrypt\KeepSecured\KeepSecuredService->renewDomainCertificate(object of type PleskExt\Letsencrypt\KeepSecured\KeepSecuredNotifier, string ‘doetinchemslotenmaker.nl’, object of type PleskExt\Letsencrypt\Bridge\Certificate, object of type DateTime, integer ‘30’, boolean true, boolean false, boolean false)
#8 /opt/psa/admin/plib/modules/letsencrypt/library/KeepSecured/KeepSecuredService.php(134): PleskExt\Letsencrypt\KeepSecured\KeepSecuredService->renewDomainsCertificates(object of type PleskExt\Letsencrypt\KeepSecured\KeepSecuredNotifier)
#9 /opt/psa/admin/plib/modules/letsencrypt/library/KeepSecured/KeepSecuredService.php(90): PleskExt\Letsencrypt\KeepSecured\KeepSecuredService->renewCertificates(object of type PleskExt\Letsencrypt\KeepSecured\KeepSecuredNotifier)
#10 /opt/psa/admin/plib/modules/letsencrypt/scripts/keep-secured.php(19): PleskExt\Letsencrypt\KeepSecured\KeepSecuredService->keepAllSecured()

The strange thing is that i see this error block with this domain a serveral times. So i think it tries to renew more then once? Can someone explain how i can fix this? I also read about “Clearing pending authorizations”, how can i do this? And is it smart to do this now?

I hope soneone recognises my issues and knows the solution. Because i really need my pendings domains to renew within 27 days.

Thanks for your help!

Rutger

2

Unfortunately a high number of pending authorizations on an account usually indicates a problem with how the ACME client is built (in this case, Plesk's Let's Encrypt plugin).

At the moment, the only way to clear pending authorizations is if you know the URL of the authorization.

I wrote such a tool that can do this, and although it was written for use with Certbot, it should in theory work with any client that produces verbose enough logs.

Unfortunately, unless your Plesk logs contain the authz URLs (they look like https://acme-v01.api.letsencrypt.org/acme/authz/CgnDk3EGMaLajCxJCy4hGqiTDLFrpdFxpftGlKWwAXA), then you're out of luck.

The only other thing you can do is to abandon your ACME account/registration and issue certificates from a new one:

However, I would caution you against just registering a new account and marching on as if nothing had happened, as the issue will probably come up again if the Plesk extension continues to cause these pending authorizations to stack up. You'd need to report this behavior to the developer of the extension. and it would be helpful to send them a long history of logs as well.

1 Like
3

Hi _az,

Thanks for your quick response.

I have installed you tool, but i get this error:
Command: sudo ./clear-authz < /var/log/plesk/panel.log
Error: sudo: ./clear-authz: command not found

Can i even run it like this? Or cant i enter the account key json? Like this?
sudo ./clear-authz /opt/psa/var/modules/letsencrypt/registrations/195e649f02aed31a83540d908567b6ec99b5e443.json

It seems to try make new authz, which is strange? In my log i cant see any authz urls. So am i out of luck then?

Or can i just wait the 7 days? And if so, how can i make sure this time the rest will be renewed after the 7 days wait?

Rutger

4

Yes. You can't clear the authorizations.

Try temporarily renaming the registration file to try forcing Plesk to register a new ACME account. Since authzs are related to ACME accounts, this would allow you to evade the rate limit.

I'm not sure how low long pending authorizations are kept open before they expire, so I'm not sure whether you'll be waiting 7 days or 30 for them to expire naturally ...

5

Ok. Can i rename the registration file without any trouble for my existing cert which got succesfully renewed?

And after checking my cron again, i see its running hourly instead of daily. Is this a problem?

i really appreciate your help. Thanks for responding!

6

There's no problem with it running hourly, it should only actually "do things" when necessary. However, when it is malfunctioning (as it has been for you), this may cause the pending authorizations to stack up very quickly.

I'm not sure, it might be dangerous. I don't use Plesk. You may need to reach out to Plesk support to figure that part out - the Plesk developers don't hang out on this forum.

It won't break your existing certificates but it could interfere with future renewals.

7

Ok thanks. I get it. Just trying: Is there someone here with plesk experience? Because the plesk forum people sends me back to this forum. Saying they cant change anything about the ratelimit.

Im not sure whats the best thing to do now. Just wait till the pending state clears feels not right.

8

That's true, but they might be able to figure out whether the necessary authorization identifier data has been logged somewhere, or whether there's a straightforward way to force Plesk to create a new ACME account.

9

i recently went through a similar problem ...

Last week it was: 7 days for a pending authorization to expire; 30 days for a successful validation to expire.

Now please don't yell at me, LetsEncrypt staff, but this is a suggestion for debugging/fixing this situation (though not the Plesk bug)

There was a boulder/LetsEncrypt update a while back where the pending authorizations are cached/re-used during the expiry. Stated differently... every time "Account A" asks to validate "domain1.example.com", it should return the same authorization challenge (I'm not sure if it will be the same once a validation is triggered or not).

With a ratelimit of 300 pending authorizations, ~300 updated domains, and ~150 that haven't updated... I would be led to believe there are (somehow) pending authorizations left on the domains which successfully authorized.

I think it should be possible to iterate over the 'correct' domains and request a new authorization for them. If the request doesn't fail from the ratelimit, that should be an existing pending authorization -- and you could then use that authorization info to issue a cancel request.

1 Like
10

I won't yell at you. :slight_smile: I think this is actually a very reasonable and good suggestion. The goal of implementing pending authz reuse was to making "pending authz rate limit" a thing of the past. Certainly the number of people having this problem has decreased a lot.

One category where people still tend to run into this problem is when issuing for more than 300 domains more-or-less simultaneously, since you can wind up having 300 authorizations pending before you start validating some of them.

One small suggestion for this particular case: Try spreading out your renewals a little bit so you have more like 100 per day. That way you're less likely to run into the pending authz limit when doing big batch renewals. In our integration guide we recommend randomizing renewals a little bit so if you onboard a bunch of domains over time, eventually your renewals spread out so they're not all on the same day.

Pending authorizations last 7 days. Once they've been validated, the "final" authorization lasts 30 days.

2 Likes
11

Thank you. The first pending error was from last sunday. So i guess ill have to wait till tomorrow and see what happens. IF they are not renewed by monday, ill have to try and make new registrations, right?

12

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.