Certbot - Too Many Pending Authorisations

Help
1

Please fill out the fields below so we can help you better.

My domain is: there are a lot of domains on this issue, I had a problem which seems to be caused by my running certbot too many times when my config files were setup with multiple vhosts per config file (at least from my interpretation of other issues I’ve found posted here, including Error creating new authz :: Too many currently pending authorizations and Error creating new authz :too many currently pending authorizations)
one of the domains is urbanjustice.org

I ran this command: certbot --apache certonly --allow-subset-of-names --email eric@openflows.com -d poker.urbanjustice.org -d www.urbanjustice.org -d urbanjustice.org -d cap.urbanjustice.org -d cdp.urbanjustice.org -d dvp.urbanjustice.org -d hrp.urbanjustice.org -d irap.urbanjustice.org -d mhp.urbanjustice.org -d pcyp.urbanjustice.org -d prop.urbanjustice.org -d snp.urbanjustice.org -d swp.urbanjustice.org -d svp.urbanjustice.org -d vap.urbanjustice.org -d asap.urbanjustice.org -d www.ujctesting.urbanjustice.org -d ujctesting.urbanjustice.org -d cdp.ujctesting.urbanjustice.org -d dvp.ujctesting.urbanjustice.org -d cap.ujctesting.urbanjustice.org -d hrp.ujctesting.urbanjustice.org -d irap.ujctesting.urbanjustice.org -d mhp.ujctesting.urbanjustice.org -d pcyp.ujctesting.urbanjustice.org -d prop.ujctesting.urbanjustice.org -d snp.ujctesting.urbanjustice.org -d swp.ujctesting.urbanjustice.org -d svp.ujctesting.urbanjustice.org -d vap.ujctesting.urbanjustice.org -d asap.ujctesting.urbanjustice.org -d www.ujcstage.urbanjustice.org -d ujcstage.urbanjustice.org -d cdp.ujcstage.urbanjustice.org -d dvp.ujcstage.urbanjustice.org -d hrp.ujcstage.urbanjustice.org -d irap.ujcstage.urbanjustice.org -d mhp.ujcstage.urbanjustice.org -d pcyp.ujcstage.urbanjustice.org -d prop.ujcstage.urbanjustice.org -d snp.ujcstage.urbanjustice.org -d swp.ujcstage.urbanjustice.org -d svp.ujcstage.urbanjustice.org -d vap.ujcstage.urbanjustice.org -d asap.ujcstage.urbanjustice.org -d cap.ujcstage.urbanjustice.org -d test.ujcstage.urbanjustice.org -d www.ujcdev.urbanjustice.org -d ujcdev.urbanjustice.org -d cdp.ujcdev.urbanjustice.org -d dvp.ujcdev.urbanjustice.org -d hrp.ujcdev.urbanjustice.org -d irap.ujcdev.urbanjustice.org -d mhp.ujcdev.urbanjustice.org -d pcyp.ujcdev.urbanjustice.org -d prop.ujcdev.urbanjustice.org -d snp.ujcdev.urbanjustice.org -d swp.ujcdev.urbanjustice.org -d svp.ujcdev.urbanjustice.org -d vap.ujcdev.urbanjustice.org -d asap.ujcdev.urbanjustice.org

It produced this output: Saving debug log to /var/log/letsencrypt/letsencrypt.log
Starting new HTTPS connection (1): acme-v01.api.letsencrypt.org
Obtaining a new certificate
An unexpected error occurred:
There were too many requests of a given type :: Error creating new authz :: too many currently pending authorizations
Please see the logfiles in /var/log/letsencrypt for more details.

My web server is (include version): apache 2.4 (2.4.10-10+deb8u8)

The operating system my web server runs on is (include version): debian linux (jesse)

My hosting provider, if applicable, is: virtual server that I maintain

I can login to a root shell on my machine (yes or no, or I don’t know): yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel): no

from the above listed urls, I’m assuming that I’ve been blocked for a period of time. Is there any way of having that block removed faster?

thanks

2

hi @ericfg

The rate limit for authorisations is described below

You can have a maximum of 300 Pending Authorizations on your account. Hitting this rate limit is rare, and happens most often when developing ACME clients. It usually means that your client is creating authorizations and not fulfilling them. Please utilize our staging environment if you’re developing an ACME client.

If you are testing add the --staging flag until you work through the errors.

Clearing Pending Authorizations
If you have a large number of pending authorization objects and are getting a rate limiting error, you can trigger a validation attempt for those authorization objects by submitting a JWS-signed POST to one of its challenges, as described in the ACME spec. The pending authorization objects are represented by URLs of the form https://acme-v01.api.letsencrypt.org/acme/authz/XYZ, and should show up in your client logs. Note that it doesn’t matter whether validation succeeds or fails. Either will take the authorization out of ‘pending’ state. If you do not have logs containing the relevant authorization URLs, you need to wait for the rate limit to expire. As described above, there is a sliding window, so this may take less than a week depending on your pattern of issuance.

Note that having a large number of pending authorizations is generally the result of a buggy client. If you’re hitting this rate limit frequently you should double-check your client code.

Andrei

3

thanks. I ran the command with the --staging flag and everything worked, but I’m still getting the error without --staging.

I think I hit the limit because I made numerous bad requests when I had my apache config files with two vhosts in each (one for port 80 one for 443) and there are about 40 urls in each request.

now I just need to get my head around how to send the JWS-signed POST to the urls in the log file
for future reference, the documentation on that is at https://github.com/ietf-wg-acme/acme/blob/master/draft-ietf-acme-acme.md#responding-to-challenges

4

wondering if anyone can help me figure out how to handle this.

from the documentation I understand that I need to send the following in a POST to the server in the logs, I understand where to find all the info except how to generate the signature line. Which key do I use? how do I generate the signature to include?

POST /acme/authz/asdf HTTP/1.1
Host: example.com
Content-Type: application/jose+json

{
“protected”: base64url({
“alg”: “ES256”,
“kid”: “https://example.com/acme/acct/1”,
“nonce”: “xWCM9lGbIyCgue8di6ueWQ”,
“url”: “https://example.com/acme/authz/asdf”
}),
“payload”: base64url({
“status”: “deactivated”
}),
“signature”: “srX9Ji7Le9bjszhu…WTFdtujObzMtZcx4”
}

I hope this is a simple question and I’ll smack my head when someone answers and I finally get this into my head.

5

Hello there. I’m struggling with this same issue on a production server. Had a crontab in testing configuration, renewing the cert was set to every minute instead of every nth hour and ofc hit the rate limit. Certs are going to expire tomorrow morning :frowning:. Disabling the HTTPS for now but that’s a huge bummer.

Any help on how to sign the request (and maybe list the /authz/XYZ/ URLs) would be appreciated. The logs have been overwritten already because of the one minute ping :expressionless:, the last one is from Sunday afternoon (UTC+3).

Lessons learned, never overdo renewals with certbot. Never leave cert renewal to the last minute.

6

hi @rampa

You may run in to issues with this though

We recently (April 2017) introduced a Failed Validation limit of 5 failures per account, per hostname, per hour. This limit will be higher on staging so you can use staging to debug connectivity problems.

Andrei

7

Hi and thanks for writing a tool for this @ahaw021. Like I suspected, all the logs have already been wiped with “new-authz” requests. So in my case unfortunately I didn’t get to switch the state of the pending requests. I’ll just have to wait for the sliding window then. Hope you’ll have better luck with this tool @ericfg.

PS. I sent you a pull request which fixes an issue I had with 404’d ACME statuses.

8

logs get wiped?

interesting - i have thought about doing this as a scheduled task so will think that about that a bit more

thanks for the pull request - there are a few things I haven’t thought of so hoping the code is easy for people to figure out and amend as needed :smiley:

Andrei

9

there is also another way of dealing to this which is slightly cheekier but should get you the certificates

A) Create a virtualenv
B) create a new certbot instance
C) Register a new account
E) Obtain your certificate (pendings authz are on a per account basis)
F) Update the renewal file to the account name of your initial account

Andrei

10

Actually, I just got the certs renewed with slightly same manner.

$ mv /etc/letsencrypt/accounts/ /etc/letsencrypt/accounts.off/
$ # and then
$ certbot-auto register
$ certbot-auto certonly --webroot -w ... -d ...

again to get fresh certs. Don’t know if this is against the Let’s Encrypt policies but at least I got a green lock from now on and no longer pinging ACME too much.

Thanks @ahaw021 for your contribution!

11

hi @rampa

I would put the initial account back with the original ID

I believe the renewal files reference this ID and you will run in to trouble if you have other renewals which have not yet “transitioned” to the new account

When certbot starts up it looks for the account key specified in the renewal configuration file so you can either leave both key paths or update the renewal files to use your new account ID

Andrei

12

Maybe this process switched them over to the new account ID because it couldn't find the expected one? I have to check what the behavior in this case would be.

13

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.