urn:acme:error:rateLimited when renewing

Help
1

So I’ve had no valid certificate for my domains since January 24th and, while my server is in by no means critical to anything important, I’d like to resolve the issue that I’m getting. I’ve looked around these forums for various possible ways to find a fix, but other people might have different causes for the issue so I’m just going to go ahead and post a new thread - I apologize if you’ve seen this a thousand times already!

My domain is:
evie.codes ; omnic.xyz ; evie-banned.me

I ran this command:
letsencrypt renew

It produced this output:

My web server is (include version):
nginx/1.10.3 (Ubuntu)

The operating system my web server runs on is (include version):
Ubuntu 16.04.2 LTS

My hosting provider, if applicable, is:
ovh.com

I can login to a root shell on my machine: yes

I’m using a control panel to manage my site: no

Further details:
https://crt.sh/?q=evie.codes indicates that I have multiple valid certificates, but for some reason, they aren’t being applied to my server.
I’ve deleted /etc/letsencrypt/renewal/evie.codes-0001.conf since it’s invalid and points to nonexistant files (it was skipped anyway)
I’ve verified that there are no extra cron jobs running on the server, only the default certbot one:

SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

0 */12 * * * root test -x /usr/bin/certbot -a \! -d /run/systemd/system && perl -e 'sleep int(rand(3600))' && certbot -q renew

Is there extra checks I can do, or things I can do to fix this?

2

Hi @evie.codes,

Could you show the output of certbot certificates and ls -l /etc/letsencrypt/live/*?

Also, do you have log files in /var/log/letsencrypt with creation dates and times that correspond to the issuance times that are shown on crt.sh when you search for your certificates? (Bear in mind that the crt.sh times are shown in UTC.)

It looks like something on your system is very frequently and successfully getting certificates for your site—not positive yet whether that’s Certbot or not. :slight_smile:

3

Hi Shoen, thanks for the response.

The /var/log/letsencrypt folder has a lot of logs - 444 of them to be precise so if I understand this correctly, It’s definitely the letsencrypt cron job that’s doing this. As far as I’ve been able to tell looking at a few of these logs (which look almost identical to the ones in my gist above) they match with the certificates I see on crt.sh.

certbot certificates shows the output that confirmed my last valid certificate expired on the 24th of January:

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Revocation status for /etc/letsencrypt/live/evie.codes/cert.pem is unknown

-------------------------------------------------------------------------------
Found the following certs:
  Certificate Name: evie.codes
    Domains: evie.codes c9.evie.codes db.evie.codes discord.evie.codes evie-banned.me guardian.evie.codes how.evie-banned.me omnic.xyz omnicbot.evie.codes proof.evie-banned.me www.evie.codes www.omnic.xyz
    Expiry Date: 2018-01-24 18:01:28+00:00 (INVALID: EXPIRED)
    Certificate Path: /etc/letsencrypt/live/evie.codes/fullchain.pem
    Private Key Path: /etc/letsencrypt/live/evie.codes/privkey.pem
------------------------------------------------------------------------------

As for ls -l /etc/letsencrypt/live/* … well this is certainly interesting. something definitely messed up in the system (which is probably my fault come to think of it - I tried to “keep” my certificates when I reformated my VPS a few months ago and most likely confused the scripts)…

lrwxrwxrwx 1 root root  39 Feb 14 00:01 cert.pem -> ../../archive/evie.codes-0001/cert1.pem
lrwxrwxrwx 1 root root  40 Feb 14 00:01 chain.pem -> ../../archive/evie.codes-0001/chain1.pem
lrwxrwxrwx 1 root root  44 Feb 14 00:01 fullchain.pem -> ../../archive/evie.codes-0001/fullchain1.pem
lrwxrwxrwx 1 root root  42 Feb 14 00:01 privkey.pem -> ../../archive/evie.codes-0001/privkey1.pem
-rw-r--r-- 1 root root 543 Oct 26 15:01 README

So it… looks like maybe it’s pointing to -0001 but my renewal is not on that “instance”? Something like that?

4

Yes, probably.

Was that an ls of /etc/letsencrypt/live/evie.codes/ or a different directory?

5

Is it possible that you renamed one of the directories in /etc/letsencrypt/live previously?

(If so, I think that doing this may have confused the renewal-detection logic, causing renewals to happen every time even though the certificates are not really due for renewal.)

6

That was exactly ls -l /etc/letsenctryp/live/*. Doing ls -l /etc/letsencrypt/live/evie.codes gives me exactly the same result.

Wait those are symlinks right? Meaning, if I pointed to ../../archive/evie.codes/cert1.pem and such, it… could restore from the last certificate?

And yes. I probably did muck around with the files not knowing exactly what I was doing.

7

No, not cert1. Here’s ls -l /etc/letsencrypt/archive/evie.codes/

root@vps83813:/etc/letsencrypt/archive/evie.codes# ls -l
total 32
-rw-r--r-- 1 root root 2025 Jul 17  2017 cert1.pem
-rw-r--r-- 1 root root 2045 Feb 14 00:01 cert2.pem
-rw-r--r-- 1 root root 1647 Jul 17  2017 chain1.pem
-rw-r--r-- 1 root root 1647 Feb 14 00:01 chain2.pem
-rw-r--r-- 1 root root 3672 Jul 17  2017 fullchain1.pem
-rw-r--r-- 1 root root 3692 Feb 14 00:01 fullchain2.pem
-rw-r--r-- 1 root root 1704 Jul 17  2017 privkey1.pem
-rw-r--r-- 1 root root 1704 Feb 14 00:01 privkey2.pem

looks like it’s 2 in here that’s being generated.

8

That’s right. xxx1.pem is the oldest file.

Certbot saves new files as xxx2.pem, xxx3.pem, etc., and then updates the symlinks, when it’s renewed or replaced.

If you replace the symlinks with “../../archive/evie.codes/cert2.pem” and so forth, everything should work.

Edit: Typo.

9

Also be sure that you don’t have links pointing from live/evie.codes into ../../archive/evie.codes-0001 or vice versa, and that the contents of renewal/evie.codes.conf consistently refer to live/evie.codes and not live/evie.codes-0001.

I think we still have some bugs where we don’t fail with a useful error if the referential integrity of /etc/letsencrypt contents gets messed up in certain specific ways, because effectively Certbot may not be checking that all of the assumptions about directory structure that it’s making really hold true.

1 Like
10

Ok well I might have a good cert in a few minutes! When I try to run letsencrypt renew now I get this:

root@vps83813:/etc/letsencrypt/live/evie.codes# letsencrypt renew
Saving debug log to /var/log/letsencrypt/letsencrypt.log

-------------------------------------------------------------------------------
Processing /etc/letsencrypt/renewal/evie.codes.conf
-------------------------------------------------------------------------------
Cert not yet due for renewal

The following certs are not due for renewal yet:
  /etc/letsencrypt/live/evie.codes/fullchain.pem (skipped)
No renewals were attempted.

I’m guessing I just need to reboot nginx so it sees this cert through the new symlinks?

11

And it did! Everything’s up and running now, thank you very much for the assistance, @schoen and @mnordhoff !

1 Like
12

Yes, that should probably work fine.

Edit: Great! Glad it’s worked out.

1 Like
13

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.