Certbot running really slow

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: test.gosaenz.com

I ran this command: sudo /usr/local/bin/certbot certonly --webroot --agree-tos -w /etc/nginx/managed-ssl/letsencrypt -d test.gosaenz.com

It produced this output: Your system is not supported by certbot-auto anymore.
certbot-auto and its Certbot installation will no longer receive updates.
You will not receive any bug fixes including those fixing server compatibility
or security problems.
Please visit https://certbot.eff.org/ to check for other alternatives.
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Cert not yet due for renewal

You have an existing certificate that has exactly the same domains or certificate name you requested and isn't close to expiry.
(ref: /etc/letsencrypt/renewal/test.gosaenz.com.conf)

What would you like to do?


1: Keep the existing certificate for now
2: Renew & replace the cert (may be subject to CA rate limits)


Select the appropriate number [1-2] then [enter] (press 'c' to cancel): 2
Renewing an existing certificate for test.gosaenz.com
IMPORTANT NOTES:

  • Congratulations! Your certificate and chain have been saved at:
    /etc/letsencrypt/live/test.gosaenz.com/fullchain.pem
    Your key file has been saved at:
    /etc/letsencrypt/live/test.gosaenz.com/privkey.pem
    Your cert will expire on 2021-12-15. To obtain a new or tweaked
    version of this certificate in the future, simply run certbot
    again. To non-interactively renew all of your certificates, run
    "certbot renew"

  • If you like Certbot, please consider supporting our work by:

    Donating to ISRG / Let's Encrypt: Donate - Let's Encrypt
    Donating to EFF: Support EFF's Work on Let's Encrypt | Electronic Frontier Foundation

My web server is (include version): nginx/1.19.2

The operating system my web server runs on is (include version): Amazon Linux 1

My hosting provider, if applicable, is: AWS

I can login to a root shell on my machine (yes or no, or I don't know): yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): certbot 1.10.1

Issue: Recently our certbot certificate registrations have been running really, really slow. It just started happening within the last 5 days and was performing fine just prior. It gets to the line in the output: "Renewing an existing certificate for test.gosaenz.com" and it will take 2-3 hours to complete. It does eventually complete but again after a few hours. I know we need certbot and OS upgrades but surprised how it just began to crawl speed-wise so quickly. Any help is appreciated.

Hi @mikesaenz, welcome to the LE community forum :slight_smile:

That is indeed a very strange behavior.
Maybe the log file can shed some light on what is going on for so long.
Usually found at:
/var/log/letsencrypt/letsencrypt.log

And in another direction: Is the system overloaded?
Please show screen of:
top
[and press 1]

I'm attaching the portion of the letsencrypt.log for a single domain. You can see that between lines 52-54, it takes 3 hours.letsencrypt (2).log.txt (33.9 KB)

Here is


top output. Thank you so much for your help!

Quoting the relevant log lines:

2021-09-17 06:30:57,072:INFO:certbot._internal.renewal:Cert is due for renewal, auto-renewing...
2021-09-17 06:30:57,072:DEBUG:certbot.display.util:Notifying user: Renewing an existing certificate for go.ratb.li
2021-09-17 08:41:41,828:DEBUG:certbot.crypto_util:Generating RSA key (2048 bits): /etc/letsencrypt/keys/401247_key-certbot.pem
2021-09-17 09:34:54,101:DEBUG:certbot.crypto_util:Creating CSR: /etc/letsencrypt/csr/401245_csr-certbot.pem

Yikes, that's quite something. It looks like it's taking a long time to generate an RSA key. If you make a key with, like openssl genrsa -out key.pem 2048 does it also take a while? Does anything else CPU-bound take a while? You say you're on AWS, is it one of the "burstable" instances that may be low on CPU credits? Anything else change around the time that it started getting slower (maybe just serving more traffic on the server or the like)? What's that ds_am taking up your CPU in that top listing?

1 Like

One of two things may be happening:

  1. The system is indeed running slow.
    These waits are a bit high:image
    I also like doing a test like:

to confirm CPU usability.

  1. Your ACME client is now implementing a (much longer) random wait time before processing.
    [that is a very extreme longshot but worth looking into - has the ACME client updated itself recently?]

Why is lestencrypt listed as a running process now?

1 Like

Not related to your current issue, but please don't re-issue certificates (which were fine already) unnecessary. There aren't many reasons to do so. The only reasons I can think of is when some certificate options has changed, such as adding or removing the "must staple" extension or when a renewal configuration parameter has to be changed (which unfortunately currently requires re-issuing the cert even if that isn't necessary, due to lack of options in certbot to change the renewal configuration parameters in a different way.....)

2 Likes

I want to thank everyone for their prompt responses. The issue was that a software update to "ds_am" a security scanner, which was pointed out above, was scanning all files and taking a very long time to do so.

2 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.