Certbot runs slow, takes 20 minutes to renew 1 certificate

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is:

I ran this command:
"/etc/scripts/certbot-auto certonly -a webroot --webroot-path=" . self::WEBROOT_PATH . " --keep-until-expiring --no-self-upgrade --agree-tos --force-renewal --cert-name $domain -d "$domain" 2>&1"

It produced this output:

04 Sep 2024 20:48:55.8512024-09-04 20:48:55,281:DEBUG:certbot._internal.renewal:Auto-renewal forced with --force-renewal...

04 Sep 2024 20:48:55.9672024-09-04 20:48:55,281:INFO:certbot._internal.main:Renewing an existing certificate

04 Sep 2024 20:57:42.3052024-09-04 20:57:41,995:DEBUG:certbot.crypto_util:Generating key (2048 bits): /etc/letsencrypt/keys/685540_key-certbot.pem

04 Sep 2024 21:06:40.2562024-09-04 21:06:40,150:DEBUG:certbot.crypto_util:Creating CSR: /etc/letsencrypt/csr/685536_csr-certbot.pem

The operating system my web server runs on is (include version):
Ubuntu 14.04 Trusty

My hosting provider, if applicable, is:
AWS

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):
1.9.0

We noticed the time to renew certificates is inconsistent. Sometimes it takes as quick as a few minutes and other times it can take up to 20 minutes (see the logs above as an example). The renewal forced with --force-renewal log is triggered but then after that it takes almost 10 minutes each for the steps Renewing an existing certificate and Generating key (2048 bits).

We've confirmed our EC2 instance is within health CPU utilization ranges and that we are not low on CPU usage credits.

Has anyone encountered anything similar? We renew a fairly large number of certificates and as a result renewing 90 certificates can take longer than 24 hours.

Hi @pixieset.andre.hoong, and welcome to the LE community forum :slight_smile:

Is the memory utilization high?

4 Likes

Your operating system is out of support for well over 5 years and is likely missing critical security updates. It also wouldn't surprise me if over time something broke, as no new software will test for your environment. I strongly suggest you consider an upgrade of your system and see if the problem persists in a newer environment as well.

(Unless you have Ubuntu Pro Legacy Support, but that doesn't cover all system packages)

6 Likes

You could also try another ACME client.
Like: acme.sh

5 Likes

Please use the staging server for testing.

2 Likes

Well, I've seen it a few times, here are some other cases on this forum I dug up:

Sometimes it's something like CPU credits on a burstable VM (which you seem to have ruled out), sometimes it's "security" scanning software taking up a lot of processing power while certbot is trying to create its keys and such.

It wouldn't shock me if it happens more often than it gets reported; generally certificate renewal is a background thing that happens automatically a month before expiration so whether it takes 5 seconds or 5 days doesn't really matter, and nobody would notice.

Please don't use that and waste Let's Encrypt's resources; use --dry-run to do testing against the staging environment when diagnosing issues.

I'd agree that this could be a good step to diagnose things. The shell-script based ones like acme.sh and dehydrated have the advantage of being just copy-a-file-to-deploy. I'm personally more of a fan of lego which similarly just allows you to copy a binary for your platform and run it.

Another check to try could just be without an ACME client at all, and just check performance of doing cryptographic operations, like by running openssl genrsa -out key.pem 2048.

I also want to emphasize this. If you're not getting security updates, nothing about your setup should be considered "secured" even if you're getting certificates, whether generating the keys for them is fast or slow.

7 Likes