Certificate Expiring Early

Greetings,

We’ve searched through existing content here but can’t find a situation quite like ours. Also looked at the built-in questions for a new topic, and though I’ve not included the default questions (since our setup is a bit different), I have hopefully still provided all relevant information. Thanks in advance for your assistance.

We’ve been using LE wildcard certs for a while now and had no issues. We renew the certs prior to their 90-day expiry as you would expect. We use the certs for services running in GCP and AWS with Kubernetes. Our cert renewal process is handled by a bash script that references certbot. The script is run on [current] macOS where certbot version 1.0.0_3 has been installed via Homebrew.

The cert renewal script prompts the user with a list of currently managed certs, which in our case are wildcarded zones. For example:

*.bitbrew.com
*.hub.bitbrew.com
*.kubernetes.bitbrew.com

then prompts to add additional certs (as needed), then begins issuing certs for our zones by performing a series of DNS challenges (we deploy new DNS TXT records in AWS and GCP as required), then, upon success:

— / —
Saving debug log to /Users//devops/local-cert-renewal/certs/logs/letsencrypt.log
Plugins selected: Authenticator manual, Installer None
Obtaining a new certificate
Non-standard path(s), might not work with crontab installed by your operating system package manager

IMPORTANT NOTES:

  • Congratulations! Your certificate and chain have been saved at:
    /Users//devops/local-cert-renewal/certs/live/kubernetes.bitbrew.com-0002/fullchain.pem
    Your key file has been saved at:
    /Users//devops/local-cert-renewal/certs/live/kubernetes.bitbrew.com-0002/privkey.pem
    Your cert will expire on 2020-05-12. To obtain a new or tweaked
    version of this certificate in the future, simply run certbot
    again. To non-interactively renew all of your certificates, run
    “certbot renew”

  • If you like Certbot, please consider supporting our work by:

    Donating to ISRG / Let’s Encrypt: https://letsencrypt.org/donate
    Donating to EFF: https://eff.org/donate-le

Continue updating/redeploy certificates? [y/n]
— \ —

The final steps perform some Kubernetes-related operations where, in the end, a Kubernetes manifest (kind: Secret) is built and saved to disk. This file contains the TLS certs. This file is merged to the appropriate environment branches and then released to each environment. Typically all good for the next 90 days.

The issue:

Our certs expired (without renewal for the first time) on:

Issued by: Let’s Encrypt Authority X3
Expires: Sunday, February 2, 2020 at 9:09:00 AM Eastern Standard Time

and while they were renewed right away (within one hour), still they were expired for that time interval on Feb 2. When renewing the certs, everything looked good and the certbot output indicated:

Your cert will expire on 2020-05-12

However when the certs were actually deployed, they are only valid for 15 days (i.e. Feb 2 to Feb 17 2020). The cert is valid, but the expiration date is:

Issued by: Let’s Encrypt Authority X3
Expires: Monday, February 17, 2020 at 4:47:51 PM Eastern Standard Time

This has never happened - where:

(1) Certs expired (before being renewed)
(2) The certbot output indicates the expected expiration date (i.e. 2020-05-12, a bit over 90 days), however when deployed across zones the cert is valid but the expiration date is not matching. That is, it’s February 17, 2020 (15 days from creation) rather than May 12, 2020 as certbot indicated.

The actual certbot command from the script is:

certbot certonly --agree-tos --no-eff-email -m ‘@bitbrew.com’ --manual --cert-name kubernetes.bitbrew.com --preferred-challenges dns --server https://acme-v02.api.letsencrypt.org/directory --config-dir ./certs --logs-dir ./certs/logs --work-dir ./certs “${certs[@]}”

Questions:

(1) We see (through ‘brew info’) that there is a newer version of certbot available, which is 1.1.0. Again, we are running 1.0.0_3. Do you recommend upgrading?

(2) More importantly, why are the cert expiration dates not matching? (i.e. why only good for 15 days?)

Note: If we run the script again, since (according to certbot, which thinks the cert expires on May 12, 2020), and it’s greater than 30 days prior to that date, the generated cert remains the same (unchanged).

Thanks for your assistance.

1 Like

Hi,

The problem is most likely that you have several different and overlapping certificates (indicated here by the -0002 part in the path). It's likely that the one that you renewed here is not the same certificate that you're actually using on your live site; rather, you're probably using an older similar certificate that you didn't succeed in renewing.

In this case, you should check to make sure that the scripts are in fact acting on the successfully renewed certificate rather than some other certificate. I can offer advice about this but I'm not sure about how you ended up in this situation.

I suggest running certbot certificates to see your exact current situation with the several Certbot-managed certificates on your system. You would probably want to specify the same path with --work-dir that you do when requesting the certificates.

2 Likes

Looking at some Certificate Transparency info, we can see that your cert which expires on 17 FEB 2020, was actually issued way back on 19 NOV 2019.

https://crt.sh/?q=kubernetes.bitbrew.com&iCAID=16418

So like @schoen is saying, you’re likely not deploying/delivering the certificate files you thought you were. You had one cert previously issued which apparently didn’t get utilized until later.

2 Likes

Hi @schoen,

Thanks very much for the fast response! That makes good sense. I know where to check in the script and the local file system. I’ll do that soon and follow-up.

Much appreciated.

1 Like

Thanks @ezekiel - that’s very helpful information as well. You guys rock. Will follow-up.

1 Like

No problem, excited to see the outcome! Also, warmest welcome to the community! First post, and hopefully a fairly simple solution :crossed_fingers: !

1 Like

@schoen and @ezekiel - apologies for the slow follow-up. You guys nailed it - all good now and have deployed the correct cert across zones. Fixed script to accommodate these directory changes. Many thanks!

2 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.