How to automate ssl certificate generation with certbot

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. https://crt.sh/?q=example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: "example1.com" "example2.com" "example3.com"

I ran this command:

#!/bin/bash 
DEPLOYMENT_BUCKET=~/certbot_bucket  ## here's my workspace
DOMAINS=("example1.com" "example2.com" "example3.com")  ## domain my ssl cert creates for 
PROJECTS=("example-demo") ## my projects

## 90 days letsencrypt cert will expiry
# now i get the expiry date and delete folde with name of expiry date and create new folder with expiry date, where my ssl cert can be placed at. 
EXPIRATION_DATE=$(date -v +90d "+%Y-%m-%d")
rm -rf $DEPLOYMENT_BUCKET/certbot/$EXPIRATION_DATE
mkdir -p $DEPLOYMENT_BUCKET/certbot/$EXPIRATION_DATE
CERT_DIR="$DEPLOYMENT_BUCKET/certbot/$EXPIRATION_DATE"
echo $EXPIRATION_DATE

## now i'm looping through domains, to create ssl cert one by one
for domain in "${DOMAINS[@]}"
do
 # now i use certbot creating my ssl cert. I want to automate the creation process and i just need to get the ssl cert. I will upload ssl certs to google cloud later
 echo certbot certonly --manual --work-dir=$CERT_DIR --logs-dir=$CERT_DIR --config-dir=$CERT_DIR --agree-tos --email "xxx@example.com" --manual-auth-hook $DEPLOYMENT_BUCKET/certbot/authenticator.sh --manual-cleanup-hook $DEPLOYMENT_BUCKET/certbot/cleanup.sh -d *.$domain --noninteractive

 certbot certonly --manual --work-dir=$CERT_DIR --logs-dir=$CERT_DIR --config-dir=$CERT_DIR  --agree-tos --email "xxx@example.com"  --manual-auth-hook $DEPLOYMENT_BUCKET/certbot/authenticator.sh --manual-cleanup-hook $DEPLOYMENT_BUCKET/certbot/cleanup.sh -d *.$domain --noninteractive
 for project in "${PROJECTS[@]}"

  # DO project stuff here. Here i upload ssl certs to google cloud projects. 
 do
 done
done

It produced this output:

Saving debug log to ~/certbot/2021-03-08/letsencrypt.log
Plugins selected: Authenticator manual, Installer None
Account registered.
Requesting a certificate for *.example1.com
Performing the following challenges:
dns-01 challenge for example.com
Running manual-auth-hook command: ~/certbot/authenticator.sh
Output from manual-auth-hook command authenticator.sh:

_acme-challenge.example1.com
vPOJ1XoDb2NSZujsEvr7vy_076D1Bmb8Q9aGFfs49-g


Error output from manual-auth-hook command authenticator.sh:
expr: syntax error
Traceback (most recent call last):
  File "<string>", line 1, in <module>
IndexError: list index out of range
Traceback (most recent call last):
  File "<string>", line 1, in <module>
TypeError: 'NoneType' object has no attribute '__getitem__'

Waiting for verification...
Challenge failed for domain example1.com
dns-01 challenge for example1.com
Cleaning up challenges
Running manual-cleanup-hook command: ~/certbot/cleanup.sh
Some challenges have failed.

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: example1.com
   Type:   dns
   Detail: DNS problem: NXDOMAIN looking up TXT for
   _acme-challenge.example1.com - check that a DNS record exists
   for this domain

My web server is (include version):
i just use certbot to generate ca recognized ssl certificate. It's not attached to web server yet. I only plan to create the ssl certificate locally. My local computer is MacOS. I think currently it has nothing to do with my web server.

The operating system my web server runs on is (include version): it doesn't matter

My hosting provider, if applicable, is: No

I can login to a root shell on my machine (yes or no, or I don't know): I just ran it locally to get ssl certs

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): No

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): i tried with certbot 1.10.1 and certbot 1.8.0. Both gave me same error.

What's with the whole bucket script thingy? Where does that come from? What's the exact reason for usage?

to help you understand i update the question and script. @Osiris
You can see more explanation behind "##" in above script.

What i'm trying to do is automate the ssl cert creation process and i only need to generate the ssl certs. I will upload the ssl certs to google cloud later

I can read and understand the script, I just don't understand the why if the script. It looks like it does functions which certbot itself can do perfectly?

In any case, it looks like your authenticator script is malfunctioning.

@Osiris
I'm new to the tool cerbot. Maybe there's some advanced functions certbot processes and i'm not familiar with.
You said some malfunctioning in "authenticator script".
Here is my authenticator script.
I need to use cloudflare dns for verification. And the scripts are copied from https://certbot.eff.org/docs/using.html

authenticator.sh

#!/bin/bash

# Get your API key from https://www.cloudflare.com/a/account/my-account
API_KEY="xxx" # TODO Pleas update to your cloudflare api key 
EMAIL="xxx" # TODO Pleas update to your cloudflare email

# Strip only the top domain to get the zone id
DOMAIN=$(expr match "$CERTBOT_DOMAIN" '.*\.\(.*\..*\)')

# Get the Cloudflare zone id
ZONE_EXTRA_PARAMS="status=active&page=1&per_page=20&order=status&direction=desc&match=all"
ZONE_ID=$(curl -s -X GET "https://api.cloudflare.com/client/v4/zones?name=$DOMAIN&$ZONE_EXTRA_PARAMS" \
     -H     "X-Auth-Email: $EMAIL" \
     -H     "X-Auth-Key: $API_KEY" \
     -H     "Content-Type: application/json" | python -c "import sys,json;print(json.load(sys.stdin)['result'][0]['id'])")

# Create TXT record
CREATE_DOMAIN="_acme-challenge.$CERTBOT_DOMAIN"
echo $CREATE_DOMAIN
RECORD_ID=$(curl -s -X POST "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/dns_records" \
     -H     "X-Auth-Email: $EMAIL" \
     -H     "X-Auth-Key: $API_KEY" \
     -H     "Content-Type: application/json" \
     --data '{"type":"TXT","name":"'"$CREATE_DOMAIN"'","content":"'"$CERTBOT_VALIDATION"'","ttl":120}' \
             | python -c "import sys,json;print(json.load(sys.stdin)['result']['id'])")

# Save info for cleanup
if [ ! -d /tmp/CERTBOT_$CERTBOT_DOMAIN ];then
        mkdir -m 0700 /tmp/CERTBOT_$CERTBOT_DOMAIN
fi
echo $ZONE_ID > /tmp/CERTBOT_$CERTBOT_DOMAIN/ZONE_ID
echo $RECORD_ID > /tmp/CERTBOT_$CERTBOT_DOMAIN/RECORD_ID

# Sleep to make sure the change has time to propagate over to DNS
sleep 25

cleanup.sh

#!/bin/bash

# Get your API key from https://www.cloudflare.com/a/account/my-account
API_KEY="xxx" # TODO Pleas update to your cloudflare api key
EMAIL="xxx" # TODO Pleas update to your cloudflare email

if [ -f /tmp/CERTBOT_$CERTBOT_DOMAIN/ZONE_ID ]; then
        ZONE_ID=$(cat /tmp/CERTBOT_$CERTBOT_DOMAIN/ZONE_ID)
        rm -f /tmp/CERTBOT_$CERTBOT_DOMAIN/ZONE_ID
fi

if [ -f /tmp/CERTBOT_$CERTBOT_DOMAIN/RECORD_ID ]; then
        RECORD_ID=$(cat /tmp/CERTBOT_$CERTBOT_DOMAIN/RECORD_ID)
        rm -f /tmp/CERTBOT_$CERTBOT_DOMAIN/RECORD_ID
fi

# Remove the challenge TXT record from the zone
if [ -n "${ZONE_ID}" ]; then
    if [ -n "${RECORD_ID}" ]; then
        curl -s -X DELETE "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/dns_records/$RECORD_ID" \
                -H "X-Auth-Email: $EMAIL" \
                -H "X-Auth-Key: $API_KEY" \
                -H "Content-Type: application/json"
    fi
fi

@Osiris error always complains "check that a DNS record exists for this domain". In authenticator script, i already provided with cloudflare api token and email, why it can't add TXT record itself and do verification ? My goal is to fully automate the ssl cert creation.

  • Don't use those example, scripts, it is clearly stated in the documentation:

Example usage for DNS-01 (Cloudflare API v4) (for example purposes only, do not use as-is)

  • Use the certbot-dns-cloudflare plugin to use the dns-01 challenge if you require it (wildcard certificate, no access on port 80 on your server or certbot is not running on the server)
  • Use the http-01 challenge if you don't require a wildcard certificate, port 80 is functioning and certbot is running on the server (which isn't the case in your situation I understand)

All the things from your script can be added to the standard certbot functionality. Certbot is by default setup that it will renew when there are 30 or less days left until expiry and the "project stuff" can be put in a script which can run as a --deploy-hook. See the certbot documentation for more info about that hook.

2 Likes

Thanks it works @Osiris

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.