Deploy cert after successfull creation (with option --manual and certonly)

Issuance Tech
#1

I have a machine that I dedicated it as (I call it) "certbot server," and this certbot server has a sole task to generate letsencrypt certs and distribute them to the other web servers that need the certificates.

/snap/bin/certbot --manual certonly --preferred-challenges dns --csr /home/fathir/cethul-rpx/cethul-wildcard-01.csr --manual-auth-hook /home/fathir/auth-hook-wpmu-rpx.sh --manual-cleanup-hook /home/fathir/cleanup-hook-cethul-rpx-wildcard01.sh

The auth-hook-wpmu-rpx.sh has a task to update my DNS, and the cleanup-hook-cethul-rpx-wildcard01.sh has a job to update my web servers' certificate.

Both scripts work well when I run it in --dry-run mode; even though I must simulate the creation of 0000_cert.pem, 0000_chain.pem, and 0001_chain.pem, the following script can run nicely.

And then, when I run without --dry-run, the process stops right after certbot writing cert into the disk. My following script on manual-cleanup-hook didn't get executed like expected.

of course, if I run the script manually after the certbot is complete like this:

/snap/bin/certbot --manual certonly --preferred-challenges dns --csr /home/fathir/cethul-rpx/cethul-wildcard-01.csr --manual-auth-hook /home/fathir/auth-hook-wpmu-rpx.sh ; bash /home/fathir/cleanup-hook-cethul-rpx-wildcard01.sh

These syntaxes fulfilled my intention about how the "certbot server" works as a certs generator.

My question here is, "why the --manual-cleanup-hook not working. What have I done wrong?"

additional information:

my domain is ugm.ac.id https://crt.sh/?q=*.ugm.ac.id&exclude=expired&match=LIKE

my web servers are apache and nginx

My web servers' OS was mostly Linux, varying between Ubuntu, Debian, and Centos, with variety in its release version.

I have root-level access on each server I manage. I distribute the certs via scp.

The certbot version I use is 1.19.0

1 Like
#2

Good question! Perhaps the log file in /var/log/letsencrypt.log can shed some light on the situation?

#3

No clue.
The log also says the process stop right away.

I searched the word "manual-cleanup-hook," only mentioned at the beginning of the log.

#4

Could you perhaps share the log file? It doesn't contain important things like private keys that shouldn't be shared with the world. Just worthless public keys.

#5

Try changing the order of things...

Maybe as:

/snap/bin/certbot --manual certonly --preferred-challenges dns \
--csr /home/fathir/cethul-rpx/cethul-wildcard-01.csr \
--manual-cleanup-hook /home/fathir/cleanup-hook-cethul-rpx-wildcard01.sh \
--manual-auth-hook /home/fathir/auth-hook-wpmu-rpx.sh
#6

Also, please show the first few lines of each of those scripts.

#7

this is log without --dry-run option
letsencrypt.log.24.txt (296.0 KB)

this is log with --dry-run option. even though the script wrote in the log different from what I post above, but it gives similar output.
letsencrypt.log.9.txt (42.8 KB)

#8

Ah wait a second.. Is it possible the authentication hook ALSO doesn't run for the production endpoint? As it seems all hostnames are already valid to begin with.. So certbot doesn't need to add nor remove DNS entries..

--dry-run invalidates already valid authorizations so it can test an authorization properly, so behaves differently than a non-dry-run run.

1 Like
#9

this is some of line of the auth-hook-wpmu-rpx.sh script. (not actual full script)

CERTBOT_TOKEN_EXISTING=`grep acme-challenge.$CERTBOT_DOMAIN /home/fathir/git-mode/konfig-ns1/eksternal/zone/ugm-zone | gawk '{print $5}' | gawk -F\" '{print $2}'`
echo token existing = $CERTBOT_TOKEN_EXISTING
echo token new = $CERTBOT_VALIDATION

sed -i 's/'$CERTBOT_TOKEN_EXISTING'/'$CERTBOT_VALIDATION'/g'  /home/fathir/git-mode/konfig-ns1/eksternal/zone/ugm-zone`

scp -P 222 /home/fathir/git-mode/konfig-ns1/eksternal/zone/ugm-zone certbotclient@172.16.30.3:/var/tmp
ssh -l certbotclient 172.16.30.3 -p 222 -t "\
        sudo mv -v /var/tmp/ugm-zone /var/named/ugm-zone; \
        sudo /usr/sbin/rndc reload; \
        echo \"sudah\"
        "

this is for cleanup-hook-cethul-rpx-wildcard01.sh (not actual script also)

CERTNAME="cethul-wildcard-01"
SAVEDIR="/home/fathir/cethul-rpx/"

if [ "$CERTBOT_REMAINING_CHALLENGES" -eq "0" ]; then
        mv -v 0000_cert.pem $SAVEDIR/$CERTNAME\_cert.pem
        mv -v 0000_chain.pem $SAVEDIR/$CERTNAME\_chain.pem
        mv -v 0001_chain.pem $SAVEDIR/$CERTNAME\_fullchain.pem

        scp $SAVEDIR/$CERTNAME\_fullchain.pem certbotclient@10.13.245.252:/tmp
        ssh -l certbotclient 10.13.245.252 -t "\
                sudo mv -v /tmp/$CERTNAME\_fullchain.pem /etc/nginx/cert/cethul-letsencrypt-wildcard-part01.pem; \
                sudo systemctl reload nginx.service; \
                echo \"sudah\"
                "
#10

I don't think so.

it seems, once the hostname has already been validated by letsencrypt, it needs to revalidate when I run the certbot again.

I also simulate how I need to do to renew my certificate (that's by executing my whole certbot syntax once again), and all the hostname need to revalidate again.

#11

Why do you have to specify bash ?

#12

Oh, It's my bad habit.

I often didn't make my bash script executable with chmod.

I know it's not a good practice, I just getting used to it.

#13

But your scripts also don't include:

#!/bin/bash

#14

Like what I've said before.

I post not a complete scripts, just a important part that do the job.

#15

I specifically asked for:

Not sure what you showed nor how that relates to my request.

#16

okay, these are a few first lines from both script

fathir@ubuntu-certbot-research:~$ head cleanup-hook-cethul-rpx-wildcard01.sh
#!/bin/bash
#echo CERTBOT_DOMAIN = $CERTBOT_DOMAIN
#echo CERTBOT_VALIDATION = $CERTBOT_VALIDATION
#echo CERTBOT_TOKEN = $CERTBOT_TOKEN
#echo CERTBOT_REMAINING_CHALLENGES = $CERTBOT_REMAINING_CHALLENGES
#echo CERTBOT_ALL_DOMAINS = $CERTBOT_ALL_DOMAINS
#echo CERTBOT_AUTH_OUTPUT = $CERTBOT_AUTH_OUTPUT
#CERTBOT_REMAINING_CHALLENGES="0"
CERTNAME="cethul-wildcard-01"
SAVEDIR="/home/fathir/cethul-rpx/"

fathir@ubuntu-certbot-research:~$ head auth-hook-wpmu-rpx.sh
#!/bin/bash
#echo CERTBOT_DOMAIN = $CERTBOT_DOMAIN
#echo CERTBOT_VALIDATION = $CERTBOT_VALIDATION
#echo CERTBOT_TOKEN = $CERTBOT_TOKEN
#echo CERTBOT_REMAINING_CHALLENGES = $CERTBOT_REMAINING_CHALLENGES
#echo CERTBOT_ALL_DOMAINS = $CERTBOT_ALL_DOMAINS
#echo CERTBOT_AUTH_OUTPUT = $CERTBOT_AUTH_OUTPUT

START_DIR=`pwd`
GIT_DIR="/home/fathir/git-mode/konfig-ns1"

I don't know if these lines can tell enough about what the scripts do

#17

OK all that looks in order.
I'm leaning towards something in the when/way those hooks are (supposed to be) processed.

A rare case of: If then else notif then try else return loop/knot/underflow/bug/feature!
LOL

#18

So... it's like some bugs from certbot regarding the hooks feature?

#19

Maybe.
Someone needs to lab this - not me - I'm bushed

#20

This part could also be reordered (and edited slightly):

As:
/snap/bin/certbot certonly --manual --preferred-challenges=dns

[this will likely make 0% difference - but who knows... sometimes the littlest of things add up]