Automating renewal by DNS-01 Options

Hi,

I use DNS-01 auth for certbot renewal. I can't use the other methods requiring FTP service, as I don't wish to set it up on the GCP server.

From what I have read, the cert created with "--manual" cannot auto-renew b/c; certbot issues a new challenge for each renewal, then expects to find that challenge in the TXT record of the (sub) domain. The only solution I have found is to run the subdomain NS (Name Server) on my localhost where TXT records can be created and purged via script. This of course requires separate NS assignment for the acme subdomain.

MY QUESTION:
Is my above conclusion correct, or are there other/better ways to handle DNS-01 challenges? Should I have not created the initial certs with "--manual", perhaps I wold not run into this issue but I wanted a wildcard cert so this is probably the only way?

Thank you for your input on this question.
Regards

THE BASICS:
My domain: .aerv.us, aerv.us
Cert Creation Command: # certbot certonly --manual --preferred-challenges dns-01 --server https://acme-v02.api.letsencrypt.org/directory -d "
.aerv.us" -d aerv.us
I ran this command: # certbot renew
It produced this output: ... unexpected error: Failed authorization procedure. aerv.us (dns-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: No TXT record found at _acme-challenge.aerv.us, aerv.us (dns-01)...
web server: Lighttpd
OS: Debian Buster 4.19.208-1
Hosting: GCP
root shell: YES
control panel: NO
certbot --version: 0.31.0

1 Like

Yes, you pretty much shouldn't ever use --manual except when testing something.

Ideally, there's a plugin for certbot for your DNS provider. This has certbot use your DNS provider's API to create the TXT record.

If your DNS provider doesn't have an API, or your security policy doesn't want to allow for your server to have DNS API credentials (as depending on the DNS provider, you may only be able to have API credentials with full access to edit the DNS zone rather than just the TXT record), then you probably want to look at a tool like acme-dns or agnos, which act as a special-purpose DNS server just for responding with the right TXT record.

5 Likes

It's possible to do what you want with your existing Geoscaling provider, using smart subdomains.

  1. Add a "Smart subdomain" for the _acme-challenge. subdomain. (I'll use _acme-challenge.geoscaling.letsdebug.net. in the example here). Set it up like this, making sure to have the "extra info" checkbox selected:

    The code is:

    if (!isset($extra_info))
    {
      $output[] = array("TXT", "No ACME challenge records were configured.");
      return;
    }
    
    $records = explode(" ", $extra_info);
    foreach ($records as $record)
    {
      $output[] = array("TXT", $record);
    }
    
  2. Next, you need the Certbot hook which will send the ACME challenges to the smart subdomains functionality. Here it is as a Python 3 script (make sure to change credentials):

    #!/usr/bin/env python3
    import json
    import sys
    import xmlrpc.client
    from os import getenv
    from time import sleep
    
    GEOSCALING_USERNAME = "USER_HERE"
    GEOSCALING_PASSWORD = "PASSWORD_HERE"
    
    #######
    
    STATE_FILE = "/tmp/geoscaling-acme-challenges.json"
    
    if len(sys.argv) != 2:
        print('Usage: update_acme.py [add/delete]')
        exit(1)
    
    act = sys.argv[1]
    payload = getenv("CERTBOT_VALIDATION")
    
    # Since we can't readily query the existing records from the API, we'll just
    # keep the list of records we're managing in a local temporary file.
    all_records = []
    try:
        with open(STATE_FILE, 'r') as f:
            all_records = set(json.loads(f.read()))
    except:
        # File might not exist or be garbage. Whatever, ignore.
        pass
    
    # Make the modifications we need
    if act == "add":
        all_records.add(payload)
    elif act == "delete":
        all_records.remove(payload)
    
    # Save the records to the temporary file again.
    with open(STATE_FILE, 'w+') as f:
        f.write(json.dumps(list(all_records)))
    
    # Update the geoscaling extra info with the full list of records to render
    with xmlrpc.client.ServerProxy("http://api.geoscaling.com/dns2/xml-rpc/") as proxy:
        assert proxy.geoscaling.extra_info(GEOSCALING_USERNAME, 
                                           GEOSCALING_PASSWORD,
                                           " ".join(all_records)) == 1
        
    if getenv("CERTBOT_REMAINING_CHALLENGES") == "0" and act == "add":
        sleep(15)
    
    
  3. Now, try use it with Certbot:

    $ sudo certbot certonly -d "geoscaling.letsdebug.net" -d "*.geoscaling.letsdebug.net" \
    --manual --manual-auth-hook "/etc/letsencrypt/geoscaling.py add" \
    --manual-cleanup-hook "/etc/letsencrypt/geoscaling.py delete" \
    --preferred-challenges dns --dry-run
    
    Saving debug log to /var/log/letsencrypt/letsencrypt.log
    Simulating a certificate request for geoscaling.letsdebug.net and *.geoscaling.letsdebug.net
    The dry run was successful.
    

Yay.

4 Likes

Thank you for that great answer. If you don't mind, could you clarify a few points for me?

#1

you probably want to look at a tool like acme-dns or agnos

These solutions require the real NS to point the subdomain (_acme-challenge.example.com) records to localhost, do they not? If that's the case, my idea of using a local NS (Unbound NSD) for the subdomain would be on par?

#2

you pretty much shouldn't ever use --manual except when testing something

I most likely missed it, but is there any other way to issue wildcard cert with DNS-01?

#3

Ideally, there's a plugin for certbot for your DNS provider.

Turns out, Geoscaling does indeed have DNS API, but as you stated I would prefer not to expose the entire DNS account management to a internet exposed server.

Thank you

2 Likes

Dude, that's just awesome!
Geoscaling rocks btw...

Thank you for this, great stuff /

2 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.