Help me understand acme-dns


#1

A while earlier, I posted a thread asking about DNS providers with suitable APIs for DNS-01 validation, and someone mentioned acme-dns in that thread. Since then, a few other threads have mentioned it, and the idea is an intriguing one. I like that it avoids deploying a global API key that can, if compromised, do anything to any of the DNS records for any of my domains. I also like that it doesn’t leave me dependent on the API of my DNS host. But I can’t quite wrap my head around how it works, possibly because I’m not entirely clear about how DNS aliasing works for DNS-01 validation.

So let me talk through an example and hopefully fill in the gaps in my understanding. I have a domain (let’s call it example.com) for which I want to be able to obtain certs. That domain name points to a Linux server over which I have full control. I want to run acme-dns on that same server. That server is already providing public and private web/mail/LDAP/etc. services.

  • First step is to download and install acme-dns–simple enough using one of the binary releases, though I suppose I’d need to come up with some sort of init script for it(?). Configure it using the config.cfg file to handle example.com (and any other domains I want to deal with), and probably to use a non-standard HTTP port (since I already have a webserver on 80/443).
  • Register at http://example.com:port/register (how?), and save the credentials received. It looks like this step would need to be repeated for any domain/subdomain that I wanted to be able to validate–so if I wanted certs to cover www.example.com and mail.example.com, as well as example.com, I’d need three registrations.
  • Set up CNAME records of _acme-challenge.example.com to longcustomname.example.com, and repeat for each additional domain (_acme-challenge.www.example.com, etc.), with separate longcustomnamedesignations for each.
  • Do I need to do anything to point longcustomname.example.com to example.com? If not, how does Let’s Encrypt know to look to example.com for the necessary DNS records?
  • Call a suitable client, specifying the appropriate credentials and API.

I’m sure I’m missing quite a bit–where does my understanding need fixing?


Convert manual to Webroot renewal
Expired certificate on permanent tld staging / test area
Help with dns challenge and namecheap?
#2

Hi @danb35,

Yes to the download and install part ;). Regarding the init script, if your distribution uses systemd you can create the service conf, for example, create a new file /etc/systemd/system/acme-dns.service with this conf:

[Unit]
Description=acme-dns Service
After=network.target

[Service]
Type=simple
ExecStart=/usr/local/acme-dns/acme-dns

[Install]
WantedBy=multi-user.target

Note: you should change the path to acme-dns with the right one.

Edit: I forgot to say that if you use the above init script the config file for acme-dns must be in /etc/acme-dns/config.cfg

And enable it:

systemctl enable acme-dns

Well, indeed you only need to define one domain, in your case I will use acme.example.com. With this configured you will be able to use it for all your domains.

Yes, if port 80 and 443 are in use you should choose another one ;).

As example.com points to the same server you installed acme-dns that will work but I think it would be a bit clear if you use another subdomain, acme.example.com.

You can use curl to register a new entry:

$ curl -X POST http://acme.example.com:port/register

And if all is correct you will receive a response like this:

{"username":"175d7b7s-6bb6-4b66-a597-cb4de62d89av","password":"Ac23UEL1hX0ZxQarQ2adBJ08IrAZkHdccHMjt4Dk","fulldomain":"a2f7df8a-e3d6-4225-a130-5ed56a1db8f3.acme.example.com","subdomain":"a2f7df8a-e3d6-4225-a130-5ed56a1db8f3","allowfrom":[]}

Yes, you need to save the credentials and you will need to repeat the process for every domain/subdomain, in your example, yes, three registrations.

Yes, using the example registration, if you want to use that registration for example.com you will need to create a CNAME record for _acme-challenge.example.com pointing to a2f7df8a-e3d6-4225-a130-5ed56a1db8f3.acme.example.com

I’m following the example of acme.example.com so you will need to create in your dns zone for example.com a NS record for domain acme.example.com pointing to for example ns1.acme.example.com and an A or AAAA record for ns1.acme.example.com pointing to the ip of the acme-dns server.

And edit the conf file for acme-dns to be something like this:

[general]
# dns interface
listen = ":53"
# protocol, "udp", "udp4", "udp6" or "tcp", "tcp4", "tcp6"
protocol = "udp"
# domain name to serve the requests off of
domain = "acme.example.com"
# zone name server
nsname = "ns1.acme.example.com"
# admin email address, where @ is substituted with .
nsadmin = "admin.acme.example.com"
# predefined records served in addition to the TXT
records = [
    # default A
    "acme.example.com. A 203.0.113.21",
    # A
    "ns1.acme.example.com. A 203.0.113.21",
    # NS
    "acme.example.com. NS ns1.acme.example.com.",
]
# debug messages from CORS etc
debug = false

[database]
# Database engine to use, sqlite3 or postgres
engine = "sqlite3"
# Connection string, filename for sqlite3 and postgres://$username:$password@$host/$db_name for postgres
# Please note that the default Docker image uses path /var/lib/acme-dns/acme-dns.db for sqlite3
connection = "/etc/acme-dns/acme-dns.db"
# connection = "postgres://user:password@localhost/acmedns_db"

[api]
# domain name to listen requests for, mandatory if using tls = "letsencrypt"
api_domain = ""
# listen ip eg. 127.0.0.1
ip = "0.0.0.0"
# disable registration endpoint
disable_registration = false
# autocert HTTP port, eg. 80 for answering Let's Encrypt HTTP-01 challenges. Mandatory if using tls = "letsencrypt".
autocert_port = "80"
# listen port, eg. 443 for default HTTPS
port = "8080"
# possible values: "letsencrypt", "cert", "none"
tls = "none"
# only used if tls = "cert"
tls_cert_privkey = "/etc/tls/acme.example.com/privkey.pem"
tls_cert_fullchain = "/etc/tls/acme.example.com/fullchain.pem"
# CORS AllowOrigins, wildcards can be used
corsorigins = [
    "*"
]
# use HTTP header to get the client ip
use_header = false
# header name to pull the ip address / list of ip addresses from
header_name = "X-Forwarded-For"

[logconfig]
# logging level: "error", "warning", "info" or "debug"
loglevel = "debug"
# possible values: stdout, TODO file & integrations
logtype = "stdout"
# file path for logfile TODO
# logfile = "./acme-dns.log"
# format, either "json" or "text"
logformat = "text"

Let’s Encrypt will ask to the zone of your example.com domain what is the content for the TXT record for _acme-challenge.example.com, then the DNS server will say, ooops I’ve no TXT record but I have a CNAME that points to a2f7df8a-e3d6-4225-a130-5ed56a1db8f3.acme.example.com and Let’s Encrypt will follow that domain, and the example.com dns server will say to LE that the authoritative dns server for acme.example.com is ns1.acme.example.com and it will give the ip too, so LE will ask to ns1.acme.example.com what is the TXT record for a2f7df8a-e3d6-4225-a130-5ed56a1db8f3.acme.example.com and acme-dns will answer that question ;), I don’t know if I’ve explained it very well :frowning: seems a bit confusing but it isn’t :wink:

Yes, as an example, if LE gives you the token abcdef1234567890987654321abcdef123456789098 in our example you need to create the TXT record as follows:

$ curl -X POST http://acme.example.com:port/update -H "X-Api-User: 175d7b7s-6bb6-4b66-a597-cb4de62d89av" -H "X-Api-Key: Ac23UEL1hX0ZxQarQ2adBJ08IrAZkHdccHMjt4Dk" --data '{"subdomain": "a2f7df8a-e3d6-4225-a130-5ed56a1db8f3", "txt": "abcdef1234567890987654321abcdef123456789098"}'

If you are trying to reach the acme-dns API from a remote machine you should consider using https instead of http.

I hope this helps.

Cheers,
sahsanu


#3

It does, thanks–I think the big piece I was missing was the NS record for (in your example) acme.example.com.

And yes, the system concerned uses systemd, so thanks for the service file.

…but since the time I posted, I took a closer look at the docs for acme.sh, and they make it sound like a single registration could work for multiple domains/subdomains. In any case, they at least give an example of specifying one set of credentials (domain/subdomain/username/password) and obtaining a cert with more than one -d flag. But, as I think more about it, that wouldn’t make much sense. The registration (username/password) is tied to a specific subdomain (a2f7df8a-e3d6-4225-a130-5ed56a1db8f3 in your example). Your desired (sub)domain has a CNAME pointing to that subdomain. Let’s Encrypt is going to ask for _acme-challenge.domain, get redirected to a2f7df8a-e3d6-4225-a130-5ed56a1db8f3.domain, and get the desired TXT record. So if I have domain, www.domain, and mail.domain, and have their respective _acme-challenge subdomains all CNAMEd to a2f7df8a-e3d6-4225-a130-5ed56a1db8f3.domain, there would be three TXT records there when Let’s Encrypt comes calling. I know they’ll look at at least two (necessary if you want a cert for domain and *.domain), but will they go through an arbitrary number of TXT records to find a matching one?

Otherwise, the only practical way this could be used would be for either a single-domain cert, or a domain and its wildcard–it wouldn’t be practical at all for a multi-domain SAN cert.

Of course–which leads to another question: What’s the difference between tls=cert and tls=letsencrypt? Is it that in the latter, acme-dns will obtain its own cert directly from LE?


#4

i’m using acme-dns. it works fine for multiple subdomains and multiple domains. IMHO the easiest way to understand it is to just install it and go through the motions of issuing a cert. Most of your questions are answered during the setup and configuration process; the reset are answered while you create the certs.

There is an open issue in their github which is an faq on setting up the domains. (there is also a forthcoming docs PR integrating that info)

the first time you use it to set up a domain, you’ll likely have to set CNAME records on your main DNS system. LE does pull all the txt records from a DNS server.

One thing I want to note is: you probably don’t want to keep acme-dns running all the time. It’s best to run only for the duration of your challenge/validation.


#5

I don’t use acme.sh but certbot so I don’t know how acme.sh implements it but using certbot you need to create all the txt records before all of them are validated and once done, LE validates them so it won’t work with only 1 acme-dns registration, well it will work for two domains because acme-dns only allows 2 txt records per registration and this was implemented to allow a certificate covering domain.tld and *.domain.tld.

As I said above, acme-dns only allows 2 TXT records per registration.

You are right, tls=cert is to specify your own cert and using tls=letsencrypt acme-dns will try to issue a cert from Let’s Encrypt automatically. In this case I would use a self-signed cert.


#6

Hi again @danb35,

Just for the records, I’ve tested acme.sh trying to issue a certificate covering 3 domains and it doesn’t work:

$ acme.sh --issue --dns dns_acmedns -d acmedns1.domain.tld -d acmedns2.domain.tld -d acmedns3.domain.tld --test
[Sun Apr  8 20:07:54 CEST 2018] Using stage ACME_DIRECTORY: https://acme-staging.api.letsencrypt.org/directory
[Sun Apr  8 20:07:55 CEST 2018] Creating domain key
[Sun Apr  8 20:07:55 CEST 2018] The domain key is here: /root/.acme.sh/acmedns1.domain.tld/acmedns1.domain.tld.key
[Sun Apr  8 20:07:55 CEST 2018] Multi domain='DNS:acmedns1.domain.tld,DNS:acmedns2.domain.tld,DNS:acmedns3.domain.tld'
[Sun Apr  8 20:07:55 CEST 2018] Getting domain auth token for each domain
[Sun Apr  8 20:07:55 CEST 2018] Getting webroot for domain='acmedns1.domain.tld'
[Sun Apr  8 20:07:55 CEST 2018] Getting new-authz for domain='acmedns1.domain.tld'
[Sun Apr  8 20:07:56 CEST 2018] The new-authz request is ok.
[Sun Apr  8 20:07:56 CEST 2018] Getting webroot for domain='acmedns2.domain.tld'
[Sun Apr  8 20:07:56 CEST 2018] Getting new-authz for domain='acmedns2.domain.tld'
[Sun Apr  8 20:07:57 CEST 2018] The new-authz request is ok.
[Sun Apr  8 20:07:57 CEST 2018] Getting webroot for domain='acmedns3.domain.tld'
[Sun Apr  8 20:07:57 CEST 2018] Getting new-authz for domain='acmedns3.domain.tld'
[Sun Apr  8 20:07:58 CEST 2018] The new-authz request is ok.
[Sun Apr  8 20:07:58 CEST 2018] Found domain api file: /root/.acme.sh/dnsapi/dns_acmedns.sh
[Sun Apr  8 20:07:58 CEST 2018] Using acme-dns
[Sun Apr  8 20:07:58 CEST 2018] Found domain api file: /root/.acme.sh/dnsapi/dns_acmedns.sh
[Sun Apr  8 20:07:58 CEST 2018] Using acme-dns
[Sun Apr  8 20:07:59 CEST 2018] Found domain api file: /root/.acme.sh/dnsapi/dns_acmedns.sh
[Sun Apr  8 20:07:59 CEST 2018] Using acme-dns
[Sun Apr  8 20:07:59 CEST 2018] Sleep 120 seconds for the txt records to take effect
[Sun Apr  8 20:10:00 CEST 2018] Verifying:acmedns1.domain.tld
[Sun Apr  8 20:10:03 CEST 2018] Success
[Sun Apr  8 20:10:03 CEST 2018] Verifying:acmedns2.domain.tld
[Sun Apr  8 20:10:06 CEST 2018] acmedns2.domain.tld:Verify error:Incorrect TXT record
[Sun Apr  8 20:10:06 CEST 2018] Removing DNS records.
[Sun Apr  8 20:10:06 CEST 2018] Using acme-dns
[Sun Apr  8 20:10:06 CEST 2018] Using acme-dns
[Sun Apr  8 20:10:06 CEST 2018] Using acme-dns
[Sun Apr  8 20:10:06 CEST 2018] Please add '--debug' or '--log' to check more details.
[Sun Apr  8 20:10:06 CEST 2018] See: https://github.com/Neilpang/acme.sh/wiki/How-to-debug-acme.sh

It works if you use only 2 domains or if you try to issue the cert again because 2 of 3 domains were already validated… but it could be a pain to renew the cert ;).


#7

Does certbot handle this differently than acme.sh? Because given the way acme.sh handles it (with the credentials specified by environment variables), it’d be all but impossible to issue and maintain a multi-SAN cert using acme-dns–there’s no way for acme.sh to use different credentials for different domains. Which means it’d be fine for wildcards, but pretty much useless for anything else.


#8

@danb35, certbot works as acme.sh but the problem here is the hook script, I’m working on a hook script (it is a shell script) for certbot, once finished I could try to adapt it to acme.sh so won’t be a limit of domains per cert.


#9

I wrote an Certbot hook script, and a library too. They handle the things with the best UX I could come up with. That is: the hook will register an unique acme-dns account for every domain you request in the background, and will prompt you to add the CNAME records to your main DNS zone. All the acme-dns credentials are then stored on the box, and reused on renewal without any prompts whatsoever.

Take a look, the hook can be found at https://github.com/joohoi/acme-dns-certbot-joohoi/

The README.md in the repo above has finegrained instructions how to get up and running with it.


#10

Is this limitation something that’s in acme-dns, something in Boulder, or something that’s inherent in the way DNS works? If the former, it seems that the easiest answer would be to alter acme-dns to remove it. If Boulder, changing that wouldn’t be as easy, but could still be do-able. If it’s something that’s baked into DNS, well, that’s obviously harder to work around.

Aside from the issue of the hook script, it just seems that logistically it’d be a pain to keep track of which hostname belongs to which randomly-generated subdomain.

Ah, I’d missed that–I looked at the hook page, and realized that it auto-registered, but not that it created a separate registration for each domain. I assume, then, that it also keeps track of which domain belongs to which registration? That would definitely avoid a lot of the logistical pain.


#11

It’s a design decision made in acme-dns. The purpose is to promote automation and to make everything as secure and flexible as possible per default. The security standpoint here is that if one set of credentials gets compromised, it wouldn’t affect certificates of all of the domains handled on that box. The flexibility standpoint in this case would be that if one of, say ten domains handled on box A is also handled on box B, the credentials for this one box can be shared, instead of sharing the credentials used to validate all the domains on box A.

I hope that didn’t come out too complicated.

This is the trick with automation. The hook linked above takes care of all this.


#12

If you are ever going to need this information, it should be easily available from your main zone, and if you lose your credentials, it’s very easy to re-create new ones and just change the CNAME in the main zone.


#13

From that perspective, yes, it makes sense. So the real idea is that one set of credentials = one FQDN, but you allow two TXT records for the sake of wildcards.

That means that a multi-domain SAN cert may need a bunch of registrations for randomly-generated validation FQDNs, but you let the software automate that correlation, rather than making the user handle it on a recurring basis.

I think I’m getting it. It sounds like I’ll need to look back at certbot–I haven’t been a fan of it in the past, preferring the shell-based clients like dehydrated and acme.sh.


#14

Yes to everything in your last response. (I didn’t want to quote the whole comment).

This functionality is naturally achievable on any ACME client, but I personally lack the time to write support for them. With Certbot I’m familiar with, so writing a hook was a no-brainer, and while I might be biased, I’m a fan :wink:

I released the important bits of the hook handling as a Python library ( https://github.com/joohoi/pyacmedns ) for developers who wish to write support for other clients however.


#15

OK, I won’t say I thoroughly understand it, but I think I understand enough to make it work for my purposes. Thanks for all the help (not to mention the software).


#16

Awesome! I think I should compile a FAQ from questions like the ones in this thread at some point in the future to make the installation and usage clearer at some point.


#17

I have one other question, though it’s more of a side note than anything else: since acme-dns already implements a DNS server, why does it use HTTP validation to obtain its own cert? Seems it would be simpler all around if it used DNS validation.

Edit: for clarity above.


#18

This functionality makes use of awesome Go package autocert, and leveraging this widely used and battletested package results into less potential of bugs and less code to maintain.


#19

So supposing I set up acme-dns, and I expose its API to the Internet, it looks like anyone would be able to register and obtain credentials (it looks like corsorigins in config.cfg would be the way to restrict that by IP?). And, for that matter, to use my acme-dns instance to do domain validation. I’m not really intending to open up a public acme-dns server, but is there any real problem with this?


#20

Hi @danb35,

Yep… but you have a directive to disable registrations:

# disable registration endpoint
disable_registration = false

CORS is not the way to restrict it by IP because using CORS headers to allow or not the resource to one domain, generally relies on the client (for example curl or a browser) to support the restrictions imposed by the server. In this case I would use a few firewall rules to restrict access.

Cheers,
sahsanu