I agree. It certainly would solve all my objections to how LE exists at the moment…
Do you have a link to your proposal again? (unsure if you posted it earlier?)
Lets see if wen can further it along?
If you’re referring to the “just use account key” proposal, that would currently be blocked by the Baseline Requirements (assuming Ballot 169 ever escapes the IPR hell it’s currently caught up in), as any domain ownership challenge needs to contain either a Random Value with limited validity (max. 30 days), or a Request Token that’s either limited in validity or limited to one issuance. (Exact definition can be found in the Ballot 169 link above.)
That’s a bit sad, but then again couldnt the account key (with maybe some extra stuff at the first setup for verification) act as Domain Authorization Document as listed in 3.2.2.4.5 of this ballot?
well to give it more exactly the account key (which is in the hands of the domain owner signs) that document and the key is used to verify this document (because it has to be on or after the request).
Domain Authorization Documents are basically documents (like, an actual piece of paper) signed by the domain owner (as listed in e.g. WHOIS). See this form from GeoTrust as an example. Not really something you can do with an automated process.
too bad, would have been epic if there would be a way to add a long-term auth which can be challenged (like the account key, which is challenged anyway)
but at the very least it would be epic if walking up a DNS-based auth would be possible, because that REALLY helps with issuing for multiple domains, especially if you are either doing it yourself (for whatever reason) or if the DNS Limits the number of changes per time to avoid abuse or whatever (isnt really impossible, LE’s Issuance is rate-limited as well)
Maybe finally this proposal can die. I couldn’t point my finger on it, but I knew that challenge and response is challenge and response for a reason and you don’t do it some other way just because it’s easier to use.
I also think the word epic is overused. 
I would be happy to use a random challenge string via http.
My proposal would be to be able to get a validation string for the challenge, upload it to http://mydomain.com/wherever (as required by the spec), then use that to issue a cert for, say, mail.mydomain.com
The only difference being is that mail.mydomain.com no longer needs to run a web server to host the auth, nor do I have to make any changes to DNS (because this almost has nothing running other than bind).
I don’t care how many times I have to change the validation file to get certs - but it would fix every problem with validation in every circumstance I have.
Already doable. If you create the hostname, you can also create the challenge CNAME at the same time. So at one point you have to make a change once anway. Delegate the challenge to your dedicated host and the setup is identical to your proposal, without weakening the process to an unknown extent.
Not sure I follow your proposal. If you spell it out, I’ll give you feedback / try it…
With DNS-01, if you want a cert for host.example.com, the challenge requires the record _acme-challenge.host.example.com of type TXT. You can alias that record as a CNAME without affecting host.example.com’s IP assignment.
If you create a host acmehost.example.com, you can delegate acme.example.com to it and alias _acme-challenge.host.example.com to host.acme.example.com.
So
host.example.com. IN A 1.2.3.4
acmehost.example.com. IN A 5.6.7.8
acme.example.com. IN NS acmehost.example.com.
_acme-challenge.host.example.com. IN CNAME host.acme.example.com.
is perfectly doable. On 5.6.7.8 you run an ACME client and a DNS server that serves acme.example.com. You’ve just delegated away the challenge and cert handling to a single confined host that does nothing else and no other host has to deal with ACME stuff. All it has to do is fetch the cert from 5.6.7.8 via SFTP or something like that.
2 Likes
Ok, so after experimenting… It seems certbot (still?) doesn’t support the dns-01 tests. I’ve hunted through about 30 or so ‘tutorials’ - and none seem to use dns-01 at all.
Running certbot in interactive mode doesn’t seem to show anything to do with dns-01 as a method at all - which leaves me back at “doesn’t work as expected” state.
Is there some random magic that needs to be done to attempt this?
1 Like
Hello @CRCinAU,
Since certbot 0.9.0, it supports dns challenge but only in manual mode, so you need to specify the manual mode and the challenge dns:
--manual --preferred-challenges dns
Edit: Adding an example.
An example for domain test.domain.tld could be (it will use staging server, it will not register any email address and agrees with the public ip logging policy):
certbot certonly --manual --preferred-challenges dns -d test.domain.tld --text --register-unsafely-without-email --manual-public-ip-logging-ok --staging
And you will get an output like this:
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Starting new HTTPS connection (1): acme-staging.api.letsencrypt.org
Obtaining a new certificate
Performing the following challenges:
dns-01 challenge for test.domain.tld
Please deploy a DNS TXT record under the name
_acme-challenge.test.domain.tld with the following value:
cRs4BldVlt07-JTDy1TWxI5Bvkxq0AyldMHg4qFf0fs
Once this is deployed,
Press ENTER to continue
At this point, you need to manually create a TXT record for _acme-challenge.test.domain.tld on your DNS server containing the challenge, in this example cRs4BldVlt07-JTDy1TWxI5Bvkxq0AyldMHg4qFf0fs
Once you have check that your dns server answer with the modified record…
$ dig _acme-challenge.test.domain.tld txt +short
"cRs4BldVlt07-JTDy1TWxI5Bvkxq0AyldMHg4qFf0fs"
…you can press ENTER on certbot screen and if all is ok, you will get your certificate:
Self verification requires optional dependency `dnspython` to be installed.
Waiting for verification...
Cleaning up challenges
Generating key (2048 bits): /etc/letsencrypt/keys/0070_key-certbot.pem
Creating CSR: /etc/letsencrypt/csr/0070_csr-certbot.pem
IMPORTANT NOTES:
- Congratulations! Your certificate and chain have been saved at
/etc/letsencrypt/live/test.domain.tld/fullchain.pem. Your cert will
expire on 2017-02-24. To obtain a new or tweaked version of this
certificate in the future, simply run certbot-auto again. To
non-interactively renew *all* of your certificates, run
"certbot-auto renew"
Cheers,
sahsanu
1 Like
So, I’d like to say that I’m kind of happy with the solution I’ve come up with. It’s a little kludgy - but it works.
I couldn’t get anywhere with the official certbot - so if you want to use that to get some kind of central management, forget trying to use certbot.
I ended up with a solution using ‘dehydrated’ - https://github.com/lukas2511/dehydrated
I configured up bind as per the suggestion from TCM - and have a ‘bastion’ host that runs a chroot copy of named. Using a key, I can use nsupdate to insert / remove DNS entries used for the challenge.
The nsupdate triggering is done via a hook. The initial hook example for nsupdate didn’t work - and a lot was missing. I updated the wiki page on the nsupdate hook to an actual working version:
It may be more workable to rework this to suit your environment.
I had to then create CNAME’s in my main zone to work all the _acme-challenge.$host responses into the namespace of the bastion host.
I then modified the hook script somewhat further to utilise the “deploy_cert” functionality to call a second script to scp the cert, key and intermediate pem files to a remote host, then run a command to reload the services using those scripts.
Its a very bloody ugly setup - and probably not for the faint hearted - but it does actually seem to work.
I’ve moved a few sites across to LE certs now to test the functionality - and haven’t encountered many problems so far. I’d say this level of work would be well beyond the majority of the Lets Encrypt audience.
3 Likes
One of the nice (and occasionally terrifying) things about computers is that only one person needs to figure out how to do it. A very large number of Let's Encrypt's subscribers today have no idea how to run a Python program, but it's OK because their web host added a clicky "Turn on free SSL" checkbox in the web control panel they use and they clicked it. In fact with things like cPanel, even the web host doesn't need to understand X.509 and the Web PKI, they just add the plugin to their cPanel install and miraculously there's a new feature for all customers.
Thanks for documenting the re-usable parts of what you did @CRCinAU
So for the sake of completeness - I have documented properly the steps on how I managed to achieve this:
I wrote the guide from memory, so if I missed a step, please let me know.
1 Like
Just some minor shell nit. Instead of
printf "server 127.0.0.1\nzone le.example.com.\nupdate add %s.le.example.com. %d in TXT \"%s\"\n\nsend\n" "${2}" "${TTL}" "${4}" | $NSUPDATE
you can do
$NSUPDATE <<-EOF
server 127.0.0.1
zone le.example.com.
update add ${2}.le.example.com. ${TTL} IN TXT "${4}"
send
EOF
Much nicer to look at, months down the road.
Edit: Another thing: You’re basically doing a “push” service, where the central cert VM manages all certs and keys and pushes them to the hosts that need them, right? I think that’s a dangerous setup. The only hosts that need the privkeys are the individual hosts. You should switch to a “pull” model where each hosts downloads its cert from the cert VM. Of course this requires a script on each host that generates keys and CSRs and uploads the CSR to the cert VM, but it avoids collecting all privkeys in a central place.
1 Like
You’re right re the shell stuff - that is much nicer - I didn’t think of that at the time. I’ve updated the page with you suggestion. Thanks!
Yes, it is a push system - as eventually, I’ll add support for pushing the same cert to multiple hosts etc. Having the target system do a pull means I have to allow external access to the cert system by some method. I’m not really wanting to do that at all - hence the only thing that gets exposed to the outside world is UDP/53.
But collecting all private keys in a central place and transferring them over the network is worse I think. Here's my setup:
- cert VM has 2 unprivileged users: "cert" and "acme"
- "cert" can do nothing but SFTP, it has write access to a csr/ dir and read-only access to a cert/ dir
- "acme" has no remote access at all, has access to csr/ and cert/ dir and the LE account key
- periodically, "acme" runs the main script that collects all CSRs from the csr/ dir and processes them, placing the resulting certs into cert/
- each client connects with SSH public key auth as "cert" to upload its CSR and download its cert
"Communication" between the server and the clients is based on the existence of files, i.e. the server runs through all certs every day and removes access to those that are about to expire. The client fetches its cert every day and checks if there are differences in the files. If it can't access the files, it will create a new private key (or optionally reuse an old one) and upload a new CSR.
The dir structure on the server is this:
drwxr-xr-x 4 root wheel 512 Nov 29 13:03 /home
drwxr-x--- 2 root acme 512 Nov 30 07:38 /home/acme
-r--r----- 1 root acme 3243 Nov 28 09:56 /home/acme/account.key
-r--r----- 1 root acme 3243 Nov 28 09:56 /home/acme/account.key.bak
-rwxr-xr-x 1 root wheel 22560 Dec 1 13:54 /home/acme/acme.pl
drwxr-xr-x 5 root wheel 512 Dec 1 13:51 /home/cert
drwxr-x--- 5 acme cert 512 Nov 30 10:05 /home/cert/cert
drwxr-x--- 3 acme cert 512 Nov 28 09:39 /home/cert/cert/[...$domain...]
drwxr-x--- 2 acme cert 1024 Dec 3 00:00 /home/cert/cert/[...$domain...]/ec
-rw-r----- 1 acme cert 34510 Nov 28 09:58 /home/cert/cert/[...$domain...]/ec/20161128-095823.log
-rw-r----- 1 acme cert 1566 Nov 28 09:58 /home/cert/cert/[...$domain...]/ec/20161128-095823_cert.pem
-rw-r----- 1 acme cert 1647 Nov 28 09:58 /home/cert/cert/[...$domain...]/ec/20161128-095823_chain.pem
-r--r----- 1 cert acme 477 Nov 28 09:55 /home/cert/cert/[...$domain...]/ec/20161128-095823_csr.pem
-rw-r----- 1 acme cert 527 Nov 28 09:58 /home/cert/cert/[...$domain...]/ec/20161128-095823_ocsp.der
-rw-r----- 1 acme cert 1200 Nov 28 09:58 /home/cert/cert/[...$domain...]/ec/20161128-095823_root.pem
-rw-r----- 1 acme cert 87 Dec 2 04:02 /home/cert/cert/[...$domain...]/ec/20161202-040001_ocsp.der
-rw-r----- 1 acme cert 527 Dec 3 00:00 /home/cert/cert/[...$domain...]/ec/20161203-000002_ocsp.der
lrwxr-x--- 1 acme cert 24 Nov 28 09:58 /home/cert/cert/[...$domain...]/ec/cert.pem -> 20161128-095823_cert.pem
lrwxr-x--- 1 acme cert 25 Nov 28 09:58 /home/cert/cert/[...$domain...]/ec/chain.pem -> 20161128-095823_chain.pem
lrwxr-x--- 1 acme cert 24 Dec 3 00:00 /home/cert/cert/[...$domain...]/ec/ocsp.der -> 20161203-000002_ocsp.der
lrwxr-x--- 1 acme cert 24 Nov 28 09:58 /home/cert/cert/[...$domain...]/ec/root.pem -> 20161128-095823_root.pem
drwxrwx--- 2 cert acme 512 Nov 28 09:58 /home/cert/csr
Every file is versioned with date and time, no file is ever deleted. Same thing on the client.
The algorithm on the client is this:
- check for CSR on server; exit if it exists, continue if it doesn't
- try to download cert, chain, etc.; if successful, compare to current files; delete downloaded copies if equal, reset symlinks if not equal
- if downloading wasn't successful, create a new private key, generate CSR and upload CSR
Algorithm on server:
- check every cert for expiry, remove symlink to certs that expire within x days
- take all CSRs, request certficates for them; if successful, remove CSR from csr/ dir
- get OCSP responses for all certs
Clients and server do this daily.
The cool thing is that you don't have to care about rate limits at all. This process ensures that the certificate generation will naturally space out over the needed period of time automatically, because if cert generation fails, the CSR won't be removed so the client will not create a new one and just try again next day. The only drawback is that if you are currently in a rate limited period, you won't be able to get a cert right away, but there's nothing you can do about that anyway. As long as you don't have so many certs that you will never get out of the rate limit, you will eventually get a cert and all your certs will be spaced out so that renewal will happen without hitting any rate limit. I think it's pretty neat.
2 Likes
Make sure those "spaces" in front of the nsupdate commands are actually a tab. The syntax of <<-EOF (as compared to just <<EOF) means that if there are leading tabs in the "input", they will be stripped automatically, but that only applies if those are actual tabs (ASCII 0x09).
intresting system you have @TCM