Switch from StartSSL to LetsEncrypt, looking for easiest route

[mje]

Please fill out the fields below so we can help you better.

My domain is: Multiple, I’ll play with secure.posix.co.za

I ran this command: nothing yet

It produced this output:

My operating system is (include version): Gentoo 4.1.15-gentoo-r1

My web server is (include version): Apache/2.4.25 (Unix)

My hosting provider, if applicable, is: I am the hoster, look at https://vweb.co.za

I can login to a root shell on my machine (yes or no, or I don’t know): Yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel): Difficult, I wrote my own.

I started out years ago, wrote my Virtual hosting system from scratch - call it VWEB. This was before things like cPanel existed.The System allows users to create an account, login, create Domains. The Domain can have multiple services, DNS, E-Mail - etc. I support IPv6, DNSSEC Signing and SSL Certificates. Almost all web sites run on a single IPv4 and IPv6 address (196.29.61.1, 2001:43f8:790:61::1). If a Site is to run SSL, then I’m probably running a web site though some FQDN’s for mail also have SSL certificates (I’m playing with DANE)

Up till now, very few web sites are SSL, except almost all my own web sites are SSL enabled, often with a shared (wildcard) certificate (*.posix.co.za). If I run SSL, I’ll be running the DNS for that Domain off the same machine.

So currently, I have a web interface that will create a self-signed certificate and allow the user to copy and send the CSR to a Cert, and when the signed CRT is returned, to over-write the existing (old or self signed) CRT and effectively add this to the System (via MySQL Database). The User can choose which Certificate combination to use. Thinks like creating/updating TLSA Records also happen at this point - which is directly written (via the DB) into the DNS.

I want to include support for LetsEncrypt but have not yet seen the most appropriate way to do this. VWEB is written in PHP. I’d like to offer LetsEncrypt as a free add-on for all my customers. I’d like to enable it via my existing interface system - hopefully a simple {enable]/[disable] selection - as is my DNSSEC.

Where do I go next?

[jmorahan]

First, you should probably read is the integration guide if you haven’t already.

If you’re happy invoking external programs via system, popen and friends, then many of the existing clients should work for you - most have a non-interactive mode at least for renewals and in most cases for initially obtaining a certificate too. If you want a pure PHP solution, there are a number of PHP clients and at least one PHP library on that list.

Since you have control over the DNS you might find it easiest to use a client that supports the DNS-01 challenge. This allows you to validate your control of the domain by creating special TXT records. See for example dehydrated, getssl, acme.sh, certbot.

[mje]

OK - so I tried using certbot (certbot certonly --manual --preferred-challenges dns -d mydom.com) and that worked well. Half way through, I copied the DNS necessary, stuck it in my DNS, etc, then continued with the script.
I’m unable to install certbot on the machine that I really want this on. Its loaded with embarrassingly old software and there is no upgrade happening soon. So now I’m looking at various PHP scripts which won’t have the ‘age’ problem - but have not yet found one that does the “dns01” challenge. The “simple script” by stefan_o is very cute - but http-1

[jmorahan]

The other clients besides certbot that I linked above (dehydrated, getssl and acme.sh) are all bash-based and so tend to be quite widely compatible and light on dependencies - and they all support the DNS-01 challenge. You could invoke one of those via system() or popen() or whatever, much as you would invoke Certbot. They generally also invoke hook programs to perform the actual challenges, which you can write in PHP or any other language (although I think acme.sh might be an exception here; IIRC it expects its hooks to be bash includes, so you would have to write a small wrapper around your PHP script).

[mje]

Thanks - I’m looking at dehydrated. Being in BASH helps me see what is going on.
Regarding “dns-01” - both dehydrated and certbot seem to work in similar ways, contact the LetsEncrypt, be given a Token, then wait for the Token to be installed in the DNS, then back to LetsEncrypt so that validation can take place - resulting in a certificate.
Is it possible to do these two tasks completely separately, one to get the token (and install into DNS) then a second to tickle LetsEncrypt to check the DNS for the token and be handed a certificate. I ask this because I could have a good five minute delay between the two events.

[schoen]

@mje

Is it possible to do these two tasks completely separately, one to get the token (and install into DNS) then a second to tickle LetsEncrypt to check the DNS for the token and be handed a certificate. I ask this because I could have a good five minute delay between the two events.

These steps are totally logically separate, but there's a time limit on the certificate authority side. @cpu, is a "five minute delay" plausible or out of the question?

[cpu]

@schoen

but there's a time limit on the certificate authority side. @cpu, is a "five minute delay" plausible or out of the question?

Let me restate to make sure I know what is being asked before I answer. Please clarify if I'm wrong!

@mje, you would like to:

1. have your ACME client create a new authorization for a domain (www.example.com)
2. receive the created authorization & challenges to know how to construct a DNS-01 challenge response
3. install & wait for the token response to be ready in the www.example.com authoritative DNS zones
4. POST the authorization to tell the ACME server to validate the challenge

and the question is whether steps 2 & 4 can have a 5+ minute delay between them?

If I'm understanding the question correctly then the answer is yes. The authorization will be created in a pending state and will stay that way waiting to be told "Ok go ahead and check" for 7 days (this might change in the future but is 7 days now) before it expires. You can take anywhere up to 7 days to provision your challenge response and then POST to the authorization when you are ready for the ACME server to check it.

The only caveat is that if your client stores a nonce it gets from the response to step 2 it will be invalid by the time you perform step 4 if you wait longer than ~30-60minutes (they expire out based on load). A well behaving ACME client will see the bad nonce error & get a fresh nonce before continuing with Step 4.

Hope that helps!

[mje]

@cpu, The up to five minute pause would be between (2) and (3).
Up to step (2) would be done by a User on the Web Admin page. They are effectively requesting a free “LetsEncrypt” certificate. This event would:

1. Fire off a new authorisation to create a certificate for a particular Domain. This would wait for the reply and create a pair of new DNS records in the DB. The activity would be recorded in the DB along with the nonce token (if that is the right terminology).
2. An unknown time later, CRON starts a process on the next five minute clock tick. It looks for new records in the Zone DB table and if found - writes them out and tickles the DNS via rndc to reload that zone file. It will take a short/finite time to distribute the zone update so that all authoritative nameservers now have the new records.
3. Now, after a second 5 minute clock tick, the ACME can be asked to validate the challenge, pick up the new certificate and install it (restart apache) etc.
   The total time should be between 5 and 10 minutes. (1) and (2) could be (1) just log the users Desire, and (2) from with in the Sync script - ask ACME for the challenge, stick it in the DB, create the DNS records - etc. This would reduce the time wait to a single 5 minute time elapse.
   I hear what you say about getting back a bad nonce and restarting the process.

A) I suppose I should have been able to find some detailed “flow” information of the process - but have not yet seen this anywhere. Does it exist anywhere? URL? (I’ve seen pictures - but they are more related as to how a particular client runs)
B) I’m ignorant to the term NONCE, and puzzled as it sounds like a session token, but lasts ~30-60 minutes, where as there is an authorisation state that lasts 7 days? - which also sounds like some sort of Session Token.

Will get there some day soon.

[cpu]

Hi @mje,

@mje

@cpu, The up to five minute pause would be between (2) and (3).

Gotcha - I believe my answer holds. It shouldn't be a problem.

@mje

The activity would be recorded in the DB along with the nonce token (if that is the right terminology).

You mean the portion of the challenge that you need to provision the TXT record? The spec calls that just a "token" (which is separate from the replay nonce that I mentioned "aging out" in my response).

@mje

A) I suppose I should have been able to find some detailed "flow" information of the process - but have not yet seen this anywhere. Does it exist anywhere? URL? (I've seen pictures - but they are more related as to how a particular client runs)

Unfortunately for this kind of documentation I think the draft RFC is the only thing going right now. I agree that a nicer "flow" overview would be great to have.

@mje

B) I'm ignorant to the term NONCE, and puzzled as it sounds like a session token, but lasts ~30-60 minutes, where as there is an authorisation state that lasts 7 days? - which also sounds like some sort of Session Token.

Nonce means (more or less) "number used only once" and in the case of ACME it's there on every request to prevent a MITM (say, like our CDN) from replaying requests. You can read more about it in Section 6.4 of the ACME draft.

For Let's Encrypt's particular implementation of ACME nonce's there is a fixed sized bucket of "active" nonces and so if you get one now in a few hours time we might have given out enough other nonces that yours is kicked out of the active set (roughly speaking) and will result in a nonce error that requires retrying with a fresh nonce.

Both pending and valid authorizations have an explicit lifetime (the "Expires" field), totally separate from the nonce validity. At present pending authorizations expire 7 days after creation, valid authorizations expire 60 days after the point of validation but that will soon be changing to 30 days.

Hope that helps clear things up!

[GadgetsOut.com]

Since you have control over the DNS you might find it easiest to use a client that supports the DNS-01 challenge. This allows you to validate your control of the domain by creating special TXT records. See for example dehydrated3, getssl2, acme.sh

[mje]

Yes - I’ve been reading the draft RFC. Most RFC’s are never the easiest things to understand but I was able to incorporate EPP into my VWEB system by reading the appropriate RFC’s. I think the difference is that there were examples of code in the EPP RFC’s (XML Sends and Responses) which don’t really exist in the Draft ACME RFC. I’ll read it a few more times.
Did a search for Nonce, and ignored the other definition, so have a good idea of what it means. Perhaps the Nonce Bucket could/should be doubled in size on the server side? Just a thought.
Anyway, ploughing through Dehydrated (I know bash quite well). Definitely need a few minutes pause half way through.
On that thought - do any of the ACME clients with dns-01 separate the collecting of the “challenge” from the ACME server, and actually getting (tickling) the ACME server to check for the challenge? (that you might be aware of).
Meanwhile - I’ll modify Dehydrated a bit.