I have three servers, with the same domain name, and round robin DNS. It's hit or miss, as to whether my certificate registers or not.
Please can Lets Encrypt pull all A records to verify it's a valid server, instead of just pulling whatever DNS decides to send.
Hi @LogicEthos, and welcome to the LE community forum 
I see a couple of problems with this:
When DNS returns only one IP (at a time - "Round Robin" )
There is no way of even knowing how many IPs are in that rotation.
Only you control the DNS that control the A record responses - LE can't put anything into any of your A records.
When DNS returns all IPs (at once) [and the client is left to decide which to go to]
This might be closer to your request...
You'd have the CA check them all and pass them (all) if any one of them has the right answer file?
OR check them all and pass only if all of them have the right answer file?
Seems like you are trying to reinvent an already working wheel.
In order to do so, you would first need an RFC and then that would have to be included by the CAB forum as a requirement on all issuing CAs.
Sorry, but I don't think LE would even be allowed to deviate from the rules (enough to satisfy such a request).
So your site is hosted on three different servers, and they don't consistently serve the same information? That sounds, to put it mildly, like a major design flaw.
No, the LE validation server uses DNS as it's meant to be used.
You could redirect requests for the /well-known/acme-challenge/ to a certain server-specific FQDN and run your ACME client on that server. And then redistribute the cert amongst all the servers.
I'm not sure why LE would check them all.
If I have 3 servers, and I add a 4th, that 4th server is going to request a certificate from LE. LE pulls the full A records list, and checks that the 4th server's IP is in the list.
I understand the problem you're solving. It's not an unreasonable request!
I don't think it's something we're likely to change anytime soon, unfortunately. We wouldn't know which IP from the A record you'd like us to use, so changing that would require a lot of software and protocol changes to make work.
No, same content. Round robin is a simpler alternative to using a load balancer. Same issue applies if your using geo-ip or other DNS steering techniques.
Copying the cert private key is the way to get around it. It just makes it less secure.
I believe the ACME specification requires the validation record to be available from all IP addresses returned by DNS. So if you have multiple web servers, you need some way to ensure that the validation record is available from all of them.
A straightforward solution would be to request the certificate from one server, then redirect (or proxy) /.well-known/acme-challenge/ on all the other servers to that main server. That way Let's Encrypt's validation servers will always be able to read the validation regardless of which A record they end up using.
Note that you'll then need some way to copy the certificate from that main server to the other servers. Certbot lets you configure a script that runs after a certificate has been renewed. You could just set up a script that copies the certificate to the other servers using rsync or something.
Yes you would. The host that is making the request!
Then there's no problem--whichever of the three (or however many) servers gets the request will be able to answer with the correct token.
Edit:
But who says the host that's making the request is the one you should contact? Just because you requested a cert from a given system, doesn't mean that's the system that should be queried for the validation token.
You can can different private keys of different servers. I tried it, by manipulating the DNS records between certificate renewal requests. Works fine, and more secure. But it's a pain to automate it.
Host makes a request from IP 123.123.123.123 to LE
LE checks DNS records. IP 123.123.123.123 is in the A record list.
LE queries 123.123.123.123
Success!
So if you can generate a request from an IP that belongs to a given domain, you can get a cert for that domain? That's an awful idea--to name just one obvious flaw, it'd let you get a cert for any other domain that's hosted on the same shared host as your domain.
Again, if all the servers are serving the same content, that would include the validation tokens. So why is there a problem? Unless they aren't actually serving the same content.
Err what? Thats how it works now. I don't think you know what your talking about.
A lot of the time the IP of the ACME API request won't match any of the A records. I.E. my company's web servers are behind a load balancer that has a single IP. Requests from our servers out to the Internet come from the IP address of the actual backend server. Not to mention, it's perfectly valid to request a certificate from a server other than the web server that the certificate will be used on.
I suppose you could say "only do this if the request IP is one of the A records, otherwise just use any of the A records", but this would add a bunch of complexity to Let's Encrypt's system - the systems that do the record validation likely aren't even aware of what IP address made the original API request, since there's no need for them to know that, so everything would need to be updated.
This is such a niche thing you're asking for that it doesn't make sense for LE to take on the extra complexity. Instead you should work on making sure your web servers are all serving the same content. Sure, it adds complexity for you, but using multiple web servers is inherently more complicated than just using one.
It obviously isn't, or you wouldn't be asking that the system be changed to work in this way. But you keep ignoring the fundamental point I've made twice before: if the servers really are serving the same content, that would include the same validation tokens, so your imagined problem isn't a problem at all. The only way it's a problem is if they aren't serving the same content.
Concretely, @danb35 is saying "if one of the servers is hosting a /.well-known/acme-challenge/<token> document, and the other servers are not, then by definition the three different servers are not hosting the same information". I agree with this assessment. One possible solution would be to redirect all requests for /.well-known/acme-challege/* to a specific host (the one currently doing renewal). Another possible solution would be to replicate the contents of that path across the three servers, so that all three of them could fulfill an HTTP-01 challenge started any one of them.
Another (and imo better) solution would be to switch from using the HTTP-01 challenge to using the DNS-01 challenge. Then validation requests will not need to be directed to any of the three individual servers -- as long as they all have control over DNS, the validation request can care only about the DNS records themselves.
We cannot do this. Specifically, at validation time, we do not know (and for privacy reasons do not want to know) what IP address originated the request. Even if we did want to do this, it would raise the question of "which origination IP should we use?". An ACME issuance request is actually many different HTTP requests: optionally creating a new account, placing a new order, querying authorizations and challenges on that order, attempting to mark challenges as complete, and more. Depending on the client's network and ACME client setup, each of these requests could appear to originate from a different IP!
I really don't see why its difficult.
Instead of pulling a single A record, you pull them all. It's just a different DNS request. One line of code.
Then add a second line of code to see if there is a match.
Just this small change, will:
The problem is that the servers that check the validation don't know what IP address your request came from. (Adding this would definitely not just be one line of code, and according to LE staff, is actually something they don't want to send.)
It's possible to use the same private key on multiple hardware security modules. The mechanism for doing so depends on the manufacturer.
Same content. Different private key. That's the only difference. If I copied the same private key between the servers, then that will be the same too. Which I could do. And probably will have to. It's just less secure, do to a policy decision by LE.
So how do you return the request?