As a part of a web server protection strategy it would be valuable to have a list of source IPs that Let’s Encrypt uses in HTTP-01 Challenge validation.
I understand the IPs can change so my suggestion is for Let’s Encrypt to make the list available via HTTP in raw text, JSON, XML, whatever format. Just make it available.
I don’t want to rely solely on allowing access to the User-agent string, currently I see “Mozilla/5.0 (compatible; Let’s Encrypt validation server; +https://www.letsencrypt.org)”, by URI, “/.well-known/acme-challenge/[token]”, or even the combination.
My cron job calling a script could get the IP list from Let’s Encrypt and update/create the necessary [web application] firewall rule(s) to couple the User-agent, URI, and IPs together each time renewal is attempted.
I searched prior to posting and so I understand there are numerous past posts about the IPs but the provided link and the link within it do not address my suggestion. As stated in my original post, I understand that they are not published and that they may change. My suggestion is to provide a list. This would be very easy - they’re simply updating a piece of data when the IPs change, at a static URI.
Said differently, past posts have asked about the IPs and the response has been that they change. Ok, they change. Doesn’t change that the list could be provided. Easily. Provide it.
I’d be happy to automate this if they’d give me access.
You’re of course free to wait for an official answer from someone associated with Let’s Encrypt (@jsha?), but I thought you were asking. Sounds more like demanding at the moment.
Hi anon, welcome to the forum! @Osiris has got it exactly right. We have a longstanding policy that we will not publish validation IPs, for the reasons explained in that FAQ entry.
No, they won't be published because by policy they aren't intended to be known. This is a deliberate design choice on the part of Let's Encrypt, not simply an oversight.
I think the idea is that by not publishing the IPs, Let’s Encrypt isn’t held back by an older API if it grows to need a huge CDN. Suppose they go to Cloudflare, then what? Are you going to whitelist everything on https://www.cloudflare.com/ips/? And if the spec had been individual IPs rather than netmasks, ouch…
HTTP-01 challenges shouldn’t take more than a few seconds anyway, that’s a pretty small window of time to lower a firewall. You can put that in --pre-hook and --deploy-hook instead of cron, as well, so that it only happens when a cert is ready for renewal, not every attempt.
I agree with what you’ve posted, But think there’s also a security consideration here. If the validation ips aren’t known and randomly change, in addition to multi perspective validation it provides a little more security in the event of route hijacking or some other routing vulnerability and would make an attack much more difficult to carry out.
Providing an IP whitelist isn’t as simple as you might think. What if they want to perform multi-perspective validation on, say, a Kubernetes cluster in a public cloud? With every VM getting its own public IP, an IP would go from not being on their list at all to suddenly being used for validation in under a minute.
Most people asking for an IP whitelist want it so they can hardcode it in their firewall rules / server config / etc. and update it periodically. That’s just not compatible with dynamic environments like this. I’ve been on the other side of this (partners/clients asking for IPs they can whitelist) and it’s very frustrating. I totally get why LE doesn’t want to handcuff themselves like this, and I commend them for sticking to their guns.
The vast majority of users don’t whitelist IPs on their web servers, and for those that do, the DNS challenge exists.
Yes, providing the data can be easy. Just make the data available, the rest is up to the user. In your example, I'd bet the solution is to use a single host for renewal and then distribute. All of that can be easy too.
While you're probably right that most want to hard-code on their firewall that's not quite what I'm wanting to do. Well, not "permanently" - just temporarily. As noted above...
Cron executes my script
My script collects Let's Encrypt published data
My script adds IPs to firewall
My script executes renewal
My script removes IPs from firewall
DNS challenge won't work for me. My "firewall" could be any of:
Web Application Firewall - on the web server, of course
netfilter - on the web server, of course
Cisco ASA firewall
Which "firewall" I choose to deploy the rules on is of no concern to this topic.
Lastly, I understand that Let's Encrypt's goal is to "protect" the IPs (or uh I believe that's the goal). However, at this point, I doubt the list is very dynamic so let's not forget that a person with malicious intent can collect them through intended use of the use. Sure, at this point we have no idea how many or their distribution. But having the IPs of the validation servers... what does that get a malicious person, meaning what are the attack vectors? I bet very few and low risk at most, should be none. DoS? Now we're back to distributed validation.
Currently I allow the validation servers by the combination of User-agent and URI (previously noted in my original post). So don't get me wrong, I understand, by and large this is very likely more than sufficient. But I'm always looking for ways to improve security. Through continued conversation maybe a solution beneficial to both sides can come to light. I'd like to hear further thoughts from Let's Encrypt. @josh@cpu@jsha
I don’t believe the benefits of source based IP filtering for HTTP-01 validation requests merit the investment in time/moving parts to make an API for blessed IP addresses. BCP-38 doesn’t have universal adoption, even in 2020.
We also see plenty of challenges on the community forum that folks run into while automating certificate renewal without trying to eke out extra security at the margin with src IP validation. I am certain we would see an increase in folks having trouble if it became an approved practice to try and automate firewall rules with our blessed src IP API. Overall it would limit our agility to add/remove/change validation vantage points, even if we kept the list perfectly up to date at all times with no errors (which as others have pointed out, is not as simple as may be thought in a dynamic web scale environment).
I think that if you don’t intend the word at large be able to access your host via HTTP then the HTTP-01 challenge is likely a bad fit for your use case. Is there a reason why you have ruled out using DNS-01 challenges? You mention it won’t work for you, but not why.
To me this thread feels like it might be a time where we’ll have to agree to disagree but perhaps we can make progress on understanding what’s preventing DNS-01 usage in your enviornment.
We don't really intend to "protect" them in a secrecy type sense. We're very aware that you can enumerate the source IP addresses in use today by performing validation requests. Even if we used a random subset of some large pool of available addresses it would be trivial for an adversary to perform enough certificate issuances to see the full set in a reasonable amount of time. The asymmetry of acquiring and operationalizing more src IP addresses vs discovering them would make this a losing battle from the start.
CAs are mostly equal. The perpetrators of elaborate attacks can choose any CA they want to. (Give or take aspects of the target's setup, like CAA records or DNSSEC, that might make one CA more feasible than another.) And Let's Encrypt can be used by high value targets (though it may be more difficult).