HTTP-01 and AWS challenge

I'm not sure what IP ranges Let's Encrypt uses for it's DCV using HTTP-01, but I know they use Amazon's AWS at some point.

The problem is, there are a TON of bots using AWS and other cloud service that wreck havoc on ordinary web hosting servers. This results in a lot of IPs being blocked from AWS (and other cloud services) which then makes HTTP-01 fail DCV because their system at AWS cannot validate the domain name.

I would propose adding another port, allow it to be customized by the request agent or hard code with whatever protocol Let's Encrypt wants to use - such that http://thedomain:647/.well-known/acme-challenge/random - would validate the domain name.

This way you can continue to use AWS or whatever cloud system you want to use to handle your DCV. Web Hosting providers, like myself, can continue to block misbehaving AWS networks that are flooding ports 80 and 443, but the DCV for Let's Encrypt would still be allowed to go through. I can restrict the webserver that is running on port 647 to only access the .well-known/acme-challenge directory - so any bot that wants to flood port 647 isn't going to do a whole lot.

I just think something is going to have to change in order to continue to allow HTTP-01 or similar validation to happen. There's just way too many AI bots scouring the whole Internet, creating a wasteful resource logjam on web servers. This would seem to be a solution that would allow for the best of both worlds.

This wouldn't have to replace HTTP-01. Call it HTTP-02 or something, but it's just a copy of HTTP-01 on a non-standard port or that allows for a custom port to be sent with the request.

Hello @meisner,

What IP addresses does Let’s Encrypt use to validate my web server?
Let’s Encrypt does not publish a list of IP addresses we use to validate,
and these IP addresses may change at any time.

Let's Encrypt uses Multi-Perspective Validation Improves Domain Validation Security - Let's Encrypt

Best Practice - Keep Port 80 Open

Port 80 is required for the HTTP-01 challenge as other ports are not as well protected in a multi-tenant setup.

As an example, I self host a web server from my residential connection. The router allows other devices to setup port forwarding using UPnP in addition to any manual forwarding, if alternate ports were allowed another device could forward an unused port e.g. 647 and get a certificate for my domains[1].

Other validation methods exist including TLS-ALPN-01 which only requires access to port 443 and uses a different application layer protocol and DNS-01 which only requires access to your authoritative DNS servers.


  1. If I wasn't using restrictive CAA records ↩︎

I know Let's Encrypt doesn't publish their challenge validation IP ranges.

And keeping port 80 (and port 443) is going to continue to become unfeasible, when Let's Encrypt is using the same cloud infrastructure (mainly AWS, but the same would be true of any cloud provider) as all of the AI scraping and resource hogging bots out there.

It's not feasible to block a 10,000 hit per second flood at the application level. Which is where the block would have to be in order to allow exemptions for /.well-known/acme-challenge paths. In order to prevent resource overwhelming, you have to block these at the network layer - and then you can't make path exemptions.

What I propose is to have another port number signified as the port to connect to for validation challenges. It would operate just like HTTP. It's effectively just having the webserver listen on another port. Only at the webserver level, you can configure that port to only work for the /.well-known/acme-challenge path. Still proves you own the domain name. And allows misbehaving cloud IPs to be blocked at the network level for regular ports 80 and 443.

I'm not suggesting that this replace HTTP-01. I'm suggesting that the protocol as defined in HTTP-01 be copied into a new protocol, call it HTTP-02, where the challenge validation takes place on port 647 (or some other classical unused port below 1024). Ideally, the requesting agent would be able to specify this alternate port in their request, but that would require a little bit more rework of the HTTP-02 protocol from HTTP-01.

Everyone is allowed to continue to use HTTP-01 on port 80/443.

But there are real instances where web servers are getting hammered by cloud based IPs that require huge swaths of IP blocking at the network level (i.e. iptables/ipset) in order to control resources on the server. For administrators of those servers, an isolated HTTP port number would still allow these challenge validations to go through - even if they use the same cloud based IP ranges.

You sound knowledgeable so sorry if this is rudimentary ...

But, Let's Encrypt challenges follow RFC 8555

The better place to ask for this as at the ACME Working Group. Personally I don't think you will get any traction but that's the better place.

You are trying to solve a BOT problem... the wrong way.
If you plan on blocking all IPs that ever scan yours, you will need to block millions of IPs and add thousands to that list every day.
[I doubt you are well prepared for such a task]

When you realize that you can't really solve it that way and you try looking for another solution, you might find what seems to work for most people on the Internet [with regards to HTTP BOT requests]:

  • Allow all IPs to request HTTP [preferably routed to a separate/dedicated proxy/system]
    [don't block any IPs here]

  • forward only relevant requests [like: /.well-known/acme-challenge/]
    [you may need to strip out random requests here - but it will be less "noise" and simple to deal with]

  • redirect all other HTTP requests to HTTPS

  • handle HTTPS as you already do

That’s a fair point, and I don’t disagree that a well-managed environment can mitigate most of these risks. If operators actively monitor traffic, maintain proper filtering, and understand their infrastructure well, custom validation behavior is probably manageable in many cases.

I think the challenge is more about what ACME has to support at scale. The protocol needs to work consistently across millions of deployments, including environments with broken NAT setups, poorly configured reverse proxies, shared hosting, CDNs, and operators with very different levels of experience.

From an individual operator perspective, using alternate ports may be entirely reasonable in a controlled environment. But from the CA perspective, every additional variable increases complexity and the likelihood of inconsistent validation behavior or edge-case failures.

So I don’t think the issue is necessarily that custom-port validation is inherently unsafe. It’s more that the ACME ecosystem tends to prioritize predictable and standardized behavior over flexibility because that keeps interoperability and security manageable across the broader internet.

You can use DNS validation as already mentioned, then none of your servers even need to be on the internet. With the upcoming dns-persists-01 this gets even easier to setup.

At a minimum you can use an application aware (e.g. http-aware) firewall to filter both user-agents and restrict requests to /.well-known/acme-challenge. Some folks even only enable this during domain validation and switch it off otherwise.

I actually agree with you that I won't get much traction. This is why I hate feature request systems. You have to be a politician to get a new "feature" into development. General helpfulness and technicality be damned, it's how can you make money?

DNS validation is a poor substitute. DNS and webserver controls are not always within the same control. See Cloudflare. DNS has a caching and propagation system, which doesn't always provide the same coverage at every specific location. DNS is delicate. Put a period in the wrong spot in a DNS zone and you can kill the whole DNS server.

We'll have to agree to disagree on this one. Every time a connection reaches the application layer, it's going to take up resources in that application layer. While it is true you can limit what those connections can do once they reach the application layer, they're still going to take up resources (i.e. spawn webserver workers, taking workers away from legitimate organic traffic). If you block those connections further down the chain, at the network layer, you never have to worry about those connections taking workers away from legitimate organic traffic.

If you are only blocking connections at the application layer, if you're never using iptables/ipset/nft, then you are (probably) allowing a ton of wasteful traffic into your application. But sometimes people just feel better when they have to pay $4000 for enough hardware to make the application work because they are getting so many hits! Oblivious to the fact that those hits aren't doing anything except taking up resources.

But again, and maybe I didn't do enough to explain this, I'm not suggesting that anything be replaced. I'm not suggesting that everybody has to specify a port on challenge requests or that everyone will have to configure a separate port for validation. If the current HTTP-01 system is working for you, that's great! Keep using it. If DNS-01 is a solution for you, by all means, use it. But I can't be the only one that's seeing these massive floods of traffic from AWS, Azure, and Google (I'm sure there are others) and realizing the resource savings of blocking these at the network level. And for any Certificate Authority - not just Let's Encrypt - that is utilizing these same providers as part of their challenge validation systems, this is going to be a problem. I'm just saying that instead of marketing DNS-01 as the solution - while avoiding the contention that web and DNS are not always managed by the same system - why not have a system that can use a non-standard web port for validation, as an alternative option? Would the option be more difficult to implement for beginning administrators? Probably. But just stick to HTTP-01 until you have a reason to need a non-standard port solution.

But I still agree with @MikeMcQ. I don't have the clout to push this through. There's no money making aspect to be deployed with this. And there will be too much noise about "Use DNS-01, because 'if it works for me, it will work for you!'" I really don't know why I made this post.

It sounds like you're just looking to vent about the state of the Internet in general and the ever increasing scourge of crawlers (both benign and malicious). And that's fine...just try to recognize what you're really angry about and don't direct it towards innocent bystanders trying to help.

A general note. I am looking at the DNS traffic of the authoritative-only DNS server that I am maintaining. I encounter the following (non exhaustive list):

  • queries for non-existent (and never existed in the past) domain names in the authoritative zone
  • extreme high rate queries
  • unsupported query types and query classes
  • queries not within the authoritative zones
  • packets to port 53 that doesn't even look being DNS

One can think that a DNS server exposed to the public access has better situation than an HTTP server in respect of scanning. I think the difference is only in the log level setting in between the two kind of servers.

Don't forget it would also need augmentation of the CA/Browser Forum Baseline Requirements.

Even with a network-level firewall that can only open and close ports as a whole, you can probably script it (with certbot's --pre-hook/--post-hook or equivalents in other clients) to only open port 80 up to all when performing a renewal, and close it to whatever ranges you find most risky at all other times.

Another difference is that DNS is a lightweight protocol especially when using a DNS server optimised for speed such as knot-dns https://www.knot-dns.cz/.

This entire topic is very, very old. It was first brought up in 2015 (back when Let's Encrypt started turning their ACME protocol into a standard), 4 years before RFC8555 came to existence and while ACME was still under active development. In fact, it's the fourth issue ever raised on ACME: Define a new challenge type that runs on a dedicated port · Issue #4 · ietf-wg-acme/acme · GitHub, although most of the discussion happened elsewhere. See [Acme] Issue: Allow ports other than 443 for reference on some of the discussions that went on back then.

Several arguments were made on both sides, both for allowing non-port 80/443 validation and forbidding it strictly. The key concept that we need to understand is what the challenge is even trying to do: An ACME HTTP-01 challenge attempts to validate if a requester controls or owns a domain name, because that's the thing that a domain-validated certificate represents: The entity obtaining the certificate owns or controls [1] the domain name.

HTTP-01 is IMHO a weak start to begin with, because the only control that it demonstrates is control over the webserver running at port 80 (or port X for a customized port). Control over the domain name is inferred by the fact that DNS resolves to the IP address at which the webserver is reachable. So we infer the relationship from there. This isn't perfect to begin with, but it's sort of accepted behaviour as HTTP challenges are quite easy to do with little hassle from the subscriber's POV.

The main argument made against allowing non-port 80/443 validation is that it weakens the relationship even further: Now we're not only inferring that the webserver at port 80/443 controls the domain name, but we also infer that some webserver running at port X (say 647 in the examples above) is controlled by the same entity that also controls port 443 - because that's where the certificate is going to be deployed, right? So the claim is made "both webservers are run by the same entity", but the ACME CA (with the current http-01 method) has no way of validating this claim - it just has to trust it. This is sort of worsened by the fact that certificates cannot bind validity to a given port number [2], so if we validate on port X the certificate is valid for any other port on the same domain. If certificates were to bind a specific port that would be much easier ("you validate on port 647, but your cert is also only valid for port 647"). But this semantic doesn't exist, it likely will never be for practical reasons.

The question is now: Is this second assumption a problem, or is it true with such a high probability that we can just ignore it? Well, the truth is we don't really know. On the mailing list a few scans were attempted to determine this. A primary concern was that shared webhosters exist which treat ports other than 80/443 differently, which could allow for takeover attacks by users sharing a webhost. CGNAT is another problematic example ("I can port forward 555, so I can get a cert for port 443 which I cannot forward").

You can of course try to fix this with authentication: Devise a new challenge type that somehow proves that the webserver at port 443 + 647 is the same entity, but that probably opens up other questions: If you need to bind the ports together anyway, what benefit is left? How does this outweigh the complexities involved?

Eventually, I think the interest did not outweigh the (possible) risks. The CA/B forum has the ultimate authority whether to allow non-standard ports for validation, and they decided to not allow it. To change this, a motion needs to be put in place, demonstrating to the members how allowing the other ports outweighs the possible risks and complexity involved. I am not saying that this cannot happen, just that this has not happened in the 11 years since we first discussed this.

[1] Source: Baseline Requirements 3.2.2.4 Validation of Domain Authorization or Control
[2] This is the source of several TLS attacks, such as the ALPACA Attack

One way would be to require port 80 for the main validation (that is queried from lets encrypt datacenter) but allow a second specific port for the multipath validation.

This would ensure a malicious person behind a CGNAT or wlan cannot acquire a certificate for the domain on the IP they are surfing from (since they do not control the webserver) while it still prevents on-path attackers from manipulating letsencrypt traffic (since then the other port requests would fail)

The only attack scenario this would work for, is if the attacker BOTH have control of the inside of the recipient's network (so they can deploy a service on, lets say 8080 for the multipath validation) AND also are in a network position to manipulate traffic från lets encrypt datacenter.

That would solve the whole ratelimiting problem.

Multi-perspective validation was introduced specifically to protect against the risk of routing manipulation.

Correct. Thats why I suggested that multipath verification should be permitted to be done from an "unauthorized" port. In this way, an route manipulator would need partial access to the target aswell (to deploy a challenge on a unauthorized port) AND also manipulerate the traffic from the main datacenter to port 80.