I haven't used HTTP validation in a long time, but those do not look like any legitimate/documented requests. I dropped that provider's entire /48 into my firewall, but it left me again questioning how to pick out actual LE we-cannot-tell-you-what-hosts-we-use traffic from malicious attacks and/or the rise of LLM slop-bots.
Any future-looking guidance that allows me to keep using LE while still denying insecure networks access to mine would be appreciated. Thanks.
HTTP-01 domain validation (which is the only HTTP based validation method used by Let's Encrypt) only requests paths under /.well-known/acme-challenge/.
If you were using the TLS-ALPN-01 or DNS-01 validation methods then Let's Encrypt would not need to make HTTP requests to your server.
Let's Encrypt's validation authority (VA) does follow redirects, so it is possible we would request paths other than within .well-known/acme-challenge/ if following a redirect. But I can confirm that IP address range is not one that we're using.
Or properly configure your webserver's access control. It shouldn't matter if a bot is poking and prodding at your site, because they're going to anyway.
I really hope there isn't someone out there granting additional access to validation requests by checking the user-agent.
Presumably that means redirects from my side, correct? And only from the same origin? Presumably there's been some adversarial testing of scenarios where attackers try to use LE to hit some target server with bogus challenges.
Regardless, there is apparently someone out there abusing the name/trademark of LE.
It's less that I'd want publication of server IPs than I'd want a protocol that is better than just having a random host start probing around a site. Especially if that host is part of some cloud provider's server farm that also gets rented out by attackers.
Correct. Let's Encrypt only sends challenge requests in response to a cert request by an ACME Client. LE would only see a redirect to follow if that requesting server replied to the initial challenge with a redirect. See: Challenge Types - Let's Encrypt
I guess that's for the lawyers to sort. But, the user-agent string is just informational and should never be relied on for any security purposes.
Except it's only probing a specific file in a specific directory. That file is a long random string only known by you and Let's Encrypt. There's nothing preventing you from configuring access to only the specific directory needed.
Although there should normally be nothing in the .well-known folder. And the challenge file is useless to anyone else. Placing a file that is then verified by a 3rd party has been a part of the web PKI since the dawn. Some webservers can even serve it directly out of memory, or there's always a DNS challenge
Well, some of us prefer not to just shrug and ignore a hostile network. I'm always going to drop that kind of probing into my firewall; I just wanted to check to see if there was any chance it was going to impact actual LE operations.
Well, there is, in that I don't know who will be requesting it, so I don't know if I have to punch a temporary hole in my firewall to allow some random cloud server to start hitting my server. The web is just a mess these days, and I'm very happy I switched to DNS challenges long ago.
Ha! Simply not true. Anyone requesting a file that I don't have is attacking me 99.99% of the time.
Seems to work fine for the majority of the internet. I assume you have a security appliance that can restrict access by URL on port 80.
If there's a better way to do validation you know of that can be accomplished automatically, I, Let's Encrypt, and the CA/B forum would probably be very interested in it
Please note. In this statement, because this is a forum discussing an ACME compatible Certificate Authority, I was using "the .well-known folder" as shorthand for the acme-challenge directory within the .well-known directory. The acme-challenge directory only ever contains ephemeral acme challenge tokens.
Then you're using a different Internet than me. On the Internet I use, compromised hosts and IoT devices seem to be creating new record-level DDoS attacks regularly. And network owners seem to ignore the abuse coming from their customers so long as the check clears.
As I've said, the web is fundamentally broken in its current form. But a better way would involve me being able to validate whether or not it was actually LE hitting my server without having to come here and ask. It remains a fact that probing for files on a web server that might result in a 404 (or worse) is something that mimics the actions of an attacker; no modern protocol or standard should be based on such behavior.
If you want a better protocol, use DNS challenges with a DNSSEC-enabled authoritative server to prove your control over the domain. I think that's really what you're looking for. Allowing for domain control to be "proven" over unencrypted/unsigned HTTP is just a concession to the reality that for many systems that's much easier to set up.
I noted that I do not use HTTP and do use DNS, which is why I was very puzzled to see requests from someone claiming to be LE in my logs! That's why I questioned if there might be some way a legitimate LE server might possibly be made to act that way (i.e., some attacker attempting to falsely get a cert issued via HTTP).
You can define which validation methods you accept in CAA records, so you could use that to tell Let's Encrypt that you only want them to use DNS-01 challenges.
If you haven't done that, then somebody else could, using their own Let's Encrypt account, try to request a certificate for a name in your domain. It'll fail, but they can still try. CAA records let you restrict which accounts can request certificates in your domain as well, so that helps there too.
Ultimately though, anybody who can set up a TCP connection can send you whatever they want over it. Heck, if they aren't concerned about a response they don't even need to establish a TCP connection. Look at what DNS requests your IP addresses receive -- and the source IP addresses on those probably aren't those of the actual sending hosts.
Getting slightly further off the point, I know, but draft-ietf-acme-dns-persist-00 is looking interesting -- I think I'll be happier using this than having to have an HTTP server running just to handle validations for IP address SANs (even though said server is only running when I'm expecting challenges).
TLDR but if you are not using HTTP domain validation, then it's not Let's Encrypt that are attempting validation (or at least, they are not attempting it for you) so you can safely block all incoming http requests that match that user agent.
They can't stop people spoofing their user agent, because basic HTTP is just text sent over a socket, but you could allow /.well-known/acme-challenge/ requests only.