Multi-Perspective Validation & Geoblocking FAQ

(This post is a wiki; other community members are welcome to edit and improve it!)

What is Multi-Perspective Validation?

Let's Encrypt needs to connect to your DNS server, and in most cases (for the HTTP-01 or TLS-ALPN-01 challenge type) also your web server, in order to validate that your client requesting a certificate actually has control over the domain name it's asking for. This is what's meant by "Validation"; see the How It Works documentation page for more details. What "Multi-Perspective" means is that Let's Encrypt checks from multiple places, to make sure that control over the domain is established the same way when looking from different parts of the Internet. (Yes, the Internet does look different depending on where one is looking from.)

For more details, refer to this blog post that Let's Encrypt made in 2020 when they first implemented it:

As well as this blog post with updates on their work with Princeton University continuing in 2024:

Why does Let's Encrypt do Multi-Perspective Validation?

Let's Encrypt (like any other public Certificate Authority) has to ensure that your domain is actually controlled by you and not an attacker so that it can issue you a certificate. But if an attacker can intercept traffic intended for your DNS server or web server IP, then the attacker might be able to get a certificate for your domain name. Checking from multiple places makes this much harder for an attacker, because rather than trying to intercept traffic in just one place on the Internet, they would need to intercept traffic in a whole bunch of places.

Are these kinds of attacks against CAs just theoretical, or have they actually happened?

They're certainly not common, but yes they have happened. A recent high-profile one was in August 2022, where an attacker did a BGP attack (getting traffic for an IP range to route to their servers instead of the domain owner's servers) against a cryptocurrency service, and once traffic was routed to them they managed to get a valid certificate (not from Let's Encrypt, but a different CA). (You can see Coinbase's writeup on this "Celer Bridge incident" if you want the full story of that attack, but the BGP attack and getting a fraudulent certificate are just one piece of that story.)

Is Let's Encrypt the only CA doing Multi-Perspective Validation?

Let's Encrypt is leading the industry, as they often do. It is already implemented by some other CAs such as Google Trust Services which calls it "Multi-perspective Issuance Corroboration". CAs and browsers have unanimously agreed to require checks from multiple perspectives from all CAs, beginning in March 2025, with requirements getting stricter over time. All CAs will be required to use at least 5 remote perspectives by the end of 2026.

Where can I read more technical details on how Multi-Perspective Validation helps?

If you look at the CA/B Forum proposal for all CAs to validate from multiple perspectives, there's an "Additional Resources" section with links to lots more information, including further presentations and academic papers on the subject.

Where does Let's Encrypt currently validate from?

Well, it keeps changing. They started with 4 perspectives in 2020, went down to 3 briefly at the end of 2022 while they were regrouping and figuring out exactly where they wanted to validate from, and went back up to 5 at the end of March 2024. They are now once again checking from multiple countries, and expect to keep adding more.

The specific places they currently validate from isn't exactly a state secret, and they'll generally let you know if you ask nicely. But they expect to keep changing these places over time, as they work out the best way to protect from attacks, so any firewall rules based on specific countries won't help over time.

But what countries do I allow through my firewall in order to get a certificate?

Because the validation process is from multiple places, which can and will continue to change, we don't recommend geoblocking the requests that are used in order to validate your domain name and allow you to get a certificate.

But opening my firewall up seems terrible for security! What's the minimum I need to allow?

You can certainly do lots of blocking of traffic to your network that you don't want, but in order to get a certificate you will need to allow this traffic in order for Let's Encrypt to validate you control your domain name during the domain validation phase of the ACME protocol:

  1. For all challenge types: Worldwide access to your public authoritative DNS server
  2. For the HTTP-01 challenge (used most commonly): Worldwide access to port 80 (unencrypted http) for requests to the /.well-known/acme-challenge/ path.
  3. If using the TLS-ALPN-01 challenge (which is really rare): Worldwide access to port 443 (https).

Put another way, here's what you can block, for the most common HTTP-01 method of validation:

  1. Anything you want to block on port 443 (https).
    • Caveat: If the way your ACME client interacts with your web server sends a redirect request from http to https, then your server will need to handle that redirected request on https port 443 in order to get a certificate. This isn't the common way that ACME clients work, however.
  2. Any traffic on port 80 (http) which isn't for the /.well-known/acme-challenge/ path.
    • It's very common for port 80 to only handle redirecting to https, so allowing all traffic on port 80 is usually not much of a security risk.
  3. Anything you want on any port — if it's not a time that you're requesting a certificate. That is, some people use advanced setups having a script called by their ACME client (such as certbot's --pre-hook and --post-hook) that opens up the firewall for the duration of the challenge, and then closes it back up again after. The ACME Protocol only requires ephemeral HTTP access to your server during the validation phase of a Certificate order, you are free to block access outside of this short window which lasts about 5–10 seconds. The need for this kind of setup isn't very common.

But I only ever have users from my own country, why do I need to open something up to the world?

Even if you're only expecting traffic from one place, the certificates that Let's Encrypt issues are valid worldwide. So even if it's not an attack that you're particularly worried about, Let's Encrypt needs to validate that in some other country, there isn't some attacker trying to get a certificate for your domain. And the only way to do that is to make sure that in those other countries, you can still prove that you have control over your domain name.

What tools are out there to help check my domain's accessibility from around the world?

To be clear, these aren't provided by Let's Encrypt, but there are some web tools that community members have found useful:

I can't or won't allow traffic to my web server from everywhere. Do I have any other options?

If your web server needs to stay firewalled off from the Internet (maybe it's only intended for users in one particular organization), then the alternative is to switch to the DNS-01 challenge type. This can be a bit tricker to set up, because it involves updating records on the DNS server when getting a certificate (which is usually a different system), rather than just updating your web server. You need to have a DNS provider with an API allowing updates in order for your ACME client to do this automatically. Most popular big DNS providers do have one now, with plugins available for popular ACME clients to use them. Once set up, control over your domain can be proven with just worldwide access to your DNS server, and your web server no longer needs to accessible to the Internet at all.

One thing some need to be cautious of: Some DNS providers only offer access token credentials that allow for full access to update the entire DNS zone, which not everyone is comfortable having sitting on the web server running their ACME client. However, most big cloud providers, like AWS Route 53 and Azure DNS offer fine-grained control over DNS zone access rights, and it's becoming a more common feature. You may need to create a separate "zone" within your DNS provider's system in order to have a credential limited to just the _acme-challenge record.

If you're using a DNS provider that doesn't offer an API, you can switch to one that does, or delegate just the _acme-challenge name to another system that does. For example, the popular acme-dns tool allows you to run a special-purpose DNS server just for answering the DNS-01 challenge when getting a certificate.

Besides what Let's Encrypt is doing, how else can I protect my domain from fraudulent certificates?

Some great ways to help enhance the security of your domain and help prevent an attacker intercepting your traffic and getting a certificate include:

  1. Implementing DNSSEC.
  2. Implementing CAA records
  3. Monitoring Certificate Transparency logs using CT search tools. (This is about detection rather than prevention, but can still be helpful.)

I'm still having trouble getting a certificate, what should I do?

Don't post in this thread. Instead, create a new topic in the Help category, answer the questions provided and give as many details as you can about your system and what might be blocking the validation requests, and the friendly people in this community will see what they can do to help.

17 Likes
What ip address do i need to put in geoblock
Let's Encrypt servers list
HTTP challenge failing Nginx HTTP to HTTPS redirect
Question on renewal and what Countries are connecting
New Issuance Chains on Staging Failing
The Certificate Authority failed to download the temporary challenge files created by Certbot
The Certificate Authority failed to download the temporary challenge files created by Certbot
Webroot-based certificate creation no longer works
Certificates Failing to renew but Nginx accessible
Timeout error when using certbot nginx
The certificate has expired: that is the notAfter date is before the current time
Renewal failures due to geographic/country network block
IP or pool ip for get ssl?
Errors renewing certificate, Apache, Win 10
Authority failed to download the temporary
Instabilidade apontada no lets status io pode ocasionar adição na blask List do Google search?
Can't renew certificates
Moving server and change certificate from Certbot to win-acme
Failing to renew certificates on step 6
Certificate generation failing. Error getting validation data
Sudden renew failures but not firewall block
Sudden renew failures but not firewall block
Timeout during connect (likely firewall problem)
Certbot failed to authenticate some domains (authenticator: nginx)
Países possíveis, origem da renovação
Error: During secondary validation: Remote PerformValidation RPC failed
What ip address do i need to put in geoblock
Error 403 - Creating a new certificate
Timeout during connect (likely firewall problem)
Let's Encrypt problems
Renew Certificate using DreamHost
Let's Encrypt is not renewing
Timeout during connect (likely firewall problem)
Secondary validation timeout
Solution for Orgs Who Use Restrictive Firewalls
Secondary validation timeout
win-ACME certificate renewal
Problem with LetEncrypt validation after blocking all server IPs
openSUSE15.6 apache2-2.4.58-150600 2ndary vaidation
Certificate failed to Renew
Homebrewed client gets "No TXT record found"
*** Certificate renewal failed ***
Trend Micro Blocking AutoRenewal
AuthorizationError('Some challenges have failed.')
IP's or network's used by letsencrypt for renew
IP addresses / FQDN for cert renewal
The CA failed to verify the changes made by Certbot
HTTP-01 secondary validation coming from AWS
Sudden 'some challenges have failed' issue on an old website
Qualys SSL Server Test vs. SSLChecker Test Conflicts
Certbot renew keeps failing, I can't figure out why it works only sometimes
Timeout when attempting to issue certificate for cerb00apps.onesoft.es, but Let’s Debug can connect
Firewall blocking traffic from let's encrypt
Connection refused [BadRequest :: urn:ietf:params:acme:error:connection]
Public IP addresses of Issuance process
Request New Cert always Error Getting Validation Data
ERROR: Could not get certificate from Lets Encrypt. Check domain name and if it is reaching the configured service
.services: Domain name does not end with a valid public suffix (TLD)
Unauthorized during secondary validation
Renewing my SSL
Impossible to renew certificate after 60 days, DNS problem
Is there a FQDN we can apply to geo-restricted port fowards for the cert renewals?
Certbot renew fails
Certificate expired? what
Problems with CAA records only with Google and Let's Encrypt
Cert creation fails: HAOS with Nginx & LE + Apache2 web server
Website zeezicht-oostende.be not find by SSL
Alternative for allowing letsencrypt file auth connections for a geo-restricted server
Alternative for allowing letsencrypt file auth connections for a geo-restricted server
Http Challenge Fails
1 of 15 domains failed certbot renew of cert? (no issues since 2019 before)
Suddenly getting 'Unable to connect to ACME server' today
Failure to poke my webserver from Cerbot servers
Suddenly getting 'Unable to connect to ACME server' today
Le nombre maximum de demandes de certificat est atteint pour ce nom de domaine
SSL Certificate Renewal Failing for My Website
Could not issue an SSL/TLS certificate for www.domain.com
Nagios is not able to auto renew the certificate
Certbot --apache -d failed. status 400. DNS problem: NXDOMAIN looking up A for xxx - check that a DNS record exists for this domain
CertSage Authorization still pending after 10 attempts
Firewall problem with certbot ubuntu nginx
ACME failure on Expressway-E 15
Update Letsencrypt certificate fails
Connection refused status 400, Apache mod_md and mod_proxy, fail2ban
Cannot generate ssl certificate even though ufw enables 80 and 443
Error accepting authorization: acme: authorization error for mydomain.com: 403 urn:ietf:params:acme:error:unauthorized: During secondary validation: xx.yy.zz.aa
Unable to get SSL certificate passbolt
Certbot renew failed: Connection aborted
Challeges failed
Timeout during connect (likely firewall problem)
Unable to renew my certificate
The Certificate Authority failed to download the temporary challenge files created by Certbot, while port 80 is open
Falla al certificar Dominio
Unable to obtain SSL Certificates Apache
Firewall Geoblocking and LetsEncrypt
Certbot certonly --standalone -d smtp.example.com
Timeout during connect (likely firewall problem)
IP source addresses
Failed some CERT renew
Http challenge failure
Renew certificate failed
90 day expiration
Servers and IP addresses used by Let's Encrypt
Whitelist LetsEncrypt Server IPs
Acme challenge accessable via browser but certbot fails
Failed validation limit
Failed validation limit
Any changes to multi-perspective validations lately?
Failed Renew Certificate Error getting validation data
Issuance of Certs
Renewal blocked by Cloudflare because challenge was coming from outside the US
Issuance of Certs
Error getting validation data
Acme Challenge 404 with webroot
Renewal Failure Starting 5/24/2024 403 Error
Let's Encrypt in FileZilla
HTTP-01 error at renewed nginx
Is DNS record for 'acme-challenge' required?
Is DNS record for 'acme-challenge' required?
Https stopped working
Error updating renewal info: "Must specify a request path"
Timeout during connect (likely firewall problem)
Challenge Failed but nginx say its ok
DNS Manual Failure Going on 4.75 hours so far
Lets Encrypt certificate automatic renewal
Certificate is not for the chosen domain

Google also supports multi-perspective validation which they refer to as "Multi-perspective Issuance Corroboration (MPIC)".

10 Likes

Thank you; I was pretty sure that other CAs had already started but I didn't know which ones offhand. I've added a mention and link in the post.

7 Likes

Want to merge in any of the material from this earlier draft?

(I think your version is more comprehensive and more useful!)

8 Likes

Thank you! I realized as I was falling asleep last night that I was neglectful and should have at least referenced and probably linked to your related helpful pinned post.

I think it can be helpful to present similar information in different ways, and I wasn't trying to usurp the place of your posts or anything. :slight_smile:

I do like your proposed documentation page, and I could see how it could be helpful to add something to this post about when using a private CA instead is a better idea. But I also was focusing on geographic block issues (since that's what's been coming up lately), and it's probably too long for people to read already. Not sure when I'll get the chance but I'll see if there's more to add here from your text at some point. (Or feel free to add it yourself, I made this a wiki for a reason.) Thanks.

7 Likes

13 posts were split to a new topic: Multi-Perspective Validation, simultaneous requests, and Apache caching

Interesting to note, Google only requires >50% success rate instead of 100%. The message they use if <50% is "X validation attempt(s) succeeded, Y validation attempt(s) failed. Must have more successful validation attempts than failed." Seems like they're currently using 6 total so 4+ are required to succeed.

1 Like

Let's Encrypt also doesn't require 100% of the secondary to succeed. Currently they allow one failure, and while they were ramping up to 5 locations I think it was temporarily two failures allowed.

4 Likes

2 posts were split to a new topic: Any changes to multi-perspective validations lately?

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.