Certificate renewal fails since challenges from multiple network vantage points

[zycavi]

My domain is:
md.masterdocs.com.au

I ran this command:
(a) Tried to do an automatic renewal (sudo certbot renew via cron)
(b) sudo certbot --nginx

It produced this output:
Domain: md.masterdocs.com.au
Type: connection
Detail: Fetching
http://<above domain>/.well-known/acme-challenge/_K8qyQ6N7aYmtFOi4XZrDPR1Wk9JdTk5kDhBdQ6C12s:
Timeout during connect (likely firewall problem)

My web server is (include version):
nginx version: nginx/1.14.0 (Ubuntu)

The operating system my web server runs on is (include version):
Ubuntu 18.04

My hosting provider, if applicable, is:
busdoc

I can login to a root shell on my machine (yes or no, or I don’t know):
yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel):
no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you’re using Certbot):
certbot 0.31.0

This has only just started popping up with the auto-renewal working for month and years - now it suddenly fails.

I believe it’s likely doe to the recently introduced “challenges from multiple network vantage points” - our hosting provider is blocking all non-Australian traffic, so I’d image before it would have chosen the closest vantage point only (which would have always been in Australia) and now it tries to validate from multiple different geo-locations of which some are being blocked.
It would be important to get this issue sorted soon as over the next weeks more and more of our domains will start to fail the renewal - with the current domain I’ll be able to do the dns-challenge “by hand”, but this is no long-term option as there are too many sites that depend on automatic renewal. (Our dns provider also doesn’t offer any automised so the dns-challenge won’t be able to be automised either.)
Unfortunately the geo-block isn’t in our hands, so while I’ll reach out the hosting provider and try to get an exception in place for “.well-know”-paths, I don’t know if that’s possible or how long it’s gonna take them.
Seeing that the introduction of multiple vantage point is a breaking change, it would be nice to have a (at least temporary) opt-in/opt-out possibility to revert to the old behaviour while the necessary adoptions are taking place.

As I can’t say for certain that the “multi vantage point” is the cause, I’m (of cause) open to all other suggestions and all help is always highly appreciated.
Thanks a lot in advance!

[mnordhoff]

For what it’s worth…

It seems like the timing is a coincidence, and this isn’t about multi-perspective validation.

While Let’s Encrypt’s validation locations are not documented or guaranteed – especially now – the fact is that Let’s Encrypt has never had validation servers in Australia.

If the original validation locations were succeeding, and the new secondary ones were failing, I believe the error message would be slightly different.

If HTTP validation worked before, it appears your provider was allowing traffic from the United States before, and is blocking the very same IPs now.

[zycavi]

@mnordhoff, thanks for the quick reply and the clarification!
What a strange coincidence - I am getting in touch with the provider as I write this, and will post the answer back here as soon as they’ve responded. (Based on your answer I’m hopeful they’ll be able to shine some light on this.)

’ marking your answer as the solution and closing for now - based on your answer this cannot be an issue with let’s encrypt! Thanks a lot!