I got this resolved.
Believe it or not, the solution was permanently to remove my primary domain from Cloudflare.
I too wonder, obviously we don’t suspect a big dude like Cloudflare.
Let me explain a bit.
I was using Cloudflare for the primary domain. This domain used to be the authoritative Nameserver to the box.
Since more than a week now, it takes more than 30 seconds to start rendering the page, especially when I force reload or open from Incognito window.
I thought that would be my new ISPs issue. And when I Pause Cloudflare and use it as a DNS Only mode, the speed becomes normal.
Trust me, I toggled this Pause Cloudflare and Resume Cloudflare multiple times with decent interval between to double-check this.
Below is a hypothetical scenario or a coincidence.
At https://www.whatsmydns.net/#A/any-hosted-domain most of them are green, except the servers from the USA.
But the primary domain’s NS were all green. All the domains hosted on this box returned the same error in this test from the USA test networks.
But now, after I change the NS of my primary domain, https://www.whatsmydns.net/#A/any-hosted-domain shows green to all the USA test systems along with other regions.
Let’s Encrypts servers or main validation, and issuance servers are/may located in the USA.
Thus this failure. Who knows. After a week or so, I will try to reconnect with Cloudflare if I get another nameserver.
For the record, my present NS from Cloudflare is cory.ns.cloudflare.com and emily.ns.cloudflare.com.