Here is the thing, you might be right, because after running this line my next attempt to
sudo certbot renew --dry-run did work.
BUT the renewal conf wasn't updated, I don't know what was changed, but something has been changed for sure. Even though the output still contains a line like:
tls-sni-01 challenge for mydomain.com
The dry-run passed, which is not possible because tls-sni-01 challenge would fail when you are behind Cloudflare.
So there is a possible issue here:
Somehow --preferred-challenge http-01 did change some config in certbot, but certbot still report the http-01 challenge incorrectly as tls-sni-01
Either way, I decide it's for the best I just change renewal conf manually and add pref_challs = http-01 for my own sanity.
An authorization can be reused for a certain amount of time -- in other words, validations are cached.
(They're attached to your ACME server account, and the production environment and staging environment have separate databases.)
However, Certbot doesn't know that. Even if it gets back an older, valid authorization, it goes through the motions of validating it again. Certbot will configure the challenge and its output will be normal, but Let's Encrypt won't actually try to validate it again.
I think what happened is:
That created valid authorizations for all applicable hostnames, using HTTP-01.
Then you did:
It probably reused all of the valid authorizations from before. Certbot set up a TLS-SNI-01 challenge, but Let's Encrypt probably never actually tried to access it.
You can confirm it by carefully examining the voluminous logs in /var/log/letsencrypt/.
"sudo certbot renew --preferred-challenge http-01" should update the renewal configuration files for any certificates that were renewed, but "sudo certbot renew --dry-run --preferred-challenge http-01" wouldn't.