Certbot fails to create the challenge file

To add a bit more information: The directory whale.seas.upenn.edu/.well-known/acme-challenge/ is visible on the internet as can be verified using either http or https (there is no forwarding).

While certbot was waiting for the challenge, I verified continuously that no fioles were created in the above directory.

I have direct physical access to the server.

For me, certbot stopped to work about a year ago after an OS upgrade or certbot update and I was never able to make it work. I used ZeroSSL for a while but now have ran out of free certificates.

I have found many related discussions on the internet by all seem to address very specialized cases that are not directly applicable to me.

Would be very grateful for help.