The problem is still the same as in the beginning. This is my first certificate renewing. Something is telling me, I will not manage to finish it on time as the certificate is valid for little more than one day So I will walk through the chicken and egg problem again. Maybe it will show another error which I will be able to cope with...
socket.gaierror: [Errno -3] Temporary failure in name resolution
I am not sure why yours just says "Try again" but the error is likely a DNS resolver issue. I don't know why Certbot / python would fail but your curl requests succeed.
Is there anything unusual with the way Certbot is installed compared to how you run curl?
I am not expert enough in python for debugging such problems. Other volunteers may know better.
My only recommendation is to try using a different acme client and preferably one not based on python. Maybe acme.sh since it's popular and bash based and should run in same environ as your curl tests.
I will have a look at acme.sh and also go through results for socket.gaierror "Temporary failure in name resolution".
$ host acme-v02.api.letsencrypt.org
acme-v02.api.letsencrypt.org is an alias for prod.api.letsencrypt.org.
prod.api.letsencrypt.org is an alias for ca80a1adb12a4fbdac5ffcbc944e9a61.pacloudflare.com.
ca80a1adb12a4fbdac5ffcbc944e9a61.pacloudflare.com has address 172.65.32.248
ca80a1adb12a4fbdac5ffcbc944e9a61.pacloudflare.com has IPv6 address 2606:4700:60:0:f53d:5624:85c7:3a2c
So finally after a lot of struggle and thanks to your support guys! I was able to renew the certificate a couple of hours before it expired
The problem was in the DNS resolution. I was new to this host environment and MikeMcQ's piece of advice about looking for temporary failure in name resolution and checking anything unusual with the way Certbot is installed on the host got me on the track to the solution.
I found that the "certbot renew" is actually executed through a script that uses the "docker compose" subsystem of the host. There was a problem with DNS resolution in the certbot docker container which I fixed by appending "dns: 8.8.8.8" to the appropriate section of the "docker-compose.yml" file like this: