There is no problem with your certificate or HTTPS connection. Your Let's Encrypt certificate is still fine.
This problem is a web server error. Something needs to be fixed inside your web server configuration.
In particular, the error message "You don't have permission to access / on this server." was served correctly over a secure HTTPS connection authenticated by a Let's Encrypt certificate!