@Osiris after certbot edits nginx configuration, do you know how does it reload the nginx afterwards. Could it be that it reloads the service asynchronously so if reloading takes longer, nginx doesn't manage to apply the changes fast enough? Or do you know anybody that could answer this? Would it be considered a bug?
Inspired by comment from @_az in this thread: Certbot renew with nginx module - returns error 404 for challenge response