Did you configure your server to send the intermediate cert?
My Apache config look like:
SSLCertificateFile /etc/letsencrypt/live/$DOMAIN/cert.pem
SSLCertificateKeyFile /etc/letsencrypt/live/$DOMAIN/privkey.pem
SSLCertificateChainFile /etc/letsencrypt/live/$DOMAIN/chain.pem