Hello @drummin, welcome to the Let's Encrypt community. 
A great place to start debugging Let's Encrypt certificates is Let's Debug
You might also want to consider updating your software including the OS and Certbot; who knows how old of version of OpenSSL that is running.
Not sure if this applies but thought I would at least point it out Email feedback: TLS 1.0/1.1 deprecation and SHA-1 deprecation
and Rejecting SHA-1 CSRs and validation using TLS 1.0 / 1.1 URLs