Every time that one of our certificates renews it shows via the browser that it's valid and everything is alright with it. Even shows that it has a root certificate of ISRG Root X1. And doing a curl locally to the server is also just fine.
However when we do a server to server curl we're presented with an expiry message:
curl: (60) SSL certificate problem: certificate has expired
More details here: curl - SSL CA Certificates
The machines are no longer using any of the X3 traces and yet it's still failing.
I've tried manually renewing/force renewing it and every outcome is the same.
The only way right now to fix the certificate after it's renewed is to manually delete last block of the fullchain.pem file and it seems to work nice after that.
Perhaps you could give me some tips on fixing the renewal process of the certificates?
There are two versions of ISRG Root X1. A self-signed one (in the browser/system store), and one cross-signed by DST Root CA X3 (eventually sent by the server as an intermediate certificate).
By doing this you are forcing any clients to use their own copy of ISRG Root X1. It can reduce compatibility with older clients but it should be fine.
I can't say more without knowing the answers (what server, os, acme client?) to the questionnaire you skipped when writing this post.
That almost certainly means the requesting server has an out-of-date CA root cert store which does not have ISRG Root X1 in it.
Also, browsers show the chain they built, not necessarily the same as what your server sends. Better to use a site like this to view chains (or openssl).
As @9peppe said, if you provide more details we could be more specific with advice.
Hey guys, thank you for the suggestions and apologies for skipping the questionnaire.
You gave some good tips on the X1 root certs.
I figured a way to solve this issue, though I'm not sure if it's the correct way of doing this. Since it's two debian machines, I went in /etc/ca-certificates.conf and commented out the mozilla/DST_Root_CA_X3.crt entry, and ran update-ca-certificates --fresh command
So now even after force renewal the certificates seem to be doing alright.