I am using IBM Cloud Secrets Manager to request public TLS certificates from Let’s Encrypt. My certificates are being issued successfully. The certificate chain I currently get looks like this:
End-entity certificate: RSA 2048
Intermediate certificate: RSA 2048
Root certificate: RSA 4096
When I try to use this certificate chain with AWS (Keycloak exposed on the internet), AWS rejects the chain. Their support explained that:
“On further analysis, we understand that you have three certificates and each certificate is signed by its top-level certificate, where certificate #0 is verified by certificate #1. However, certificate #1 fails during verification for certificate #2. This is because Certificate #2 appears to be 4096 bits (512 bytes) and the expected length was 256 bytes for this key. We have confirmed that this mismatch in the key sizes between certificates in the chain is causing the issue as the signature validation follows OpenJDK's implementation: jdk/src/java.base/share/classes/sun/security/rsa/RSASignature.java at e172e6a4e37cf06afee662625a01193f5528a2b1 · openjdk/jdk · GitHub
So the issue seems to be the mismatch in key sizes between the intermediate (2048) and the root (4096). AWS claims their Java validation fails in this case. This really surprises me, as from I have read so far, it looks like the key sizes in my chain is a common one. I can request an End-entity certificate of 4096 bits, but I cannot find I way to ask for intermediate certificate of the same length.
My question is:
Is there any way to request an intermediate certificate with a 4096-bit key (so that all certs in the chain are consistent)?
Or is there an alternative approach with Let’s Encrypt that would solve this mismatch issue?
That surprises me too, and I suspect that they're misunderstanding something. It's very normal for a chain to include keys not of all the same length.
No, Let's Encrypt doesn't have any 4096-bit RSA intermediates. You can see all their intermediates on their Chains of Trust documentation page.
Well, you can see if their logic works any better with ECDSA certificates & chains, depending on why it is that you chose to use RSA certificates in the first place.
And usually with AWS it's easiest to just use their built-in AWS Certificate Manager certificates, which are at no additional cost if you're only using them within AWS services. Trying to import certificates from some other systems should work, but it may be that their support isn't as familiar with using them. I'm not familiar with Keycloak, but you might want to see if staying within the AWS ecosystem makes things easier for you. (It certainly might not, depending on what exactly your requirements are.)
I agree. Looking at that java code, it's checking that the signature on a certificate is the right length for the pubkey which produced that signature. That's a good check to do, and not at all what that support person was claiming.
Hi Rudy. I guess you mean you want to see them? How would be the best way for me to share them? Checking them directly in the browser could be an option? The public domain is https://dthdevsso02.ibmgcloud.net/
I also asked for a log file but the response from AWS was:
The logs used are Internal Service logs which are not available to the customers.
Besides from what I shared, the only extra info in the support ticket was:
I would like to inform you that our internal teams have reviewed your issue and we understand that you are facing the issue when attempting to create the OIDC provider, which returns an OpenIdIdpCommunicationError, indicating a communication breakdown between AWS IAM and the identity provider.
And:
Recommendation: We would like to recommend having certificate signatures that are of the same size as the signing public key.
As mentioned, AWS response so far was that it was a problem with the key sizes of our certificates, but we cannot see how that's an issue and you all have very kindly confirmed our thoughts. I contacted AWS support again to see if they can dig further into this.
As suspected by everyone who commented in this thread, the problem wasn’t the key length. The problem was that, for some reason, our certificate provider sometimes deliver the root certificate, bundled with the intermediate certificate. We were not aware of this, so the key chain we were using had the intermediate certificate twice, which was causing the error “Chain issues - Incorrect order, Extra certs” a validation exception in AWS IAM application. After deleting the duplicated intermediate certificate from the key chain, everything worked as expected. Thank you very much for all of you who gave us their opinion on this matter.