Server D-H key verification failed - Java Error

Hi,

I have created the SSL cert for the domain example.com and intsalled it. It's working fine. I have configured Java on another host. When I'm trying to connect to the domain example.com from the remote host using java, I'm getting below error.

2018-07-30 04:57:13,478 [ERROR] [http-nio-8081-exec-5] [memdeskLogger] [38dd6d6c-4e46-448b-ba2c-cd6e3832c858_1532941033453] - Server D-H key verification failed
sun.security.ssl.HandshakeMessage$DH_ServerKeyExchange.(HandshakeMessage.java:858)
sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:272)

I have downloaded the Root CA of LetsEncrypt from wget https://letsencrypt.org/certs/lets-encrypt-x3-cross-signed.der and added it to the Java trusted certs. And the domain backend servers using tomact as web server.

So far, I have executed below commands.

    1)wget https://letsencrypt.org/certs/lets-encrypt-x3-cross-signed.der
    2)sudo keytool -trustcacerts -keystore $JAVA_HOME/jre/lib/security/cacerts -storepass changeit -noprompt -importcert -alias lets-encrypt-x3-cross-signed -file lets-encrypt-x3-cross-signed.der 
    3)keytool -list -v -keystore $JAVA_HOME/jre/lib/security/cacerts and the results shows the letsEncrypt cert added to the java cacerts.
    4)sudo /var/juno/bin/rc stop tomcat-memdesk
    5)sudo /var/juno/bin/rc start tomcat-memdesk

When we made transaction from the remote host to the domain which have letsencrypt configured, then it throwing below error.

2018-07-30 04:57:13,478 [ERROR] [http-nio-8081-exec-5] [memdeskLogger] [38dd6d6c-4e46-448b-ba2c-cd6e3832c858_1532941033453] - Server D-H key verification failed
sun.security.ssl.HandshakeMessage$DH_ServerKeyExchange.<init>(HandshakeMessage.java:858)
sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:272)

I'm not getting this error if tried to connect to other domains that have GoDaddy SSL cert configured. What made the error? Can anyone please help me.

I'm using below java version.

openjdk version "1.8.0_171"
OpenJDK Runtime Environment (build 1.8.0_171-b10)
OpenJDK 64-Bit Server VM (build 25.171-b10, mixed mode

I’d suggest that probably there is not enough information here to make a diagnosis.

Have you tried connecting with openssl s_client and seeing what happens?

openssl s_client -connect tomcat.srv:443 -servername tomcat.srv -debug

If s_client is able to successfully establish a TLS connection (both locally and remotely), then it would help narrow things down a bit - or reveal an error.

Yeah, I have tried it both locally and manually. I’m able to make a connection. I believe the issue is about Java.

Well, Java 8 doesn’t need anything additional to be able to talk to hosts with Let’s Encrypt certificates. Importing the intermediate into the trust store is not needed.

Can you run this program, adding your Tomcat hosts to HOSTS_TO_TRY? It would verify your system’s ability to establish a TLS session with the remote Tomcat.

if it works fine, then the Java API is being misused somehow - or the changes you made to the trust store have broken something.

public class LetsEncryptConnectTest {

  static final String[] HOSTS_TO_TRY = { "untrusted-root.badssl.com", "letsencrypt.org" };

  public static void main(String args[]) {
    for (String host : HOSTS_TO_TRY) {
      System.err.printf("--- Trying %s\n", host);
      try (final java.net.Socket s = javax.net.ssl.SSLSocketFactory.getDefault().createSocket(host, 443)) {
        final javax.net.ssl.SSLSession session = ((javax.net.ssl.SSLSocket) s).getSession();
        
        final java.security.cert.Certificate[] cchain = session.getPeerCertificates();
        System.err.println("The Certificates used by peer");
        for (int i = 0; i < cchain.length; i++) {
          System.err.println(((java.security.cert.X509Certificate) cchain[i]).getSubjectDN());
        }

        System.err.println("Cipher is " + session.getCipherSuite());
        System.err.println("Protocol is " + session.getProtocol());
      } catch(Exception e) {
        System.err.printf("Host %s experienced an error: %s\n", host, e.toString());
        e.printStackTrace();
      }
      System.err.println();
    }
  }
}
$ javac LetsEncryptConnectTest.java && java LetsEncryptConnectTest
--- Trying untrusted-root.badssl.com
Host untrusted-root.badssl.com experienced an error: javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
        at sun.security.ssl.SSLSessionImpl.getPeerCertificates(SSLSessionImpl.java:440)
        at LetsEncryptConnectTest.main(LetsEncryptConnectTest.java:11)

--- Trying letsencrypt.org
The Certificates used by peer
CN=www.letsencrypt.org
CN=Let's Encrypt Authority X3, O=Let's Encrypt, C=US
Cipher is TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
Protocol is TLSv1.2
1 Like

Are you sure you’re using Java 8 on the client side as well? You shouldn’t need to manually install the root certificate with Java 8, as it was included with update 101.

If you’re using Java 7 or Java 6 client-side, they have known issues with DHE key exchange you’ll need to work around in your server.

Thanks much. The program you provided helped me to a lot.

At last, Identified the root cause. We didn’t enable the D-H key verification on the load balancer for that domain example.com that have configured with letsEncrypt cert. (my bad)

Because of that, Java was not able to do the key verification.

Still, the browser showing the domain as https enabled but in the back end, because of key verification disabled the data can’t flow in an encrypted way (in a secure way). Am I right?

–
Rajesh

You have a load balancer? Maybe you are right and you have selected a ciphersuite/public key combination on the load balancer that Java does not support. Whether Diffie-Hellman is used or not is, and what type of DH handshake takes place, comes from the ciphersuite.

A browser and Java 8 may not have the same overlap of ciphersuites. The “cipher suites” table on https://docs.oracle.com/javase/8/docs/technotes/guides/security/SunProviders.html#SunJSSEProvider contains what I believe is the list for both Oracle and OpenJDK 8. DH is denoted by ‘DH’, ‘EDH’, ‘EDHE’, etc.

If you can share how the load balancer is configured (i.e. what ciphersuite it is configured for, what type of key the certificate uses), that would help.

Yeah, we have load balancer. We install the certificate for the particular VIP on LB so that if anyone tries to hit that domain. The request will comes to LB and verifies the SSL configuration. Based on that, the connection will establish.
and generally When you request a HTTPS connection to a webpage, the website will initially send its SSL certificate to your browser. This certificate contains the public key needed to begin the secure session. Based on this initial exchange, your browser and the website then initiate the ‘SSL handshake’. The SSL handshake involves the generation of shared secrets to establish a uniquely secure connection between yourself and the website. After handshake completed, the data flows b/w browser and the web server is in encrypted.

We are using following cipher suites:

TLS1-DHE-RSA-AES-256-CBC-SHA
TLS1-DHE-RSA-AES-128-CBC-SHA

and RSA as key exchange mechanism.

Java 8 shouldn’t have a problem with either of those ciphersuites, but I don’t have a NetScaler device to test with.

Yeah we have configured those cipher suites but disabled it for that particular domain and that made the issue.

Now we have enabled it and things are working fine.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.