Https troubles with Reverse Invoke server and self signed certs

, ,
1

I have been unable to find any posts specifically about this scenario but I’m hoping someone will have the insight to see what I’m doing wrong. I have no prior SSL experience and am pretty sure I am just stumbling on setting up my certs properly. I’ve got two 6.5 Integration Servers set up; one in the DMZ and one internally. I have a SOCK reverse invoke connection setup from the internal out to the DMZ and have Http connections from external to the DMZ machine working. I am now trying to introduce Https into the fold. I have created a private key using openssl on my DmzIs and created a certificate from that.

openssl req -x509 -nodes -days 3650 -newkey rsa:2048 -keyout ./keys/dmzPrivateKey.key -out ./certs/dmzSelfSignedCertificate.crt

On the DmzIs I have nothing set up in Security → Certificates. Under my Https Port I have the following configuration


HTTPS Listener Configuration
    Client Authentication	 None
Listener Specific Credentials (Optional)
    Server's Certificate	/DmzIs/IntegrationServer/config/ssl/certs/dmzSelfSignedCertificate.crt
    Authority's Certificate	/DmzIs/IntegrationServer/config/ssl/certs/dmzSelfSignedCertificate.crt
    Private Key	/DmzIs/IntegrationServer/config/ssl/keys/dmzPrivateKey.key
    Trusted Authority Directory	/DmzIs/IntegrationServer/config/ssl/certs

I have also enabled SSL debugging as outlined in this post: How to enable SSL handshake debugging for IS

When I enable the Https port I see this message in the server logs

[ISC.0006.0008D] Listener HTTPSListener@XXXX loaded certificate authorities from location /DmzIs/IntegrationServer/config/ssl/certs

I have a Java application that I coded to test the port for Http and have modified it for use with Https. I have imported the dmzSelfSignedCertificate into a keystore and use the keystore while connecting to the DmzIs. When I try to connect from java I see this in the server log:

[ISP.0047.0018V1] Aborting connection from 11.194.109.94/11.194.109.94:

In the client (java) output I see this:

javax.net.ssl.SSLException: Received close_notify during handshake
	at com.ibm.jsse2.n.a(n.java:38)
	at com.ibm.jsse2.sc.a(sc.java:442)
	at com.ibm.jsse2.sc.a(sc.java:563)
	at com.ibm.jsse2.sc.b(sc.java:8)
	at com.ibm.jsse2.sc.a(sc.java:381)
	at com.ibm.jsse2.sc.g(sc.java:437)
	at com.ibm.jsse2.sc.a(sc.java:544)
	at com.ibm.jsse2.sc.startHandshake(sc.java:124)
	at com.ibm.net.ssl.www2.protocol.https.c.afterConnect(c.java:48)
	at com.ibm.net.ssl.www2.protocol.https.d.connect(d.java:39)
	at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:899)
	at com.ibm.net.ssl.www2.protocol.https.b.getOutputStream(b.java:47)

In my std err output on DmzIs I see:

ssl_debug(1): Starting handshake (iSaSiLk 3.03)...
ssl_debug(1): Received v3 client_hello handshake message.
ssl_debug(1): Client requested SSL version 3.0, selecting version 3.0.
ssl_debug(1): Shutting down SSL layer...
ssl_debug(1): Sending alert: Alert Warning: close notify
ssl_debug(1): Closing transport...

My current train of thought is that perhaps I need to use SSLSOCK for the reverse invoke connection or to have certificates set up between the DmzIs and InternalIs.

Has anyone been down this road before and have any tips? They would be greatly appreciated :smiley:

thanks,
Mike

2

Hi Mike,
As per my knowledge you need to follow the following points:

  1. Copy private key of your self signed certificate into the CA cert location of the IS and RI.
  2. you need to create another certificate for your client and private key of that certificate you need to configure into your java client and configure its public key to the IS and RI.
  3. Copy the CA cert of Java client to the CA cert location of the IS and RI and leaf cert to the cert location and map this certificate to the user name (Java Client).

Now what happens when you try to connect to RI/Is using your java client, first server gets authenticated using IS/RI public key then server exchanges the public key which to authenticate your java client. Once authentication is done then your java client is allowed to do the transactions.

Thanks,
Vikas

3

Thanks for the response Vikas. I tried what you suggested and I’m still encountering the same issue. That being said I have taken a few steps backwards and set up an Https port on our internal Integration Server and have tried communicating directly with the internal server via my Java Client.

I was still encountering the same issue with this new https port so I switched the client authentication mode from Request to None and started including basic authentication credentials in the request header. Even with client authentication set to none and credentials in the request header I am still seeing this in the SSL debug log.


ssl_debug(1): Starting handshake (iSaSiLk 3.03)...
ssl_debug(1): Received v3 client_hello handshake message.
ssl_debug(1): Client requested SSL version 3.1, selecting version 3.1.
ssl_debug(1): Shutting down SSL layer...
ssl_debug(1): Sending alert: Alert Warning: close notify
ssl_debug(1): Closing transport...

It is starting to look like it isn’t an issue with the certificates after all (and definately not related to the reverse invoke setup). My hunches are JVM compatibility issues, ssl library issues, other?

Anyone else run into this in the past?

Thanks in advance,
Mike

4

Hi Mike,
If you have followed the steps as i mentioned then no issue in configuration part. From logs it seems to me that problem could be with SSL version. Which version of WM you are using?

Thanks,
Vikas

5

Hi Vikas,

The integration server is version 6.5 sp3. It looks like it is could be an issue with the JVM that we are using for our integration server. I found KB article #1719865 yesterday that indicates we need to apply the JCE Unlimited Strength Jurisdiction Policy Files to our JVM since we aren’t using the one that came with the integration server. I’m having our technical systems team apply the policies. I will reply later to indicate if it was successful or not.

Thanks for your help,
Michael

6

This was indeed a JVM issue. We tried first applying the JCE Unlimited Strength Jurisdiction Policy Files to our JVM which had no effect. I then tried switching back to just using the JVM that came with our IS and SSL began working. Fortunately there was no reason for us to not use JVM that shipped with the IS. So this is my solution for the time being.

– Mike