ssl handshake failure

Dear all,

I’m facing a problem to access an https client webservice. I’m using webMethods integration server 9.0.1 on a windows env.

I tried different debug options but the result remain the same.
This is the wrapper.log ssl debug snap:

ssl_debug(2): Starting handshake (iSaSiLk 3.03)…
ssl_debug(2): Remote client:1*.1*.2*.***:443, Timestamp:Tue May 30 12:04:24 CEST 2017
ssl_debug(2): Sending secure renegotiation cipher suite
ssl_debug(2): Sending v2 client_hello message, requesting version 3.1…
ssl_debug(2): Received alert message: Alert Fatal: handshake failure
ssl_debug(2): SSLException while handshaking: Peer sent alert: Alert Fatal: handshake failure
ssl_debug(2): Shutting down SSL layer…
ssl_debug(2): Closing transport…

Does it mean the client I’m trying to call is rejecting my server certificate ? or my server is not trusting their certs ?

I share also some extended settings that may help:

watt.config.systemProperties=javax.net.debug=ssl
watt.net.ssl.debug=true
watt.net.ssl.server.clientHandshakeTimeout=60000
watt.net.timeout=300
watt.security.cert.wmChainVerifier.trustByDefault=true
watt.security.ssl.client.ignoreEmptyAuthoritiesList=true
watt.security.ssl.ignoreExpiredChains=true
watt.ssl.iaik.debug=true
watt.ssl.iaik.serverAllowUnboundRenegotiate=false
watt.server.httplog=true

I also added the exception of the proxy to the URL I need to access (telnet 443 on the URL I need to access is OK)
Finally, I added the root CA of the server I’m calling to my truststore list (in case of…)

I hope the topic is enough detailed to get some help. I’m really stuck with this problem.

Thanks to all in advance.

Best Regards,
Walid J.

Hi Walid,

when invoking an external https url you have to trust the external server certificate (at least their CA chain).

Addtionally check if you have applied the appropriate IS Core and SCG Entrust Fix.

Might be that your are trying to use a protocol version which is not allowed by your partner.

Most servers do not allow any SSL protocols any longer, only TLS potocols.

In Knowledge Base in Empower there a corresponding articles for this.

Regards,
Holger

Hi Roger,

Thank you for your reply. Here is my actual fix lvl:
IS_9.0_SP1_Core_Fix11
Java Entrust Toolkit Version: Entrust Authority™ Security Toolkit for the Java(R) Platform version 8.0 Patch 192190 FIPS

Am I able to check if my partner is using TLS or SSL and which version is he using ?

Is there any exception I can make (in the extended setting somewhere?) that may help me identify the problem ?

To trust the external server certificate, is it enough to add a truststore containing the CA Root of that server ?

What about the SSL debug Log I put in my initial message ? does it give us a hint about which server is rejecting the other ?

Regards,
Walid J.

Hi Walid,

putting a truststore in the IS under Security → Keystores and clearing the SSL cache should be sufficient.
The Truststore only needs the intermediate CA certificates as the Root CA certificate should be in cacerts store of the JVM by default.

You can try to use SSL Server Test (Powered by Qualys SSL Labs) to check your partners server for ssl version.

See https://empower.softwareag.com/sl24sec/SecuredServices/KCFullTextASP/viewing/view.asp?prdfamily=Integration&KEY=113752-15316048 for details on the ssl configuration.

Can you provide the current settings for the mentioned parameters?

  • watt.net.ssl.server.handshake.minVersion=tls
  • watt.net.ssl.server.handshake.maxVersion=tls
  • watt.net.ssl.client.handshake.minVersion=tls
  • watt.net.ssl.client.handshake.maxVersion=tls

If you have OpenSSL available you can try the following command and provide us the outcome:
openssl s_client -connect server.com:443

Regards,
Holger

Hi Holger,

Here is the following parameters when I posted the topic:
watt.net.ssl.client.handshake.maxVersion=tls
watt.net.ssl.client.handshake.minVersion=sslv2
watt.net.ssl.server.handshake.maxVersion=tls
watt.net.ssl.server.handshake.minVersion=tls

Looks like this link https://empower.softwareag.com/sl24sec/SecuredServices/KCFullTextASP/viewing/view.asp?prdfamily=Integration&KEY=113752-15316048 is a private one. I dont have access to it even logged in Empower.

Unfortunately, I dont have openssl but I’ll try my best to install it to test the cmd you provided me.

Regards,
Walid J.

Hi Walid,

sorry for the broken link. Even I am not able to access it directly.

Search Empoer KnowledgeBase for Poodle and check for Article #1760862.

Additionally you can check for https://empower.softwareag.com/Products/Security/poodle.asp in Empower for further informations.

Can you change watt.net.ssl.client.handshake.minVersion=sslv2 to watt.net.ssl.client.handshake.minVersion=tls and restart the server?
Then try again to call your url.

If you are connecting thru JSSE instead of Entrust/IAIK, there are other settings for that.
Informations can be found in the Fix Readme for IS Core Fix

Regards,
Holger

Hi Holger,

I changed the properties as you suggested and made the test (A) again = same result.

Meanwhile, the client provided me a new URL of a testing environment (B). HTTPS protocol too. And I was able to request the webservice without any problem.
The two servers are using the same Chain and Root. Only the public key differs between A and B.

I’ll dig this with my client. This is weird.

I also need to check if I’m using JSSE or Entrust/IAIK on my server. I’ll update soon.

NB: I have the same error when I use SOAPUI to requet the webservice (A). And I was able to request the testing URL (B) without any issue using SOAPUI too

Best,
Walid J

Hi Walid,

I dont think that this is related to the certificates.

More likely the two servers have different SSL/TLS configuration causing the handshake to fail.
Please have your partner check the two server instances w.r.t.

Can you provide the ssl debug log for succesful handshake?
This might point out why the handshake is failing on instance A.

When handshale succeeds, the next step will be the certificate.
When there are issues relateed with the certificates these will be usually reported as ChainVerifierException.

Regards,
Holger

Hi Holger,

I asked the guys on the remote server to check their configuration and firewall access on the port 443

This is the wrapper.log of a successful call on their env. B
| ssl_debug(2423): Starting handshake (iSaSiLk 3.03)…
| ssl_debug(2423): Remote client:10..2.:443, Timestamp:Wed May 31 10:21:28 CEST 2017
| ssl_debug(2423): Sending secure renegotiation cipher suite
| ssl_debug(2423): Sending v3 client_hello message, requesting version 3.1…
| ssl_debug(2423): Received v3 server_hello handshake message.
| ssl_debug(2423): Server selected SSL version 3.1.
| ssl_debug(2423): Server created new session 6A:E9:69:87:0C:00:10:F9…
| ssl_debug(2423): CipherSuite selected by server: SSL_RSA_WITH_3DES_EDE_CBC_SHA
| ssl_debug(2423): CompressionMethod selected by server: NULL
| ssl_debug(2423): Received certificate handshake message with server certificate.
| ssl_debug(2423): Server sent a 2048 bit RSA certificate, chain has 3 elements.
| ssl_debug(2423): Received server_hello_done handshake message.
| ssl_debug(2423): Sending client_key_exchange handshake message (2048 bit)…
| ssl_debug(2423): Sending change_cipher_spec message…
| ssl_debug(2423): Sending finished message…
| ssl_debug(2423): Received change_cipher_spec message.
| ssl_debug(2423): Received finished message.
| ssl_debug(2423): Session added to session cache.
| ssl_debug(2423): Handshake completed, statistics:
| ssl_debug(2423): Read 4429 bytes in 5 records, wrote 418 bytes in 4 records.

Regards,
Walid J.

Adding the openssl log:

[color=darkblue]D:\TOOLS\OpenSSL\bin>openssl s_client -connect instanceA.srv.all
Loading ‘screen’ into random state - done
CONNECTED(00000174)
depth=2 /C=DE/O=SERV Group/CN=SERV Group Root CA II
verify error:num=19:self signed certificate in certificate chain
verify return:0

Certificate chain
0 s:/C=DE/O=SERV/CN=instanceA.srv.SERV
i:/C=DE/O=SERV Group/CN=SERV Group CA
1 s:/C=DE/O=SERV Group/CN=SERV Group CA
i:/C=DE/O=SERV Group/CN=SERV Group Root CA II
2 s:/C=DE/O=SERV Group/CN=SERV Group Root CA II
i:/C=DE/O=SERV Group/CN=SERV Group Root CA II

Server certificate
-----BEGIN CERTIFICATE-----
MIIEJjCCAw6gAwIBAgIDANbxMA0GCSqGSIb3DQEBBQUAMFAxCzAJBgNVBAYTAkRF

ADS9KeTz3J+HPw==
-----END CERTIFICATE-----
subject=/C=DE/O=SERV/CN=instanceA.srv.SERV
issuer=/C=DE/O=SERV Group/CN=SERV Group CA

No client certificate CA names sent

SSL handshake has read 5183 bytes and written 453 bytes

New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
Protocol : TLSv1
Cipher : DHE-RSA-AES256-SHA
Session-ID: BBAE893AF423F06FA627221B418B35832B0E738013FC99D348

Session-ID-ctx:
Master-Key: E2CAE92C49BEC20C2274D329331C246EE665F61085A123EAB7

8EFB643BCE9B4A8B49B948C69E3CD2EE
Key-Arg : None
Start Time: 1496221620
Timeout : 300 (sec)
Verify return code: 19 (self signed certificate in certificate

closed[/color]

Regards,
Walid J.

Hi Walid,

in this case the connection is using TLS V1 as the SSL version 3.1 is an equivalent for TLS V1.

TLS v1.1 and TLS v1.2 are only supported in JSSE mode and when running the IntegrationServer with JVM 1.7 or newer.
The Entrust library being used by SAG is currently not able to handle TLS v1.1 and TLS v1.2.
Might be worth a feature Request to upgrade the Entrust library to become TLS v1.1 and TLS v1.2 aware.

Regrds,
Holger

Hi,

If you are getting two different results when calling the same server, from the same service for two different URL endpoints, then the you should ask the other side what the difference in configuration.

If you are using the same WSD consumer for both endpoints, are you using a Web Service Endpoint Port Alias for each?

User a tool like OpenSSL or KeyStore Explorer to compare both public keys.

Best Regards,

Dear All,

As I told you before, I was facing the same problem with SOAPUI from my local PC. One of my collegue figured out what is going on with all theses handshake failure: AES256 is disabled by default in java 8. To enable it, we had to download the following files JCE Unlimited Strength Jurisdiction Policy Files for JDK/JRE 8 Download and replace the existing local_policy.jar and US_export_policy.jar

Accordign to this, I switched my testing on a 9.9 IS with JVM 1.8. and replaced the same files (ofc I did a backup) under SoftwareAG99\jvm\jvm\jre\lib\security, restarted the IS but unfortunately still getting the issue.

Something still needs to be fixed somewhere. But I dont know exactly where. This is the config I have on my IS9.9:

watt.net.jsse.client.enabledCipherSuiteList=default
watt.net.jsse.client.enabledProtocols=TLSv1,TLSv1.1,TLSv1.2
watt.net.jsse.server.enabledCipherSuiteList=default
watt.net.jsse.server.enabledProtocols=SSLv2Hello,TLSv1,TLSv1.1,TLSv1.2
watt.net.ssl.client.cipherSuiteList=default
watt.net.ssl.client.handshake.maxVersion=tls
watt.net.ssl.client.handshake.minVersion=tls
watt.net.ssl.server.handshake.maxVersion=tls
watt.net.ssl.server.handshake.minVersion=tls
watt.security.cert.wmChainVerifier.trustByDefault=true
watt.security.ssl.client.ignoreEmptyAuthoritiesList=true
watt.security.ssl.ignoreExpiredChains=true

and this is the error log:
ssl_debug(3): Starting handshake (iSaSiLk 3.03)…
ssl_debug(3): Remote client:1*.1*.20.1**:443, Timestamp:Fri Jun 02 10:48:44 CEST 2017
ssl_debug(3): Sending secure renegotiation cipher suite
ssl_debug(3): Sending v3 client_hello message, requesting version 3.1…
ssl_debug(3): Received alert message: Alert Fatal: handshake failure
ssl_debug(3): SSLException while handshaking: Peer sent alert: Alert Fatal: handshake failure
ssl_debug(3): Shutting down SSL layer…
ssl_debug(3): Closing transport…

What did I miss ?

Regards,
Walid J.

Hi Walid,

please check the java.security file in the same directory where the jars are stored.

Hopefully you will find the disabled algorithms at the end of it.

Regards,
Holger

Dear All,

Many thanks for all your replies and ideas.

During my tests, with differents param and changes on the extended settings, JRE security lib etc, I found an input named “useJSSE” in the pub.client:http service.

Setting that to “true” made the call successful.

I tried the same on the second node of our clustered environment (using default setting on that second node) and the call remains successful.

So, I contacted Empower support to ask how to force all routing requests I set on Centrasite to useJSSE when the call goes through Mediator server and solution was to add to the extended settings the following line to the server extended settings:

watt.net.ssl.client.useJSSE=true

Please check the following details given from Empower:


[color=darkblue]IS currently ships with 2 security providers, IAIK and JSSE, first one is deprecated and remains due blackguards compatibility.

When setting watt.net.ssl.client.useJSSE=true all connections will use JSSE except for those services such pub.client:http or pub.client:soapclient which has an optional parameter named useJSSE which can be used to overwrite defaults, in both directions IAIK <> JSSE.

Any other service not having that optional parameter will use system default controlled by watt.net.ssl.client.useJSSE extended setting, which is the case for mediator.

JSSE does support all protocols IAIK does, but not vice versa, however you can limit protocols with java.security file and/or extended settings to limit ciphers, protocols etc.[/color]

Hope this will help some of you facing the same problem.

Best regards,
Walid J

1 Like

Addendum,

recent Builds of Java 8 as well as newer versions beginning with Java 9 contain both sets of the policies and are using the unlimited set by default.
Additionally these builds are using TLS v1.2 by default.

Regards,
Holger