pub.math: subtractFloats producing incorrect results

, ,
1

Hi All,

I am using webMethods built in service to subtract two float values.I found some incorrect results generating out of it.Please check out the following test results.
Test1
num1:18.08
num2:3.0
result:15.079999999999998

Test2
num1:15.08
num2:3.0
result:12.08

Note: Precision is optional since we are not sure what extent will be there in the input.

Please can anybody suggest what could be the reason for this unexpected behaviour of subtractFloats service.Is there any issue with the service.

Appreciate your response and help on this.

Thanks,
Raveesh

2

Strange :lol:

It gives me the correct results:

Test1
num1:18.08
num2:3.0
result:15.08

Test2
num1:15.08
num2:3.0
result:12.08

Run the service with/ with out passing “precision”.

Precision gives you the Number of decimal places to which the difference will be rounded. The default value is null.

Just my thought - See if you have this settings present on IS watt.server.math.floatOperation.mode
If not reload the WmPublic package from developer/IS :idea: and re-try

If it still does not work write your own java service or contact global support.

Let me know your results after this.

3

I can reproduce your issue on both 8.01 and 9.0.1 version of WM.
once you specify precision, it works fine.
You can just specify a big enough precision for your case.

What’s the reason you can’t set precision?
You may also open a ticket with SAG, to see what they say about it.

4

You may try with precision for example and outputs as expected:

Test1
num1:18.08
num2:3.0
precision:10
result:15.08

Test2
num1:15.08
num2:3.0
precision:10
result:12.08

HTH,
RMG