Best Approach to Compare Two large Files and find Delta

, ,
1

Hi Experts ,

Can you please tell what will be the best approach to compare two largefiles(flat files having size>300mb)
and report delta .

Please also let your views on whether this comparison should be done on ESB layer or not ?

2

Rakesh – Till wM8.2 we don’t have any utility service to achieve this, you can write java service to get your work done. Please let me know if you need something more.

Thanks,

3

The easiest way is to compare is to to check the byte size of each file and compare them. When you say Delta, are you looking for the actual differences? then you have to actually compare contents line by line. In Java 8 use can leverage the NIO Framework for this.

“Please also let your views on whether this comparison should be done on ESB layer or not ?”
It depends on what are you trying to do. What is the use case here?

4

Hi Akki ,

Thanks for the response. Yes I am looking to get the actual differences and prepare a delta file of differences.

The only thing that I have to do , is to compare the two files and prepare the delta file which will be used another system for further processing. So, I wanted to know instead of doing it in ESB if we can do the comparison outside the ESB . Will it be good to compare two large files(as it will be memory intensive and impact other running processes) in ESB.

5

Hi Rakesh,

if possible I would prefer to do the comparison outside of the IS i.e. by using os commands.

On Unix:

diff file1 file2 > delta

Regards,
Holger

6

I would suggest to do this outside webMethods. You can raise a feature request in SAG brainstorm :slight_smile: