IS crash resulted in duplicate data to SAP

, ,
1

Hi, We have an interface which is implemented using process model which receives flat file as a input. Each line in the flat file should be mapped to Idoc and inserted. To enable easy monitoring, this whole flow was designed into one process model where one step maps the flat file data to Idoc list, and other step basically loops over the Idoc list & inserts into SAP system.

One day our IS crashed (JVM crash), when the process instance was running at the step where the Idocs were getting posted to SAP system. Due to this crash, half of the Idocs from the list were posted to SAP and remaining were pending. When the IS was brought up, the process instance got resumed and started inserting the Idocs again from the beginning of the list which resulted in duplicates at SAP.

Do we have any way to control this duplicate issue? Re-designing the interface flow to have a separate thread for each line insertion into SAP will lead to more effort during daily monitoring, as the no. of lines in the flat file is high.

Is it possible to insert Idoc list into SAP like all or none?

2

By default process model triggers when deployed are not configured as exactly once.

If thats a requirement for you, you should change triggers configuration.

Refer to [url]http://techcommunity.softwareag.com/ecosystem/documentation/webmethods/wmsuite8_ga/Integration_Server_and_Process_Engine/8-0-SP1_Administering_Process_Engine.pdf[/url] page 34.

I think wM 7 is the same, still take a look to the proper version documentation you have.

3

You might also investigate the check-pointing functionality that is provided that can be used to pick up where things left off.

4

Check out duplicate event detection property of Process engine…

5

Rob, Can you please give me some more information on "check-pointing functionality "?

6

I assume you’re doing so already, but if not, log the process to the step level.

Refactor the process the loop isn’t a single step. Let the model control the loop, maintaining an index to the IDoc list. The “insert 1 IDoc” step accepts the full list and an index. This basically creates a “checkpoint” for each insert, saving the index value for the step input to the audit logs. The saved pipeline will have the value of the last item inserted (if the step completed successfully). You can then resubmit the step, editing the pipeline if needed to indicate which index from which to continue.

7

Hi,

Thanks DevNull43 & shahid for your suggestions on detecting the duplicates. We have implemented the same and tried replicating the IS crash case and as expected the PRT blocked the document successfully saying duplicate.

Now as next step, we were working on Rob`s suggestion to implement the ‘Check point functionality’ to reprocess the failed instances to process only the Idoc documents which were not inserted into SAP (avoiding the duplicates).

Will keep you all posted for the updates.

– Ravi.