SAP integration - RFC - Large volume issues

[Sudheer_s7]

Hi All,

webMethods integrating with SAP through RFC.

When we call the RFC,through SAP adapter service, we are receiving large volume of data(5 lakh records).

Integration server is getting hung/not responding when the adapter service is executed.

Could you please let me know any possible solution to receive SAP large volume without any issues in wM.

[Holger_von_Thomsen]

Hi Sree,

please provide some version information about IS, SAP Adapter and JCo Libraries.

Please check your IS configuration for proper java memory settings (you might need to increase these values to avoid swapping or OutOfMemory exceptions).

You might want to rethink your implementation approach.

Instead of directly processing the large volume just publish the splitted data to a Messaging provider, i.e. UM or Broker, and then listen to this messaging queue and process the data in smaller chunks.

Regards,
Holger

[rmg]

Yes…Also please consider async approach using pub/sub to process that kind of large volumes records and work with your SAP team as well to send the records as in batches 100 or approx. per each run to the adapter from processing/performance point of views.

Just my few cents!

HTH,
RMG

[Holger_von_Thomsen]

I agree with RMG regarding batched processing.

We have agreed with our SAP Partners that there should not be more than 100 IDocs in one master IDocList when sending IDoc to us.
This is due to the fact that our IDocs can be quite large (about 1.500 fields which will be 150.000 fields in total plus EDI_DC40 control header).

Regards,
Holger

[Sudheer_s7]

Thanks all for your replies.

@Holger,

FYA…

Description webMethods SAP Adapter
Adapter Version - 7.1.0.16.906
Updates
SAP_7.1_Fix14
SAP_7.1_Fix15
SAP_7.1_Fix16
JCo Version 3.0.14 (2015-12-16)
JCo Middleware JavaRfc
JCo Middleware Version 2.2.17
JCo Native Library Version 721.619

After doing some R&D , we understand like we cannot control the incoming records from SAP through RFC in webMethods.
So we followed batch process, where SAP will divide 5Lakh records from their Master tables, into multiple files in a folder and webMethods will pick and process.

Regards,
Sree

[rmg]

Sree…

It sounds like a good plan and you might be using Al11 file based approach !!

Cheers!
RMG

[Christian_Schuit1 (Christian Schuit)]

If this is still a challenge, have a look at this thread:

Large SAP Idocs

Thanks Mahesh/Abhir/MR/Holger, I am about to request to my infrastructure team all SAP fix,version, Jar and lib file going to used for this implementation but still my client/support team like to know about about Q 2 and Q 5 ie Once sap idocs is triggered from sap system, How ESB server/middleware or How do you determine incoming idoc file is large files? If any transient or network breakdown happen in Integration flow then how/where do you store incoming large file for time being? or wh…