Hi all! I have an integration that was designed to handle large volumes of small documents. The integration receives PAYEXT IDOCs from SAP into partner manager, the trigger service picks up the IDOC and maps it to our Payment canonical, publishes it to the broker, subscriber on the external IS puts the canonical into TN, TN processing rule triggers the mapping service to create the externally facing format (in this case EDI X12 820 document), envelope, deliver to TN to a public queue for delivery to the trading partner. This integration was designed for small documents and immediate delivery.
Unknowingly our users have been in the process of setting up a new vendor to pay thru this integration. This new vendor is a monthly run that creates a huge PAYEXT IDOC. Our typical size is 17k and this IDOC delivered to our production system was over 27 megabytes. Of course it had a major impact on our system when they submitted the IDOC. Our estimates on runtime were 3-6 days with the way it was running before we cancelled the job.
So now here is my dilemma I have only written integrations that handle frequent, large volumes, small size documents. How do I in a single integration handle both small and large efficiently? The initial receiving of this large document is on our internal server that does not have TN so I cannot use TN large document handling functionality to do this, at least not for that piece of it.
Any hints, tips, and/or pointers on where to go to research how I can accomplish this task efficiently will be much appreciated.
TIA,
Dawn