We were getting a new interface where we receive the flat file (around 30 MB) over HTTPS. We were planning to route the flat file content to TN and process it further from there. (We have large document handling configured in TN and the file size of 20 MB will be considered as large file).
When IS service is invoked from processing rule, flat file content part is retrieved using ‘getContentPartData’ as a stream and passed for flat file parsing. During flat file parsing if we enable 'iterator’to process top level records, will it only load the chunk which is used for parsing into memory or will it maintain the whole flat file content in memory? As per documentation, regardless of file size it looks FF adapter will maintain all FF content in memory.
What will be acheived out of enabling the iterator during FF processing? Is it only the ffvalues which is dropped out of pipeline for that chunk saves some memory?
As well IS document generated out of parsing 30 MB ff has to be carried over the broker for target side processing. Will there be any issues in carrying a huge document over broker?