Handling Large Flat file Urgent

, ,
1

Guys,
I am trying to parse Large Flat file. I am using convertToValues in REPEAT loop to parse the file. I understand that I need to EXIT the repeat when ffIterator variable is null. I am not able to set this condition.

Can someone help me how to check ffIterator for null and then EXIT the repeat.

Thanks,
Brain.

2

Brain,

How much large is your flatfile?
Since i have parsed 28-30MB with out any problems using wm.flatfile:convertToValues,i didnt set ffIterator param at all.

Just schedule the service or directly run the service with out step thru.

HTH,
RMG

3

Hi,
My fine is about 20-25 MB. ffIterator is an object. How do you compare if the object is null.

Thanks,
Brain.

4

you can ignore setting ffIterator in the convertToValues,25MB and more size file will straight away can parse…

HTH,
RMG

5

Brain,

You don’t need to explicitly set ffIterator. After your pub.flatFile:convertToValues, set up a BRANCH on ffIterator. Underneath the BRANCH, put in an EXIT with (Label = $null), (Exit from = $loop), and (Signal = SUCCESS).

Make sure your REPEAT is set to Repeat On Success with a count of blank or -1 and your convertToValues has iterate set to “true.”

Dave

6

Thanks Dave.

I got the it working.

7

Thanks Dave.

I got it working.

8

Thanks Dave. I had the same problem in utilizing the ffIterator properly. One look at this thread and your reply has solved it.

9

Thnx dave…

10

Hi Dave,

I am doing exactly same what you have suggested. But in this way, I am missing to process last record of the file as it is exiting from the flow as ffIterator comes null when convertToValues pull last record of the file. How should I fix it? Please suggest.

TIA,
Shweta

11

Either process the last record after the loop is exited, or change your loop logic to put the test to exit after processing the current record.

12

Yeah, I was also thinking of processing that record after exit, just wanted to be sure if there is any better way of doing it.

Thanks for your reply! :slight_smile:

13

Hi RMG,

Can you elaborate when you say "directly run the service with out step thru.

14

I meant via scheduled job instead of manual running service/debug mode from developer:

HTH,
RMG