flat file sorting/grouping

, ,
1

this is my requirement. please provide steps to achieve this.

  1. receive a large flat file from a application.
  2. using tspace to process this large flat file.
  3. transform this flat file into a XML file.

i have to group the fields based on their values

sample
test1|name|part|partname1
test2|name1|part1|partname1
…
test3|name4|part4|partname3
test5|name3|part3|partname1

now I have to group all partname1 values into one group and create a recordList with remaining fields from that line

how can this be achieve ?

2

You can feed the entries into a database table, and use SQL API for grouping.

Using hash table is one of the other ways.

3

database is not possible.

can you provide some guidelines on how to code using hash tables

thanks

4

If you cannot use database tables then you’ll need some scheme that writes records temporarily to different temporary files.

Probably a way to figure out how to best do this without loading the entire file into memory is to think about how you’d do this if you were coding it all in Java. Then take a similar approach but use IS services and techniques for the different parts.

5

If the file (and it’s processing) fits into memory I would use an XSLT service, as XSLT provides grouping of elements. XSLT is also a good alternative if you have greater changes in the structure which are difficult to implement using map steps like movmeent of nodes in the document hierarchy, changes of grouoings etc.

6

Excellent suggestion. The original post mentioned a “large file” but perhaps it is small enough to do what you suggest.