convertToValues with multiple record structures

, ,
1

I have inbound flatfile with header ,Detail, trailer. where all 3 have fixed length but fields definition is different. and Detail can be unlimited. And all header , Detail and Trailer records are Nth filed defined.
Don’t have any record identifier in all 3 records.
Can someone assist how can we use convertToValues here.

2

One approach to consider:

  • Structure things to just ignore the first record. Presumably you’re using a REPEAT so you can tell which iteration you’re on.
  • To detect the trailer record there is likely a field that can be checked for the presence of a value. Regex of /\S/ may work for your case. If a detail field always has it and the trailer never does then that can be your signal that you’ve reached the end.

I assume the header and trailer are padded to the record length so that all records are the same length. If not, that may present challenges.

Another possible approach:

  • Open the file and read the stream yourself to move the file pointer passed the first record.
  • Call convertToValues as needed.
  • If convertToValues fails in a specific way for the trailer record, you can catch that and ignore it.

HTH

1 Like
3

Hi reamon,
thankyou for quick replay.
I have to read one element from header that is mandatory and from Detail I need to extract some data. problem is both detail and header following different structure even through length is same.
do i can define both header and detail in same schema?

4

Yes, you can define both but the challenge is getting the parser to recognize each record type. Without a record identifier of some in the records you’ll only be able to use a “Default Record” layout – which might work depending upon the field boundaries. Would you be able to share some detail about the file records and fields? Perhaps someone will see a way to structure the parsing.

2 Likes
5

Unfortunately don’t have any recordIdentifier. if i want to read all Header , Detail and Trailer do I can define all schema separately and call Header schema for first record from file and Trailer for last record and Detail for remaining Detail records using convertToValues?

6

Possibly. You’ll need to do some trial-and-error to figure out if such an approach will work. The transition from header to detail is easy – only the first record read as the header. The challenge will be determining when all detail records have been read and now at the trailer. No way to know without parsing it. Don’t forget to handle the case where the file has only a header and trailer, with no detail records.

1 Like
7

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.