Generating a SINGLE document after Large Flat File Parsing a

, ,
1

Hi
I am having a problem in handling Flat Files of Larger size. The size of flat file is round about 70 MB. I want to create a document out of it after parsing this flat file, and have to publish this document.
Is there any solution to two of my problems.

  1. Firstly I cannot parse the Large file to a single publishable Document, it gives Out of Memory Exception.
  2. Secondly the Despatcher gives Out of Memory Exception when I try to publish a Large Document.

A. What am I Parsing ?

Flat file of 70MB size, comprised of about 130,000 records (each record consisting of 600 characters)

B. What I need to do ?

Parse the file and generate a SINGLE publishable document out of it.
Publish that large document.

C. Version of webMethods

6.0.1

D. System Specs

Processor: 2.6 GHz
Main Memory: 512 MB

Any specific configurations ??
Any proper solution provided by webMethods itself for such problems ??

2

On wM 5.01 we use a Flat File I/O Adapter to parse a 40 MB file. To do this we had to set the heap size to at least 128MB.

3

You need a heap size at least triple your document size, and avoid using Unicoode strings in your document type definition. It’ll still be flaky though