Out of Memory Error

, ,
1

Hi,

I have an 65 MB file that I am trying to process. I am able to read the file as an stream using getFile. After getFile I am able to call xmlStringToNode to get a node representation of my document- note i also use the filestream parameter when calling xmlStringToNode.

The structure of my file is as follows:

<?xml>
<vendordata>
<vendor> … </vendor>
…
…
…
<vendor> … </vendor>
</vendordata>

This file has thousands of vendors’ data. I wanted to retrieve the SenderID and the ReceiverID from the root node (VendorData), so i called queryXmlNode with the following XQL queries:

/VendorData[0]/@SenderID
/VendorData[0]/@ReceiverID

For a small file the above query works, for my 65 MB file I get Out of Memory. I thought using the stream parameters for my services was suppose the keep the file on disk until the actual data is needed. Please advise.

Additionally, if i omit the call to queryXmlNode and just iterate over the Person nodes using the getXmlNodeIterator and the getNextXmlNode services I am able to process all the nodes. I get no out of memory error. I have not tried using queryXmlNode to look up any Person nodes.

Thanks in Advance.

  • nilesh
2

Nilesh,

Just an update,I believe Streams will not keep the file on disk,but it will consume less memory during runtime(Java InputStream,BufferStream reading context),so the performance will be faster.And your approach of processing largedocuments using NodeIterator functionality is on track.

Regards,

3

RMG,
Thank you very much for update.

But I have not yet figured it out why its failing with “Out of Memory” when i try to process large file using NodeIterator even.

Regards,
Nilesh