Running out of heap space

, ,
1

I’m trying to run a simple query against a real-world XML file (about 600,000 lines) and I’m running out of heap space, with the following message:

d:\quip\quip.exe: fatal error: Windows programs can only use 256Mb of heap; sorry!

Is this a limitation of the Java Run-time engine under Windows, or a limitation of Quip? (I’m running under Windows 2000, using JDK 1.4.0 and Quip 2.2.1.1).

Any suggestions?

Thanks.

2

Hello David,

I am not sure if it is still relevant, but the information in the following posting may be of interest to you: heap full

Greetings,
Trevor.

3

Thanks. I’m not getting my data from a Tamino database, or any other database, so I guess there’s not much I can do to limit the amount of data, and if I understand the replies to the other post, there’s nothing I can do to increase the heap size, either. I think all I can do is run the data through some XSLT processor first, to filter out unnecessary data. But in that case, I may not need to use Quip anyway, I can probably do everything in XSLT.

Am I correct in thinking that Quip reads the entire document into memory, rather than processing it sequentially? My query is actually very simple, and could be processed sequentially, if the tool works that way.

4

600,000 lines of XML sounds far too big for an in-memory processor like QuiP. You need a database for this kind of data!

I’ve seen XSLT processors (which also have an in-memory architecture) run successfully on 50Mb source documents, but you need an awful lot of memory (typically 10-20x the raw document size) and a lot of patience.

Michael Kay