large result sets

I am using a servlet running under tomcat to access tamino. Tomcat is run in a 200MB partition.

When I do a query that will get a large number or hits >40,000 I get an out of memory message from tomcat. If I use the Tamino Interactive Interface I get the results properly.

My guess is that I need to use a cursor to eliminate the large result set being returned all at once. I can’t find a good example of how to do this.

Has anyone run into this problem and can show me a possible solution. The code I am using go do the query is as follows:

public void performQuery() {

try {
System.out.println("QueryBil.performQuery – _baseQuery = "+_baseQuery);
TQuery query = TQuery.newInstance(_baseQuery);
System.out.println("QueryBil.performQuery – Collection: " + accessor.getAccessLocation().getCollection());
TResponse response = accessor.query(query);
System.out.println(“QueryBil.performQuery – query preformed”);
_XMLObjectIterator = response.getXMLObjectIterator();
System.out.println(“QueryBil.performQuery – got object iterator”);
_count = traverseResultsGetCount();
}
catch (Exception e) {
e.printStackTrace();
}
}


It is pretty basic and straight forward

Hello there.

To retrieve results with cursoring you can use the other query method of TXMLObjectAccessor - the one which has the second parameter “int quantity”.

There is a catch - you must be running in LocalTransaction mode. Here is an excerpt from the JavaDoc:


For more information on switching into Local Transaction mode, see the documentation.
This page may be a good starting point:
   /Help/ADVCONC/TRANSACTIONS.HTM#Transactions

The API’s JavaDoc for TConnection will also be helpful.

Hope that helps,
Trevor.

Trevor,

I ahve looked at that but am a bit confused about being able to jump to arbitrary parts of the result set. The user interface I am creating allows the user to move forwards and backwards through the result set by pages or by jumping to a specific page or record. I can’t figure out how to do this with the commands you have listed.

You can see the prototype at:
http://acer.harvard.edu:81/ted/servlet/deliver?_collection=Bil

search for mice and you will see how it works

Hello there.

If you use the getXMLObjectIterator() method from the TResponse, you can step backwards and forwards through the individual result documents - but not through pages.

Perhaps someone else knows how to accomplish this in a page based manner.

Or maybe you can work with this mechanism and just assemble multiple result documents onto a page as required. (Though this would not allow for jumping to particular pages.)

Sorry that this isn’t much help,
Trevor.

Trevor,

Unfortunatly the system runs out of memory at the line "TResponse response = accessor.query(query);"

which I believe means that the response object being returned is too big. This is why I think that a cursor must be necessary.

Any advice on how to use a cursor for this purpose?

public void performQuery() {

try {
System.out.println("QueryBil.performQuery – _baseQuery = "+_baseQuery);
TQuery query = TQuery.newInstance(_baseQuery);
System.out.println("QueryBil.performQuery – Collection: " + accessor.getAccessLocation().getCollection());
TResponse response = accessor.query(query);
System.out.println(“QueryBil.performQuery – query preformed”);
_XMLObjectIterator = response.getXMLObjectIterator();
System.out.println(“QueryBil.performQuery – got object iterator”);
_count = traverseResultsGetCount();
}
catch (Exception e) {
e.printStackTrace();
}
}
null

I think that TXMLObjectAccessor is more useful when you need to work with particular document. For example to update it according to user input.

If you want to display search results in a table, TStreamAccessor works great. The first advantage that it has methods for page support. Plus you don’t even parse result, you just apply a stylesheet to a stream and output is ready.

Some technique is required to move from one page to another. I use sessions to save the context of connection and I pass cursor ID through parameter.

Check openCursor and fetchCursor methods.