I am wondering how the webMethods architecture works when Trading Networks database is queried from the tn.query folder in the WmTN package or queried from MWS. I have seen in the Trading Networks Administrators Guide that it mentions that a certain number of query results are stored in a session object on Integration Server, and the rest are stored in Ehcache. Is this always the case?
Specifically I want to know when I query Trading Networks documents and use pagination to process all records, I’m wondering what the server impact will be. Say that I run a query that contains 1,000,000 records, and I return an enumeration ID to process the 1,000,000 records by 50 records at a time, are the remaining records that have not been processed stored somewhere? For each page that I return, is the service reaching out to TN every time, or is it storing the remaining records in memory as mentioned above? Also, any recommendations on increasing performance with large TN queries so that I do not inhibit Integration Server performance when querying TN?
Hi,
Query results that are stored in the IS session can be configured using the property
tn.query.threshold. Remaining records are stored in ehcache. If you see TNQueryResults cache
configuration, by default, the number of elements stored in heap is 500. Remaining entries are stored
in the disk with default value set to 5000. You can configure the cache depending on the
environment specifications. Also, to improve the speed of query execution, it is always advisable
to configure the TN property tn.query.maxrows. You can refer the TN Administrator guide for
details of the above mentioned properties.
Regards,
Anil
Hi,
Thanks for your response. All of that part of the architecture makes sense.
What does not make sense to me is when I query a high volume of documents from the TN database, I see erratic behavior. For example, if there is a high volume of documents, I query TN documents, try to page through the results, and then after a small number of pages, no more results are returned. I am assuming this has to do with some memory or caching issue. From the built-in services, I do not receive any error when this happens, but from MWS, I receive this error:
9999:wm.tn.portal.queries:getQueryResults encountered execution error. Error encountered while fetching search results, page 3 requested is not found for queryId=xxxx. Contact your administrator to optimize the configuration of the TNQueryResults cache.
I have altered page size to match the tn.query.threshold value of 50, and that actually seems to be the worst page size value as it does not return any results after a couple pages of data.
How can I ensure that when I have large volume, I can page over the data without the cache expiring or returning no results for some reason? Also, shouldn’t there be a backup where it reaches out directly to the database if something like this occurs?
Thanks,
Aaron
Hi,
Thanks for your response. All of that part of the architecture makes sense.
What does not make sense to me is when I query a high volume of documents from the TN database, I see erratic behavior. For example, if there is a high volume of documents, I query TN documents, try to page through the results, and then after a small number of pages, no more results are returned when there are millions of documents in the TN database. I am assuming this has to do with some memory or caching issue. From the built-in services, I do not receive any error when this happens, but from MWS, I receive this error:
9999:wm.tn.portal.queries:getQueryResults encountered execution error. Error encountered while fetching search results, page 3 requested is not found for queryId=xxxx. Contact your administrator to optimize the configuration of the TNQueryResults cache.
I have altered page size to match the tn.query.threshold value of 50, and that actually seems to be the worst page size value as it does not return any results after a small number of pages of data. If I were to set tn.query.maxrows to a finite number, I don’t believe there would be a way for me to return all the results I need without looking at timestamps and executing multiple queries.
How can I ensure that when I have large volume, I can page over the data without the cache expiring or returning no results for some reason? Also, shouldn’t there be a backup where it reaches out directly to the database if something like this occurs?
Thanks,
Aaron