I am working on an EAI project (webMethods 6.01 platform) that involves transporting large volumes of data “near real time”, which has been defined as “from source to target resource in 15 minutes or less.” I am trying to determine the optimal document size for IS to publish to the Broker. I crunched some numbers, and here is what I’ve found:
All of these tests were with guaranteed documents. Volatile documents yielded slightly better results, but nothing to get very excited about. These numbers suggest that the optimal document size for publishing to the Broker is about 1000 KB. I began seeing drastic diminishing returns as I increased the size from 1000 KB.
Also, IS and Broker are both installed on the same Sun Solaris 9 box.
Was the Broker designed to handle very large document sizes? Do these these numbers add up to you guys?
The Broker is designed as a message-router and shouldn’t be troubled with large document sizes, but I remember days that importing to the broker (lower version) had a I believe an 8 megabyte limit on documentsize (they have changed that thank wM). I always try to keep only the very necessary in guar-docs and keep their size as limited as possible for the best performance. I wouldn’t say broker has not been designed for processing large documents, but I would say that broker was designed for performance, that means the smaller the data is fed, the better it is consumed by it.
Ryan,
Your numbers may not represent correct picture. I have seen better throughput. If your numbers are correct that IS may be limiting you performance (it has to keep everything in memory). Are you also doing any processing at IS side?
Also make sure that you turn off client side queuing. IS will write any guaranteed documents to its own local store. That may also be adding to delays.