Trying to get an idea of everyone’s IS scheduler tasks. Not sure if there’s limitation of how many IS schedule tasks one can create but I’m about to create 60+ scheduler tasks. Also, each tasks will be executed every 30 seconds so I want to be sure it doesn’t “overlap”. I recall that’s a flag in the schedule tasks but I do not see it. Please let me know how I can enable it not to “overlap”. Basically, each task will get data from MQ (without a listener) and invoke a IS flow servie to publish it to the broker immediately. We’re trying to solve the ordering or sequencing of the messages and MQ does not guaranteed that. Anyways, I’ll have about 60+ schedule task created and each one will get data from each MQ queue and publish it to the broker. Has anyone create many IS schedule tasks and how is it performing for you.
The setting on the task is “Repeat after completion”. You might consider having just one scheduled task (and top-level service) that performs the work for all queues.
You may find [URL]MQSeries.net :: View topic - Does MQ guarantee message ordering? helpful. Whether or not order can be disturbed will depend on the items listed there. If none of these apply in your situation, then there shouldn’t be an ordering issue. And if they do apply, then pulling from the queues and publishing via Broker won’t help. The order in the MQ queues can still be disturbed.
Thanks for the info reamon. This interface we have requires that the order the message receive in MQ is critical. The source application proclaims that they submitted message in the correct order to MQ. Today, we have a listener that pulls the data and publish it to the broker. Since it doesn’t work most of the time, it was decided that a custom java service should be written to simply get the data and immediately publish it to the broker. I still don’t have the level of confidence that this is going to work 100% but this was what suggested. We’re trying to find the best approach to make sure the wM process the message in the order that was received in MQ. If you have any ideas, please do let me know. Thanks.
I assume you’re using the MQ Adapter as the listener.
What has led to the conclusion that things are processed out of order? I’ve no doubt that the source application puts things to the queue in order. The questions are:
- Is the queue maintaining that order (subject to items posted earlier)
- Is threading introduced anywhere along the path, whereby multiple messages are processed in parallel (resulting in a race condition)
If order is important, then only one thread can be processing off of the queue and no threads can be created along the processing path until it reaches the end point.
Even the end point shouldn’t create threads–it should not return “success” to the IS service that is calling it until it is finished.
If you introduce Broker (seems unnecessary with the info given) then the triggers must be configured to be serial.
But as I mentioned, if the order issue is with MQ or the interaction with it in terms of reading from the queues, push them through to Broker doesn’t help with that. I’d focus first on determining where the messages are getting out of order.
Yes, we’re currently using MQ listener. The trigger properties for “Processing mode=serial” and “Max execution threads=1” so should be single threaded. MQ does not guarantee the order however there’s a way you could group them but the source application does not want to provide that to MQ since their single-minded response is “that it worked on another middleware then why can’t you make it work on wM.”
So, we’re faced with an alternative solution instead of using a listener. I’ve created a MQ get adapter service to pull the date from MQ via a java service which then will publish it to the broker. A schedule task will be running every 30 seconds to execute the flow service. My take a this is that it the source application when sending the data to MQ should group the messages so that the middleware can be sure it process them in order or whatever features that MQ provides to guranteed the order in which it received the source application should make it happen. But this need to be implemented in a timely manner and need a good turnaround fix.
I think you’re still back to determining where exactly is order being messed up. If it is messed up by MQ then there is nothing at all that can be done other than “fix” MQ.
If it is the adapter messing it up, then switching from a listener to a get most likely will not address the issue either–my guess is both use the same underlying get.
Perhaps tech support from IBM and/or wM is needed to help isolate the issue. Without knowing the root cause you’re shooting in the dark.
We’re suspecting it’s the MQ adapter that’s getting publish to the broker out of order.
Do you have multiple IS instances connecting to the queue?
We do have a clustered environment.
Check to see if the MQ Adapter is cluster aware. If more than one IS instance is getting from the queue at once, that’s the problem. Even if they alternate turns it introduces a race condition–IS1 picks up and starts processing, IS2 picks up and starts processing and for some reason (load, response time, etc.) IS2 is able to finish before IS1.
Thanks Reamon. How do you check to see if it’s cluster aware?
The documentation should mention it. Or contact support.
If the docs don’t mention it, it probably isn’t.
You would think the MQ adapter is cluster aware provided by SAG.
I reviewed the Installation and User’s Guide for the 6.5 adapter (the latest). It does indeed state that it supports a clustered environment. The section is conveniently titled “WebSphere MQ Adapter in a Clustered Environment.”
In that section it states: “Neither the WebSphere MQ Adapter nor WebSphere MQ guarantee the order of messages received from WebSphere MQ. The application processing the notification must handle any reordering of messages that is required.”
Later, it also states: “It is assumed that the application that processes the resultant notification documents will perform any necessary reordering. The order of the resultant notification documents is not guaranteed in a clustered environment for version 6.1 or later of the Integration Server.”
Implied by this is that it is clustering that introduces the race condition. Each IS instance pulls data from the queue simultaneously, thus you’ll occassionally see out of order processing.
One approach that may work: On just one of the IS instances, enable the MQ listeners. Failover, if needed, would be a manual process.
Can you not use a JMS alias instead? It should allow publish ordering I’d think or else there’s no way a JMS consumer can be in a cluster of any sort?
JMS is just an API, correct? As such, it relies on the underlying messaging implementation. If that implementation doesn’t guarantee order, then the JMS API won’t be able to add it.