Hi,
Is there any setting from which we can specify that serially processing documents are processed exactly after the complete processing of the first one in clustered environment.
Other than configuring Trigger to be serial, you need to ensure that, the Trigger acknowledgement mode is set to CLIENT_ACK.
If you are using JMS topics for pub/sub pattern, you have to set the Order By to ‘Publisher’
-Senthil
It seems like below configuration is for JMS trigger .We are using Broker trigger. Is there anything similar configuration for broker trigger also ?
Serial trigger and ack mode set to CLIENT ACK should be sufficient I guess
-senthil
Umesh – Did you see challenges only by configuring serial ?
Thanks,
@Senthilkumar G : I can find the property of Acknowledgement mode in JMS triggers but unable to locate such option in case of Broker triggers.
@MR as173d : As we are using 2 IS in clustered environment so there is a possibility that documents are extracted from broker in serial manner but were executed parallel in different IS.
My bad. I just noticed that, ACK MODE is not present. In that case, setting it to serial should be sufficient I think.
-Senthil
Also, make sure that the Integration Servers connecting to the Broker use the same client prefix.
Umsh – Have you tested this ? I still doubt that given conditions may not be sufficient. I hope you should control it even in service with some programming. Can you please update your observations. It’s really an interesting scenario.
Thanks,
Why do you think “the given conditions may not be sufficient”? Perhaps I misunderstood the question, but isn’t he simply asking if you can ensure that documents are processed serially in a clustered environment? If so, then yes, this would be sufficient and it’s a pretty standard scenario.
Now, one more thing to take into consideration is exception handling on the subscribing side. If maintaining order is absolutely critical, then if an error occurs when processing a document on the subscribing side, you may want to suspend the trigger so no other documents are processed until the error is resolved. I can get into more details if necessary.
Percio
If someone have clustered environment, can you give a try and update the thread ?
Thanks,
I work with several clustered environments now and I have worked with plenty others before. We use serial processing with many of our triggers and it works as expected. Documents are processed one at a time and in order.
The Publish-Subscribe Developer’s Guide has details about how it works in a section called “Serial Processing in Clustered Environments and Non-Clustered Groups.”
Percio
Hello Percio,
Let’s say a doc arrived to a clustered environment which has 2 IS servers (IS1Node1, IS1Node2) which are been manged by LB. Let us consider a doc arrived to LB at 10.00 AM EST, so, LB handover the doc to IS1Node1 server, it takes 3 hrs to process the 1st doc. At 10.02 AM EST, a second doc arrived to LB, in this specific case doesn’t LB handover the doc to IS1Node2 ? as we set the maxExecutionThreads as 1 in the IS. If I understand correctly, the second doc should go to IS1Node2 at 10.02 AM EST when the IS1Node1 is processing the doc which takes 3 hrs time to finish.
Kindly share your thoughts on this… It’s great help for all.
BR, Anil.
Anil,
If you have documents arriving over HTTP to a load balancer that distributes them to a cluster of IS’s, how do you maintain order? Is that the question?
One easy way is to use the Broker in the back-end. In other words, as soon as the IS receives the document, it should publish it to the Broker. This way, once the second document comes in, it also gets published and if the trigger is serial, the document will get queued up behind the first one and it will not be processed until the first one is done.
Now, I’m hoping the “3 hours to process” is just hypothetical because with any significant volume, this would quickly build a dangerous backlog.
Just to be clear in case I misunderstood your question, the communication between the IS and the Broker (or UM) does NOT (and should NOT) go through a load balancer. In fact, it’s a much different pattern. The Broker does NOT deliver messages to the IS’s. The IS’s are the ones that go retrieve messages from the Broker.
Hope this helps,
Percio