Duplicate Documents and Exactly Once Processing

If you have multiple integration servers connected to the same Broker using the same client group and client prefix when you publish a document to the broker will only one of the Integration servers process the document, Right? Do you need to cluster the IS for this behavior, excluding the condition for triggers with join conditions needing shared Repo. When would you need to implement “Exactly Once Processing” or “At Most Once Processing” to prevent duplicate document processing between the multiple Integration Servers from a single publish?

Right.

No.

To cover the case where the communication between IS and Broker is interrupted. To provide this, the IS instances must use the same document history database (use of a document history database is optional). According to the Publish-Subscribe Developer’s Guide, to share a document history database, the IS instances must be in an IS cluster.

Okay so I am not crazy. I have 2 IS instances clustered with shared state queing configured and I am still getting the same trigger in both instances processing the same document.

I put a savePipelineToFile in the Flow service on each server and named the file according, “flowservice_machineName1”, and “flowservice_machineName2”. When I publish the document I check for the files being created on both servers and they are?

this is a 6.0.1 upgrade to 6.5, in case that bit of knowledge makes difference. I know in 6.0.1 Exactly Once processing did not exist, it came out in 6.1.

Any thoughts as to why the shared state queue is not working?

Check the Broker Client prefix that is configured in both IS instances. They should be exactly the same. If they are not, that is the cause–the Broker is seeing two different subscribers for the published doc.

everything was identical. So I deleted the broker and broker server and started over with the configuration. Once everything was back up the issue has not occurred again.