Asynchronous publishAndWait scenario

, , ,
1

Hi folks,

I am using wm7.1.2 and I am facing the following design issue:

Need a process (BPM) that produces two levels of output data, master and detail sets. The input for the process is detailed data, so first the detailed results are produced and then the master (aggregated) data is produced as result of processing the detailed data. I ended up with two processes definitions - one to deal with the detailed data and one to handle the overall job. Now, I want to call the first process as a first step in the second, as many times as many source data items I get. I only want to continue with the rest of the parent process when all the instances of the child process have finished.

The problem is that simply looping the child process synchronously is not acceptable as I may get a hundred thousands detailed items and I don’t want to process them sequentially as it will take ages for the process to complete. For the same reason (possible very high load), I can’t just throw them to a stored procedure at once as this will kill the database.

I want to make the child process asynchronous, so that I can control the throughput and this way strain the parent process in the time, equalizing the load on the IS and the DB.

Now, I came across with a solution like this:

  1. Loop over the input items list
    1.1. Put an entry in a custom queue (db table)
    1.2. publishAndWait (async=true)
  2. Invoke service waitForAll

The child process:

  1. Recceives a published message.
  2. Does what it does
  3. pub.publish:reply

waitForAll service:

  1. REPEAT on SUCCESS
    1.1. waitForReply
    1.2. Remove the item from the custom queue (delete it from the table)
    1.3. Check if there are any left items in the custom queue (count)
    1.3.1. If no more items EXIT from loop and signal SUCCESS

Can anybody tell me a better way of doing that. For some reason I don’t like the idea of maintaining a custom queue for this purpose. I must be really daft, but I can’t get my head round this all day. There must be some native way (for Designer) to implement this scenario. I was looking at the possibility to use referenced/sub processes but both of them (in the docs) don’t mention how can I make them asynchronous.

Any ideas? :confused:

Thanks in advance!