We have got IBMi (AS/400) application publishing some JSON messages on WebSphere MQ and have an IS adapter running on that queue. We need to process these messages in a multi-threaded fashion.
Currently, we have a synchronous notification service that calls a flow service in a sequential manner.
We want to call the same flow service in multi multi-threaded manner. From the online SAG documentation, it seems, an asynchronous adapter service can help but I have never used an asynchronous adapter service and need guidance.
Hi @Holger_von_Thomsen, I am connecting using MQ Adapter Connection/Listener/Synchronous Listener Notification currently. I want to change the Synchronous notification to multi-threading using the same flow service but nowt sure how I can configure a new ASync Notification service to call the same flow. I have never used it. OR do we have any other alternative to make it multi-threaded.
please have a look at the 6-5_WebSphere_MQ_Adapter_Install_and_Users_Guide, which has a chapter for how to configure listener notifications.
Asynchronous notifications are supported by the Adapter.
In this case define a trigger on the publishable DocType created by the asynchronous notification which in turn invokes your service handling the messages retrieved from MQ.
Remember to set the document name (as DocumentReference) in the input of the service to the fully qualified name of the document name as otherwise the service will not recognize the message.
Remember to add the correct version of jars matching the version of your Websphere MQ Server to the WmMQAdapter/code/jars directory of your instance.
Please see the webMethods Adapters SystemRequirements Guide for details about this.
Please check your Fix level for MQ Adapter, latest Fix I know of is MQS_6.5_Fix48.
Thanks, Holger @Holger_von_Thomsen . I will have a look at the document. Do you perhaps know if an asynchronous Notification Service can invoke(publish a document to) an IS flow service without involving UM or Broker? If so, a little guidance will help for a quick reference…
when no messaging component is configured IS will publish the document locally and the trigger will receive it locally.
But this approach should be avoided as it is not failsafe.
When IS goes down for some reason, all these documents will be lost and cannot be recovered.
You will have to ask the sending system to resend them in this case.
@DT30 while you can do local publish, the documents are published to IS. If you make them guaranteed, they are persisted on the disks. When IS is restarted, IS tries to reload these internal queues.
As @Holger_von_Thomsen pointed out - this approach should be avoided.
There may be other approaches that you can use.
What is the overall requirement? are you trying to deliver messages to some destination that needs to happen at faster rate? are you applying any translation in IS?
How many threads do you want to use for listening from MQ?
Like @Holger_von_Thomsen and @CHIRAG mentioned the local publish should be avoided. But not only for reliability, although this is usually the primary reasons. The local publish is also not exactly fast and may even be slower then synchronous invocation.
Please also note that multi-threading may or may not yield an overall performance improvement. The overall prerequisite for it to be helpful is basically this:
The processing of a single transaction on the receiving system takes a relatively long time.
At the same time all involved systems (MQ on AS/400, IS, receiving system) are able to process additional transactions in parallel without increasing the time per transaction. In other words they have sufficient capacity left.
The overhead from multi-threading is sufficiently smaller than the gain on the throughput by going parallel.
You will probably incur a performance penalty in terms of latency. This needs to be weighed against the throughput gain.
As a general comment, and I am not sure whether or not this applies here, it is always worth to first try to improve performance on the single-threaded processing. Not only because of the costs of multi-threading (see above). But also because the complexity for operations grows considerably. And I am not even talking about things like in-sequence processing.
Would like to share something as you mentioned ‘multi-threading’
MQ Listener is single threaded ( underlying, it will create a single TCP/IP connection ESTABLISHED state between MQ server and IS MQ Adapter when you enable the listener). Through this thread, it reads the messages from the queue like a stream. If you have 2 Integration servers connected to MQ, there will be 2 TCP/IP connection. Messages flow through these in much higher speed.
Listeners can be associated with MQ Transactional connection or MQ Non-Transactional connection. As the name indicates, transactional provides guaranteed message delivery (from MQ to IS) and non-transactional is like fire-and-forget.
As every one mentioned, if you need faster processing, it can achieved only with Adapter notification of type ASYNC.
Your need of multiple requests to be processed concurrently starts with ASYNC adapter notification, followed by other tunings if necessary ( connection type, transaction type, more connections on the listener, trigger properties etc., )
@Senthilkumar_G For same connection, you can have more than one listener.
If UM is an option, then using Async - all the notifications can be sent to UM queue and a multi-threaded trigger can be used to apply more thread with increased stability (but additional component needed).