Trading Networks in the EAI context

, , ,
1

When we build a pub-sub solution involving several publishers and subscribers exchanging different document types using webMethods platform (Broker, IS, adapters etc.), does it make sense to use Trading Networks for monitoring transactions? What are the advantages and disadvantages of using it in the EAI hub context?

2

I’ve been involved with several projects that have used TN as broker (lower-case ‘b’ is intentional). Here are my thoughts:

  • TN can indeed be used to monitor transactions, regardless of origin. That is its forte. Think of the end-point applications as “partners” and everything falls neatly into place.

  • TN can be used instead of Broker. There are several threads in the forums that explore this. My contention is that TN is far more applicable to the majority of business integrations than is Broker.

  • IMO, there should be no distinction made between “B2B” and “EAI”. The concepts are basically the same. The methodologies for gathering requirements and designing solutions are the same. The tools can be the same. IMO, EAI and B2B are dated concepts–there is simply “integration.”

  • You’ll want to consider using Modeler. TN and Modeler approach the same basic problem (see note 1 below) in different ways. TN tackles it via document-oriented approach: figure out what document you have, determine which rule to invoke, take the action indicated in the rule, record the document and its status. Modeler takes a process-oriented approach: entry service is invoked, model kicked off, steps followed, pipeline stored at indicated steps. Part of the fun as an integration architect is determining which of these, or a combination of these, best fits the scenario at hand (including technical and non-technical considerations).

  • A pub/sub style approach can be implemented with TN, if solutions call for that sort of thing. There is an e-zine article on this site that describes an approach to doing that.

  • TN, in comparison to Broker, is pokey slow. You won’t see nearly the throughput in TN that is possible with Broker. On the other hand, a TN-Broker comparison is an apples and oranges proposition. Broker does one thing very well–manages subscriptions and publications. It does little else. That’s one of the reasons it’s a rock-solid component–it has great focus. TN does a whole bunch of things. It can parse documents (XML, flat-file, EDI, etc.) for identification and content-based handling. In can invoke services, send e-mail, send docs via http, ftp, smtp. It stores documents in a DB. It logs activity to a DB.

  • Using TN Console and TN Web one can check status of documents and reprocess them if needed. One downside is that providing end-user access to this can require some customization.

  • Modeler and Monitor provide process tracking and retries too. You’ll want to be diligent about how models are created and what they save to the DB, how they handle errors, etc. or you’ll quickly find yourself in a “everybody did it differently” mess. (This applies to using TN too.)

Note 1: I imagine someone will take issue with the claim that TN and Modeler address the same “basic problem.” We can take that up in another thread if desired.

TN has been used in internal integrations successfully, but the far more popular belief is that its really only good for interactions with external entities. So you may be wise to use TN only for that, lest you cause someone undue heartache. :wink: You should investigate Modeler thoroughly. With a combination of Modeler and TN you’ll have most everything covered.

3

Rob,
Thanks for the insight.It is really a good knowledge gain.
Keep posting related/same kinda knowledge capsules
Regards,
Puneet Saxena

4

Rob,

Sorry for a delayed response. I have read your message much before though and your detailed articulation was excellent. Thank you for taking the time.
One further question, any idea on how TN accomplishes asynchronous invocation of services? I assume it must be a database based technique.
I was trying to seek clarity if it is “literally” an asynchronous call.

Thanks much,
Srinivas Yeruva

5

To give you a more idea of the requirements for the system to be built are:

  1. Approx. million transactions per day
  2. High speed messaging bus
  3. The Bus has to be asynchronous (The subscribers may be offline)

That is why I was seeking comparision on TN as hub in the EAI context v/s using Broker and building the additional processing logic around it in Integration Server.

Thanks,
Srinivas Yeruva

6

Srinivas,

If I understand your question concerning asynchronous invocation of services; I think what you are seeking is known as a “service task”. When you call a service task, you are registering a task with the Integration Server. Once the task is successfully completed (or fails) the Task Manager will either remove the task or mark it as failed. In the case of a failure, the Admin will need to manage the failure manually. Though, there is probably some solution developed by a board member that automates this task!

7

srinivas:

Processing rules that are configured to execute a service as either “Asynchronous” or as “Service Exectution Task” use threads. An async rule simply spawns a thread in which to execute the service. If the server/service dies, the thread is lost never to be resumed. A service execution task creates a task entry in the TN tables and spawns a thread. If an SET dies, the thread can be kicked off again by restarting the task.

For you requirements:

  1. You’ll need to spec out an appropriately sized infrastructure, regardless of the components you choose to use. One thing to keep in mind is that 1 million business transactions a day (orders, invoices, etc.) will result in many, many more “low-level” transactions. You’ll want to benchmark throughputs early and often in the solution development.

  2. IMO, “high speed messaging bus” isn’t a requirement per se. That’s a technical solution to the need of being able to process the necessary volume of business activity. If you focus on “Broker can process millions of messages a day” you risk losing sight of the fact that most of the tracking, transformation, processing, etc. doesn’t happen on Broker, it happens on IS.

  3. Target systems being off-line must be addressed in some manner. Async communication can be accomplished using Broker or IS/TN or a combination. Be cautious of using Broker only to queue things for off-line entities as you risk filling up the guaranteed storage and crashing the Broker.

An engagement with wM PS or some other services group may be helpful in guiding a design to meet all the needs. Clearly we’re limited in this forum to how deep we can dive.

HTH

8

Thanks Rob for your input.

Srinivas Yeruva