What is the best mechanism to publish and subscribe to the SOAP messages to webMethods broker. My publisher wants to publish a SOAP message to the Broker. My subscriber wants to subscribe to the SOAP message and then call a different web service using this SOAP object. My goal is to avoid the marshalling/unmarshalling of the SOAP payload during the pub/sub to the webMethods broker if possible. Can i create a publishable broker document with SOAP object as payload? Broker should not care for the type of payload. Right?
I htink the question was not so much whether you could publish a document that contained a string field populated with a soap message, but whether your publishable doc type could contain an object variable populated with the webMethods soap message object (com.wm.app.b2b.server.SOAPMessage).
I have attempted this in the past and seem to remember that for some reason the object was converted into a string between publication and subscription. I never did get to the bottom of why that was happening, but just added a check in the triggered service to check the object’s class and convert the string back into a soap message, if necessary.
I used to have problem publishing documents (result of SOAP message extraction) which contains some characters that are not supported in Broker publishable document type. I converted the document in string before being able to publish in the Broker.
Based on the description of your problem, you have one publisher and one subscriber, so why even bother with the Broker? Additionally, it sounds to me like you’re planning on having a web service that will accept a SOAP request, pass it on to the Broker, then you’ll have a subscriber who will simply invoke another web service with the same SOAP request. Why? Why not have the source application send the SOAP request directly to the ultimate target web service?
Don’t take me wrong… I am all for converting a point-to-point integration into a pub-sub architecture if it makes sense. However, I’d have to say that if you want to go down that path, I really think you should consider not using a SOAP object as your publishable document. Instead, I believe you should consider publishing to the Broker the actual canonical representation of the data contained within the SOAP object. In my opinion, this allows for greater flexibility. Moreover, in a true pub-sub integration, the publisher should not care that the subscriber(s) will invoke a web service, neither should the subscriber need to know that the publisher received the data via a SOAP request.
I agree with the questioning of the use of Broker (pub/sub) in this situation. But I don’t agree necessarily with the “source application send(ing)…directly to the…target web service.” IMO, direct linkages such as this are to be avoided in general.
Interactions between any two systems should usually be handled by an intermediary. Sometimes this intermediary should be a pub/sub mechanism such as provided by Broker. In this specific case, IS is probably the most appropriate intermediary*. This decouples the end points (to a degree) and provides a control point for future flexibility.
Footnote: IS did a fine job of being an intermediary long before Broker ever came around. It still works fine in that capacity for many, many types of integrations.
In this scenario a client calls the publisher service. The publisher service acknowledges the SOAP request and turns around immediately and sends the response back to the client, apart from calling other services synchronously. At the same time, the publisher service will also publish the SOAP request object to the broker, so that a different subscriber can asynchronously subscribe to the document containing the SOAP object and call another web service which does further processing.
We want to accomplish two goals:
Quick response to the client acknowledging the request
Asynchronous processing
Avoid point-to-point integration for future flexibility (Just a secondary benefit)
A quick response can be achieved by spawning a thread to do the actual processing. See doThreadInvoke in the API.
Again, threading can be used to do this. Obviously, publishing does this too.
You are already avoiding point-to-point communication by having the end points talk to IS instead of directly to each other. However, by pushing the entire SOAP envelope around, instead of just the payload in the SOAP body, you are increasing the coupling of the end-points. In some circles, this is viewed as worse solution than a direct source to target integration.
Have you measured the overhead? Do you know that this overhead adds an unacceptable amount of time to the process? The SOAP envelope probably isn’t the interesting part. It’s the body that’s of interest. I would suggest dumping the SOAP envelope in the publishing service. Otherwise, you’re forcing all subscribers to have to deal with the SOAP envelope, even when they don’t care about it at all. Plus, the subscriber may very likely need to re-envelope anyway, depending upon their target.
I definitely agree that having an intermediary between two systems is the way to go, but it wasn’t clear to me what value the IS and the Broker were adding to the mix.
The reasons provided are definitely worth it though. I’d stick with the Broker solution, but I’d stay away from publishing the SOAP envelope for reasons Rob and I have both described. Instead, I’d publish the actual data in a universal format that future subscribers can use.
I personally prefer to use the Broker and/or TN for asynchronous processing over spawning my own threads because of the extra, built-in exception handling and retry capabilities.
Thank you all for a great conversation. These are very good points from great minds. Rob, i would definitely like to hear your 3 rules of optimization.
You are right. When i do a object.getClass().getName() call on the SOAP object just before doing a publish in the publisher service, i get “com.wm.app.b2b.server.SOAPMessage”. When i do the same call in the subscriber flow service i get “java.lang.String”. So i had to use the pub.soap.utils:stringToSoapData service to convert it back to the SOAP object in the subscriber service.
This is what i found when i did a search -
“Broker documents are not really built to support a random Java object data type. It basically supports the primitive data types, their arrays, or the structures of those primitive types as normally used in C language. When you create a publishable document type that has field of Object type, the IS code would actually pass a String form of an Object pointer into the broker document unless your object implements the IDataPortable interface.”
Could you throw some light on " pass a String form of an Object pointer "?
If you really do need to stuff the entire soap message into a publishable doc type (see comments above why this is not necessary), then just do so as a string. However, in addition to the body of the message, you may also want to include any headers if they provide metadata about what is in the payload (body) of the message.
My current project is receiving soap messages, performing a security check on the WS-Security token form the soap header, performing XML validation, if enabled for that service and then creating and publishing a JMS message to BrokerJMS to be consumed by Websphere. When all of that happens (less than 1 sec total elapsed time in all testing so far) we generate a message acknowledgement and return it in the soap response along with a unique request id that can be used to retrieve results when they become available (or for us to include when pushing results to the original caller).
I’m afraid the 3 rules don’t really provide any great insight. The rules are a tongue-in-cheek derivation of the 3 rules of real estate (location, location, location). The 3 rules of optimization are 1. don’t optimize yet, 2.don’t optimize yet, and 3. don’t optimize yet. One should optimize only after profiling an application/integration and identifying the bottlenecks. Then one can optimize the areas that will provide the greatest impact.
All Java objects are derived from java.lang.Object. This class provides the method toString(). The java.lang.Object implementation of this simply returns the address of the object. Subclasses can override this method to return the most appropriate string representation of themselves. Based on the snippet you quoted from the docs, IS apparently calls toString() on the Java object when publishing the doc.
Those are great rules Rob. I cannot agree with you more that one should optimize only after profiling an application/integration and identifying the bottlenecks.
In this scenario we have multiple publishers and a single subscriber. Also subscriber may not always be up. That is the reason why we chose the pub/sub using broker.
Also i agree that we have different binding options (SOAP, XML, Java etc) and we do not have to publish SOAP objects in this context. We could use xml binding and publish messages in a universal format and utilize the flexibilty offered by pub/sub.
One should optimize only after profiling an application/integration and identifying the bottlenecks. Then one can optimize the areas that will provide the greatest impact.