60 architecture seems very limiting

, , ,
1

We have a rather mature Broker/Messaging environment and also use IS as part of our integrations (currently at broker 5/is 4.6).

In reviewing the 6.x materials, there appear to be many limiting factors:

  • an IS can only talk to a single broker. why?
  • an IS can only utilize a single client group. why?
  • with the IS being the new adapter runtime the above two limitation greatly reduce/preclude any advantage you might have received from running multiple adapters in the same IS/adapter runtime.

It seems that in finally ‘integrating’ the activeWorks and webMethods product lines that webMethods have succeeded in destroying the flexibility that made the activeWorks messaging products so great to work with.

Am I just misreading this, or are these limitations for real? And if so, why? Are there viable ways of working around these limitations (without proliferating ISs all over environment)?

2

You’re not misreading things. The limitations are real–and seemingly arbitrary.

If by ‘integrating’ you mean to be sarcastic, I share your view. ES and IS are not yet fully integrated. They are still two distinct environments. And the Jekyll and Hyde positioning of IS as sometimes a server and sometimes an adapter environment for ES adds to the confusion.

Hopefully this is just a stepping-stone where the next logical iteration seems to be a full-blown integration where the pub/sub facilities are folded into IS (merging TN and Broker seems to make sense). I hope they move to a single server environment, with all the features of both servers, and provide a small-footprint IS to be used as an adapter environment.

3

Arbitrary, yes. I might go so far as to say disasterous. I mean, I think that there is a reasonable expectation that a vendor is not going to remove substantial functionality and flexibility from a product in the name of improving it. Attempting to upgrade to 6.x is going to ADD a huge amount of complexity to our integration infrastructure for very little gain.

The limitations of one IS to one broker reminds me of the ‘bad old days’ of point to point interfaces.

4

Agreed. This constraint is going to be a huge sticking point for existing customers that want to migrate.

I truly hope this is simply a transition step to something better (“can’t do everything all at once”). Having a single adapter restricted to one ES seems reasonable but have the environment that hosts multiple adapters restricted to one ES is goofy.

5

Folks… I agree there are some issues with the v6 product architecture requirements… but my understanding of IS-ES communication is different than what I’ve seen here…and maybe I just missed something here…

Yes an IS instance can have only one ES defined, BUT!! ONE ES can have several/multiple IS instances connecting to it…

6

Raymond,

You are right that one broker can connect to multiple IS instances. But the fact that the IS is what contains (for the lack of another term) adapters, it would be helpful if an IS could also connect to different brokers. It would provide for more flexibility. This restriction leads to unecessary instances of IS to be created just to be able to connect to different brokers. As of now there seems to be no other option execept putting all the brokers in a territory.

7

True, but pre-6 didn’t have that limitation. So we have a bunch of customers that have a single IS instance talking to multiple ES instances for different integrations. To move to 6 they will now have to move the integrations to different IS boxes–increasing maintenance efforts and costs.

Forcing ES to be THE link between IS boxes is arbitrarily limiting. There is little to no advantage to doing so and is contrary to the architecture that was available before.

8

You’re right on Rupinder. And adding brokers to a territory has its own set of issues. wM needs to address this soon.

9

Is webMethods aware of these concerns ? If so do they have any plans on giving an IS the flexibility to connect to multiple brokers ?
This appears to be something that should be addressed sooner rather than later.

10

Actually, in the webMethods Enterprise 4.x/5 architecture, the adapter
runtime could only connect to one Broker and could only be a member of
one client group. So the model in webMethods 6 where an Integration
Server is a member of single client group on a single Broker is exactly
the same as the old model. No functionality is lost.

Some of the new and improved capabilities provided by the IS 6
architecture for connecting to the Broker, include:

  • the ability to host multiple adapters in the same process
  • providing a transaction manager to enable synchronous transactions
    across multiple adapters in a single service
  • the ability to host multiple subscriptions to the same document type
    within the same process
  • the IS establishes 1 broker client per trigger (subscription), versus
    the Enterprise model of 1 client per adapter process, providing a much
    more scalable and tunable solution
  • the IS establishes a pool of publishing clients to the Broker,
    providing a more scalable model where multiple IS service threads can
    publish in parallel
    … and many more.

It is true that the IS attaches only to one Broker and in only one
Client Group. If the IS were to allow connections to more than one
Broker, the IS itself would be a broker, as it would have to determine
how to route messages between those multiple Brokers. Additionally,
users would have to identify specifically which brokers upon which to
publish/establish subscriptions, destroying the pub/sub value prop and
turning it into a point-to-point model. The fact is that the webMethods
6 architecture in fact retains and expands the pub/sub model. It is
most definitely not a point to point model.

11

“the adapter runtime could only connect to one Broker and could only be a member of one client group. So the model in webMethods 6 where an Integration Server is a member of single client group on a single Broker is exactly the same as the old model. No functionality is lost.”

I beg to differ about the lost functionality on two fronts:

  1. Yes, a single adapter could only connect to one Broker. But it was trivial to run multiple adapters on a single box. The Adapter Monitor & Config tools managed multiple adapters on a single machine, each of which could connect to a different broker. So IS as a runtime environment is less capable than the previous adapter runtime environment. (But IS isn’t just an adapter run-time environment–more on this below.)

  2. IS 4.6 and previous could connect to any number of Brokers. This is a significant loss of functionality.

All the benefits listed are Good Things if you’re coming from the ES world. From an IS POV, they are much less interesting. For “* the IS establishes 1 broker client per trigger (subscription), versus
the Enterprise model of 1 client per adapter process, providing a much
more scalable and tunable solution” one could argue that this is a LESS scalable solution since each subscription creates another client q, whereas a process focused approach would allow multiple subscriptions (to different doc types) via a single q.

“… the IS itself would be a broker”

From my view of the world, Integration Server IS a broker. It always has been. It is an intermediary between parties that communicate. It manages things quite nicely without ES/Broker. One can even do pub/sub (though it is a bit more work than doing so in Broker). Trying to lock it down to be an intermediary only between some resource and only the ES Broker is breaking existing infrastructures and an unnecessary constraint.

“Additionally, users would have to identify specifically which brokers upon which to publish/establish subscriptions, destroying the pub/sub value prop and turning it into a point-to-point model.”

I disagree. Integration designers need to identify which Broker a specific adapter connects to. Isn’t this exactly how AW/ES/Broker has always been? The IS is the environment in which multiple adapters run. The config specifying which Broker to connect to should be at the adapter level, not the IS level. As an analogy, consider if the database adapters could only be configured to connect to a single DB instance, configured at the IS level. Wouldn’t that be far too limiting?

Putting control of which Broker to use at the adapter level would not destroy the concepts of pub/sub–if it did then AW/ES suffered the same problem. Rather it gives architects the flexibility to structure an appropriate architecture, trading off performance, maintenance, administration, licensing costs, etc.

Pub/sub needs to be rolled into IS. ES/Broker as a distinct entity should disappear. This would provide the opporunity to create P2P, hub-and-spoke, and hybrid infrastructures as desired. That IS already has “local” pub/sub should make it easy.

Of course that’s just my opinion. I could be wrong.

12

Oh, one more point. Since the Broker is essentially equivalent to WebSphere MQ, MSMQ, etc. (of course Broker is far better! :wink: ) why does it make sense to limit connectivity to the Broker from IS when IS can connect to an arbitrary number of different MQ facilities? If the constraint is maintained, it seems like wM would be pushing customers toward tools that are functionally equivalent to Broker.

13

I agree with Rob’s points wholeheartedly.

Any advantage that one might have achieved by running multiple adapters in the same IS/adapter run time is watered down by the fact that the broker you deal with is controlled by the IS and not by the adapter.

I think that webMethods needs to recognize the ‘one IS one broker connection’ is broken and needs to be fixed in a service pack sooner rather than later.

14

DITTO!

“I think that webMethods needs to recognize the ‘one IS one broker connection’ is broken and needs to be fixed in a service pack sooner rather than later.”

15

Been watching this interesting thread: I can see that in here, this topic shows two different point of views, those who come from ES world and those who come from the B2B world…

  • But anyway, may be the GFA JMS Adapter may leverage the connectivity to multiple Brokers from a single IS (It is not validated to work with Broker JMS provider, but it does not mean it will not since JMS is supposed to be a well detailed spec, I successfully tested it on MQ), or may be the Bridge 4.7 ? Though these components will definitely not provide the features we would like to have as in the native IS-Broker dispatcher (inbound/outbound queues, trigger control, message duplicate detection, etc).

  • I can also understand that in old ES, we used to dynamically create new “light” ES adapter instances easily, which made the fact that it connects to only one Broker a non-issue. In 6.x, it is not like that but you can sort of get close to it and we are doing it here at my customer: Using Installer and Silent Script install (no-gui and non-interactive install), we can create “light” IS with minimal memory footprint quite quickly through automation scripts. It is not as dynamic as with the old ES, but we are content with it

riad

16

Riad: I certainly respect your POV but I must disagree about “two different point of views, those who come from ES world and those who come from the B2B world.” inasmuch as the comment pertains to my posts. My views are based on my experience with both platforms. I’ve used both environments extensively.

“Using Installer and Silent Script install (no-gui and non-interactive install), we can create “light” IS with minimal memory footprint quite quickly through automation scripts. It is not as dynamic as with the old ES, but we are content with it.”

This is an excellent point and is one I think needs to be fully embraced and supported officially by wM–with the release of a small footprint IS dubbed Adapter Environment or something. The primary hurdle will most likely be licensing terms, not technical issues.

17

This was posted by Fred Hartman on Advantage. Thought I’d cross-post it here for further discussion. My response to Fred’s response to follow in a separate post.

Posted by Rob Eamon on Thursday, August 28, 2003 - 04:43 pm:

[start Rob]
“the adapter runtime could only connect to one Broker and could only
be a member of one client group. So the model in webMethods 6 where an
Integration Server is a member of single client group on a single
Broker is exactly the same as the old model. No functionality is
lost.”

I beg to differ about the lost functionality on two fronts:

  1. Yes, a single adapter could only connect to one Broker. But it was
    trivial to run multiple adapters on a single box. The Adapter Monitor
    & Config tools managed multiple adapters on a single machine, each of
    which could connect to a different broker. So IS as a runtime
    environment is less capable than the previous adapter runtime
    environment. (But IS isn’t just an adapter run-time environment–more
    on this below.)
    [end Rob]

[start Fred]

That is a strong condemnation that the IS is a less functional runtime
environment simply because it is more difficult to start two instances
of it one the same machine. It is, of course, entirely possible to
start 2, or more, instances of the IS on a single machine and connect
them to different Brokers. An IS has no larger a memory footprint than
the 4.x ADK, so that is not an issue. The large amount of additional
functionality provided in the IS actually makes it a significantly more
capable environment.
[end Fred]

[start Rob]
2) IS 4.6 and previous could connect to any number of Brokers. This is
a significant loss of functionality.
[end Rob]

[start Fred]
IS 4.6 could do that through the Enterprise Package (which was
essentially and adapter), however, there are some serious considerations
one must be very careful not to put the Brokers into the same territory
or connect them through a gateway (if they are connected, publisher
ordering will no longer be guaranteed and one must take care not to
create subscriptions to the same document type on different brokers or
duplicate processing will result). In the model you describe where the
IS connected to two (or more) Brokers, the IS acts as a bridge between
two Brokers that must be managed by the programmer. However, the Broker
has built-in capabilities to do the very same thing safely: territories
and gateways.
[end Fred]

[start Rob]
All the benefits listed are Good Things if you’re coming from the ES
world. From an IS POV, they are much less interesting. For “* the IS
establishes 1 broker client per trigger (subscription), versus
the Enterprise model of 1 client per adapter process, providing a much
more scalable and tunable solution” one could argue that this is a
LESS scalable solution since each subscription creates another client
q, whereas a process focused approach would allow multiple
subscriptions (to different doc types) via a single q.
[end Rob]

[start Fred]
It is most definitely not a less scalable solution. Having multiple
clients on the Broker provide a level of possible parallelism impossible
in the old model from a single ADK instance. For example, consider an
adapter that is receiving two types of documents:

Doc Type A: Large, 10MB docs containing batches of transactions that
need not be processed with low latency, but must be processed in order

Doc Type B: Small, 1K request documents that require a very low latency
reply, but need not be processed in order.

In the Enterpri

18

Fred wrote:
“That is a strong condemnation that the IS is a less functional runtime
environment simply because it is more difficult to start two instances
of it one the same machine. It is, of course, entirely possible to
start 2, or more, instances of the IS on a single machine and connect
them to different Brokers. An IS has no larger a memory footprint than
the 4.x ADK, so that is not an issue. The large amount of additional
functionality provided in the IS actually makes it a significantly more
capable environment.”

Yes, it is indeed a significantly more capable environment–except if you need to connect to more than one broker. For this specific function, IS 6 is a less capable environment.

Fred wrote:
“IS 4.6 could do that through the Enterprise Package (which was
essentially and adapter), however, there are some serious considerations.”

There undoubtedly constraints to be aware of. However, by restricting the number of brokers that can be connected to, you take away a reasonable architecture option that will work just fine in many instances (when I don’t care about ordering, etc.). And what I’m after isn’t IS acting as a bridge between 2 Brokers, but independent integrations where one integration interacts with 1 Broker and the other integration interacts with another Broker.

Fred wrote:
“It is most definitely not a less scalable solution. Having multiple
clients on the Broker provide a level of possible parallelism impossible in the old model from a single ADK instance.”

I concede the point. It was a point that should have been left on the cutting room floor–I need a better editor! :wink:

Fred wrote:
“While one may consider the IS a broker by itself, more apt description
is “dispatcher.” The IS by itself (prior to local pub/sub capabilities
implemented in IS 6) implements a point to point dispatching mechanism
only - a single incoming message is mapped to a single service within
that same process. A Broker does much more. A Broker is aware of and
provides location transparency for the entire distributed environment,
providing sophisticated metadata synchronization and message routing
capabilities. Of course, as you mention, it is entirely possible to
replicate that functionality in the IS, but is much, much more than a
“bit more work.””

I figured this might head into the “what’s a broker” area. Does a pub/sub feature alone make something a “broker”? I’m inclined to think not but I’m aware that the popular view/definition of broker is exactly that.

Wrt Broker vs. IS, to argue degrees of difference, a broker does not, IMO, do “much more” than IS. In the scheme of things, it’s actually pretty basic–keep a list of subscriptions/filters, copy received docs to the appropriate queues. Done. IS/TN does indeed do pub/sub out of the box–a processing rule is a subscription. Calling receive is the publish. The only functional difference between TN and Broker is that TN only delivers to the first “queue” that matches the subscription–a limitation that is easily overcome. In fact, I’ve implemented a pub/sub capability using TN. It truly was very straight-forward.

Fred wrote:
“Certainly integration designers must specify the Broker to which to
connect. It does not, however, follow that each adapter should be able
to specify a Broker.”

Isn’t that in fact what adapters in 5.0 and before do exactly? Isn’t that what IS 6, positioned as the uber adapter, does exactly? Is IS the adapter, or are the modules that run within IS considered the adapter? Or is this why things are starting to be called modules instead of adapters? IMO, making

19

My head is ready to explode. Lots of great/interesting comments. Forget combining the Broker with the IS…Maybe webMethods should license the Universal Business Adapter. Remember that IBM commercial ?

Scene: A bunch of executives examining a contraption that looks like a Magic 8 Ball with a series of ports and interfaces bolted to it.

CEO: “What is it?”

Product Sales Guy (PSG): “It’s a UBA.”

CEO: “UBA?”

PSG: “Universal Business Adaptor.”

CEO: “What does it do?”

PSG: “It connects anything to everything.”

Here the CEO begins pointing to the various ports.

CEO: “What’s this for?”

PSG: “Your laptop … your mainframe … call center … Unix servers … Linux servers … Internet … supply chain … payroll system … H.R. … e-mail.”

CEO: ‘Slick’ “Is it affordable? Fast? Easy?”

PSG: “Very.”

CEO: “Does it work in Europe?”

And all eyes turn to the guy who has brought in the device.

PSG: “You need an adapter.”

Check out this link if you want to see the actual commercial:
[url=“http://www-3.ibm.com/e-business/doc/content/ondemand/tvspot.html?P_Site=S94”]http://www-3.ibm.com/e-business/doc/content/ondemand/tvspot.html?P_Site=S94[/url] (bottom of the page)

20

Hi ,

 i am new to webmethods.i never worked on sap adapter configuration.

i came to know that there are great people here in this group who can help me out!
Could you post the documentation on SAPadapter configuration.
if the documentation was existed,where can i get that?
I already searched in our forums,but i didn’t get the clear picture
for the environment of webMethods6.0.1,windowsNT,SAP4.6b.

Advanced thanks a lot for the information.

kind regards
sirapu