IS out of the box meets this (very loose) definition as do most other integration tools in existence today.
I don’t have the time or energy to comment on all of those points. However, in general I have used IS to build ESB functionality that meets most of these requirements.
I have not yet used WS-Addressing, but it’s simply a new set of headers that can be used by a custom soap processor.
WS-Security support can be provided, but generally its success depends on the availability of an enterprise-wide identity management solution that has some type of a policy server component that can be queried by IS to determine whether user x (as represented in some security token) has been authenticated and has authorization to invoke web service y (as derived from the qualified name or QName of y’s soap request).
Support for XML Encryption and XML Digital Signature can be created, but is subject to performance issues when dealing with “large document” SOAP requests as the entire message must be read into memory to be signed or to have its signature computed.
I do not recommend using WSDL files generated by WM Developer. However, it is very feasible to use a standalone XML and WSDL design tool such as XML Spy Enterprise Edition to meet this need.
As a note, many of these requirements can be offloaded to an upstream intermediary such as Infravio X-Broker or an XML appliance such as ones from IBM Datapower or Forum Systems.
Thanks for your prompt relpy. It is nice to know that someone has “used IS to build ESB functionality that meets most of these requirements.”
Even though we own IS and Broker I believe. For future projects we are not planing to use the main capability of the Integration Server - that is Integrating into underlying mainframes. We do have mainframe vendors providing APIs for us (xml and java) to this.
That being said - here comes the question - what value add would IS bring to the table in such an envirment?
Can you please also comment on out of the box concurency capablities of IS. In a J2EE environment we rely on Stateless Session Beans and J2EE Container to provide this out of the box. And also the same question goes for transaction managment - out of the box proivided by J2EE container.
The context of the above question is that we can NOT make remote EJB calls in our environment (due to admin credentials and security policies). So if deploy ejb jars as plain java jars to IS how do provide concurency and transaction management?
IS can certainly integrate with mainframes, but that is far, far from its “main capability”.
Pros for building an ESB using IS:
Most ESB’s will need mature, robust integration core capabilities such as support for many protocols simultaneously, rich transformation capabilities and adapter support for those apps not yet able to publish services to an ESB
Ability to work in Flow, rather than java or other low-level languages can increase productivity dramatically during both development and maintenance
Multi-threaded, concurrent execution of services without the headaches and bloated overhead of J2EE
Leverage existing investment in IS and experienced IS Flow and java developers
We chose not to invoke EJB methods remotely either. Instead we published JMS requests to MDB’s residing in the app server and let them talk to the stateless session beans. We chose to use topics to allow multiple subscribers to pick up the same JMS message in the future so that we could plug in additional logging or auditing functionality.
Thanks again for sharing your valuable exprerience - publish JMS requests to JMS topics instead of calling EJBs remotely.
Since this is a forum type discussion I hope you don’t mind me probing a little further the above IS-ESB pros:
“support for many protocols simultaneously”. From our existing business requirements we seldom need more than one protocol (mostly SOAP, a few JMS, and fewer FTP) supported for a specific solution implementation.
If multiple and simultaneous protocol support are not needed why use IS?
“rich transformation capabilities” I have looked at the IS transformation language set - WQL on top of XQL. I compare it with XQuery and find out that XQuery is the current XML W3C standard (a superset of XQL) versus WQL being a proprietary language format. There are more XQuery open source helper libraries available that are maintained worldwide. By far more tutorials and knowledge sharing. Other tools like MetaMatrix have standartised on XQuery as well. Implementing transformations in XQuery for enterprise has the effect of less vendor dependency - both WM and MetaMatrix tools can share the same enterprise transformation library.
Does wM have plans to standartise on XQuery?
“work in Flow” - will have to read more upon this - I am somewhat new to IS. But wouldn’t this be the job of a specific business process and workflow - not necessary an ESB feature?
“Multi-threaded, concurrent execution of services” We have successfully used Stateless Session Beans to accomplish both - our developers have never had to manually manage threading nor transactions - J2EE container given. Except Stateless Session Beans I agree on - “bloated overhead of J2EE”
IS at its core is a centralized ESB type. That has cons - most real business requirments are simple and do not require various set of IS-ESB capabilities. If one does not need simultaneous protocol support, integration adapters have been already implemented somewhere else, and in Flow is also implemented somewhere else - what is the value of IS? In this case a light-weight distributed ESB would do a better job. Axis2 with corresponding WS-* implementations would be a good alternative, right? Why should one use a swiss army knife for all his needs? Isn’t it the case that in reality we use a different knife to cut bread, another for meat, and etc.?
Again - speaking my mind here … hope you don’t mind.
I agree with Mark C about using the IS and Broker as the core to an ESB platform. The IS does provide the key to legacy integration. There are some down sides however that are probably worth mentioning.
Not really a core capability anymore. webMethods has EOL the mainframe adapter. They use a third party now, which is Web Services based. Pretty much any platform can integrate with that.
I know what Mark C is saying here about J2EE but one could argue that the IS server is becoming increasing difficult to keep at a small footprint especially with the addition of the MwS.
Other Cons
IS is a not a great platform for Web Services support at this stage. There are a lot of workarounds or custom development required in order for it to support the standards.
The flow language does make you much more productive but also locks you into a vendor. Most open ESB’s use XLST (IS can use XLST) to handle the transformation which is portable across platforms. But as Mark C said, productivity is much better with Flow.
WS-* support does not exist without writing custom processors. In the open ESB’s this is a configuration setting.
Deploying lightweight endpoints is difficult both from a footprint and from licensing standpoint.
As I said, I agree with Mark C about the capabilities and pros of IS as an ESB. But you should know what the limitations and workarounds are before diving in. The IS server is a very capable proven platform. If webMethods handles the integration with Infravio correctly, I can see a really robust ESB immerging.
If I had a blank slate and no existing investment (financial or skill base) in webMethods and someone asked me to create a SOAP/JMS, lite-transformation only ESB, I would not rush out and spend the bucks to purchase Integration Server and Infravio X-Broker. I would be more likely to select one of the open source ESB solutions such as ServiceMix or Synapse.
That said, the number of enterprises in the real world that can say in advance that they will only need a couple of protocols (SOAP/JMS), only minor transformation capabilities (X-Query or X-Path) and no support for application portfolios that don’t have web services or JMS interfaces are very, very small.
Do you need an ESB or do you need enterprise class integration? If you need both then, IS + X-Registry + X-Broker may be a good solution.
If you love your J2EE app server, then your vendor of choice will sell you something they call an ESB. Knock yourself out.
I find myself reacting negatively to your demand for answers to specific questions. This is an all-volunteer forum and I do not have infinite time to answer your questions. Perhaps you should contact your local webMethods sales rep and / or systems engineer.
That said, I will do my best for this round of questions. After that, work calls and I won’t be back to this thread for some time.
Transformation almost always involves more than changing variable names and message structures, so-called “lite transformation”. Often the transformation engine must lookup reference code values, perform complex calculations, aggregate results, etc. Those things just can’t be done using only X-Query / X-Path. I would like to see X-Query support added, but not as a replacement for what is typically done in Flow with MAP statements and transformers.
BTW WQL is seldom used (probabaly still in the product for backwards compatibility). XQL is not used for transformation, but rather to extract portions of an XML document to work with.
A core ESB functionality according to the oft-referenced Burton Group reference model is “transformation”. While some vendors provide only XML tranformation capabilities, I would assert that this is seldom adequate for large scale integration efforts. Flow is a highly capable tool for rich transformation.
I disagree with most of your assertions here. I don’t think IS is a “core ESB type” whatever that means. It is a mature, robust integration engine.
Most business requirements start simple and quickly become complex. The arguments you are making (multiple tools) have lead a major banking institution to create integrations using over 15 separate integration toolsets. They are now spending tens of millions to undo that mess. Sure one size almost never fits all, but most enterprises need robust integration core competencies, not simply a SOAP-only ESB.
Any regular reader here knows I have been quite harsh in my criticism for WM’s rather weak support for web services standards in Integration Server. It both makes my job much more difficult and provides lots of work for my consulting business.
As frustrated I have become with this shortcoming, however, I still think Integration Server is a great solution for most enterprise-class integration architectures and think a great many of the ESB “products” being rolled out will fall by the wayside once in use for a year or two.
I have to disagree with as well. I currently work for a fortune 250 company and have worked for several others. My experience suggest business integration needs are rarely if ever simple. In fact they are most often very complex and involve a lot of legacy systems. This is the suite spot for webMethods.
If your Integration needs are truly simple then the webMethods product lines are probably too much for you. Consider any of the numerous open-source ESB’s to do message routing, simple (And I do mean simple) transformation, multi-protocol conversion (as long as it is JMS and HTTP) and Web Services hosting. If you have real Integration needs (Complex Transformations, Data Enrichment, Complex Decision Logic, Legacy Adapter Support, and Orchestration) then webMethods is a more serious consideration for you.
As I pointed out, there are some con’s to using the IS server as the core to the ESB but the positives far outweigh the negatives when serious integration work is to be done.
Our integration needs have been solved by underlying mainframe vendors. They do provide integration points using soap+xml and/or ejb. So there is no need for us to use IS for integrating into mainframes.
Speaking of complex transformations - Since XQuery is a superset of XQL, it is also a much capable transformation language than WQL/XQL. IMO - enterprise transformation components should not be a vendor locked in solution - but rather rely on W3C XML transformation standard language and make various vendors like wM and MetaMatrix support it.
Speaking of Axis2. Have you compared Axis2 module plugability feature against the document literal custom soap processor approach? The best thing is that Axis2 keeps it simple - don’t need it don’t add it.
What mainframe vendors are you talking about, are you referring to integration middleware running on the mainframe? I’m sure that could work, but if your organization is happy with that, why purchase webMethods in the first place? You don’t see too many companies hosting the core of the integration capabilities on the m/f. Possible, but hardly leading edge.
XQuery is fine, if you only need to transform XML or flat files. It’s far from my top priority for WM to add to their product since there are already components that do that quite well in the product. To quote the anonymous author, “The great think about standards is that there are so many to choose from.”
Not on mainframe - but rather on app server in form of ear, war that expose the mainframe apis as soap+xml and/or stateless session beans.
XQuery also queries into XML databases as well - but it just emerging in this area. The biggest gain is XML to XML and depending on XQuery impl you get validation and update capabilities - much needed, ans so much easier than working on DOM, most impls use stax so it is fast too.
OK, so your co. used an app server vendor’s integration tools or possibly custom code to expose legacy apps as web services. That is a role that IS was meant to play. What is providing the orchestration of all these web services? Most ESB’s are weak in the area of orchestration and that is often better left to web services aware BPM tools.
Again, my assertion is not that Integration Server is an ESB… it is definitely not. My assertion is that most enterprises, need far more than an ESB to satisfy their integration needs.
If your organization has requirements for a rich integration infrastructure, an existing base of IS developers and a healthy distrust of ESB pure play vendors that won’t be around as the ESB “market” consolidates, then it is at least worth evaluating the option of using IS to provide ESB capabilities as one of its many roles in the IT infrastructure.
With adequate skill and considerable custom development it can provide the functionality of an ESB, especially when combined with Infravio’s registry and intermediary capabilities.
If you just need a dab of XML transformation and the ability to invoke existing web service operations from endpoints exposed by other tools, you don’t need IS, download one of the still unstable Axis2 nightly builds, fire up Eclipse and have at it.
Radu,
Sounds like your integration needs are pretty lightweight. The webMethod’s suite of products is really intended for more serious integration needs that you don’t seem to have. There is no comparison between a developer’s productivity using webMethod’s Integration Server IDE and Axis(1 or 2). webMethods will when that contest every time.
Standards are good and I highly recommend using them where it makes sense. The public side of an interface is a good example. However, don’t believe that standards are a cure all by themselves. The standards still have to be implemented by some vendor whether its webMethods or someone else and there is still an underlying physical implementation.
Truly portable standards based code is a noble goal but it is very difficult to achieve in practice. Proper design with the right mix of abstraction and loose coupling will ultimately give you the best solution regardless of the vendors involved. My two cents.
One of the Requirement of the ESB is Security - (ideally based on WS-Securiyt, XML-Encryption, XML-Signature, and possible SAML).
How can this be achieved using IS 6.1?