Integration Server as application server

, , ,
1

Thought I’d start up a purely academic and hypothetical discussion around using IS for something it was not intended for–hosting “business logic.”

I’d venture to say that the majority of the wMUsers participants are quite familiar with n-tier application architecture idea. An application might have these conceptual layers:

Presentation
Presentation logic
Business logic
Data access
Data

There are many, many tools available to implement such an application. J2EE, .Net, etc. WebLogic and WebSphere are clear market leaders in the J2EE app server space.

The focus of my hypothetical is on the business logic tier. Business logic is typically implemented using beans, EJBs, C++. C#, .Net assemblies, and so on.

What would be the implications, pros and cons of designing and implementing the business logic using Integration Server? And I don’t mean exposing business logic hosted somewhere else. I mean really implementing the “guts” of an application in IS.

Some immediate “no ways” come to mind for me but there are also some “could be cools” also. Particularly when the communication between the layers would be web services.

Thoughts?

2

Rob,

You have no idea how exciting it was to see that you started this thread. Just today, our integration team had a meeting to discuss exactly this topic. Coincidence or fate? :slight_smile:

I too agree with you that “business logic” does NOT belong in the IS. However, I believe the problem we all run into is: how does one define “business logic”? Clearly, given that the IS is normally part of an enterprise system whose primary function is to support a business, anything and everything implemented could be viewed as business logic, including simple mappings. So, where should the line be drawn as far as what’s appropriate to be implemented in the integration layer and what’s not?

As a rule of thumb, I believe the IS should only be responsible for mappings, transformations, and validations that can be derived directly from the data that is provided by the source system. IMHO, the IS should NOT be involved in any validations or transformations that require it to read, create, or update any outside data sources. For example, verifying that a product exists in an ordering system before submitting the order or looking up an internal product ID based on a customer product ID.

However, even this rule is too broad. A lot of business logic can still be written just based on the data provided by the source. For example, is it ok to check if an item quantity is greater than zero? Or is it ok to calculate a total price given the unit price, quantity, and unit of issue?

I tend to believe that even this type of simple logic should be avoided within the IS simply because if the target system changes in the future, the IS logic should not have to change. This, however, requires that the target application provide an interface that takes care of validating and processing the data, and of reporting back any errors. As we all know, this is not the case all the time (or most of the time.)

I apologize for the wordy post. It’s hard to discuss this topic without going on and on because there are so many things to consider. I’m anxious to hear what the rest of you have to say.

  • Percio
3

Hosting business logic on the IS is an attractive option. However I just can’t see how to justify the increased licensing cost associated with executing business logic on the IS versus weblogic, or another app server.

4

I hope I didn’t turn anyone off with my long post. :stuck_out_tongue: While we wait for other people to jump in, let me play Devil’s Advocate for a little bit.

How can we justify not including business logic in the integration layer when one of the big selling points for the Integration Server is that it comes with a Business Process Modeling tool (ie. Modeler)? Should we limit our BPM designs to very simplistic models so as not include any business logic?

One Modeler example that webMethods presented (I don’t remember where or when) and stuck with me was that of a Loan Application BPM. The model checked the amount requested for the loan and the user’s credit to determine whether the loan should be automatically approved, rejected, or sent to a workflow.

Thoughts?

Thanks,
Percio

5

One of the main issues I see with adding business logic is on the support side. If you are integrating numerous backend systems and include business logic within the integrations, the support team now needs to have an understanding of the back end systems and the logic involved(which I feel isn’t possible in larger settings). If the IS is as void of business logic as possible, the IS support staff supports the integrations and errors associated with that and the app support teams deal with errors pertaining to the logic.

6

One could argue that the loan example is an example of a new application not an integration per se. Modeler and Workflow are really tools for creating applications. IMO, business logic should be minimized in the integration layer.

The hypothetical proposed imagines that one uses the tools to create a full-blown application–integration with other apps would be a secondary concern. For example, consider the latest .Net or J2EE app you’ve built. Now imagine implementing that app completely using IS and associated tools.

7

The “how much business logic should be in the integration layer” is a good topic. It wasn’t the focus of why I started this thread but we certainly can explore that here.

Rather, I was hoping to discuss the use of what are normally considered integration tools to develop applications. In other words, use IS and associated tools not to integrate but to create a new app.

8

I think I agree with you that integration tools such as IS should not be used to create applications that require both business logic and presentation. Yes you can do this in IS, but I would avoid it for no other reason than the presentation layer options are severely limited unless you add Portal/My webMethods/webMethods Access/.

But I would suggest that using IS to create new services is the way to go.

Using Flow to orchestrate multiple calls to applications using disparate technologies and operating platforms is one approach for exposing new, more coarsely-grained business services. These “orchestrations” invariably involve adding some amount of business logic even if it is limited to retry logic. In some cases, very little data transformation may take place other than what is required to make the calls to the application-resident business logic.

I would assert that this is a use of integration tools that is not only acceptable, but that is a mainstream approach to realizing SOA when legacy (mainframe, J2EE, .Net, you-name-it) applications are in the mix that don’t yet speak web services or that do so inefficiently (e.g. every EJB method or COTS-package API call is a “service”).

If you slap some vendor’s portal UI on the front of this new business-level service, you have just developed a new “composite application”. Plug the service into someone’s BPM tool and you have just realized one of the main goals of SOA, easily recomposed business processes.

Yes, you can move this orchestration up a layer in the stack to some BPEL-compliant BPM toolset. However, unlike IS, most of these tools can only consume lower-level services if they have exposed a web service interface.

Mark

9

Orchestrating the activities of other applications is a very common use of IS. Exposing services, at whatever level of abstraction, is something IS is very good at.

What if you put ALL of the logic for a given application in IS? Not for integration purposes but as a means to deliver a new application? Develop FLOW and Java services to be the system of record for a given business function?

10

One word. “Why?”

11

“Why” and “why not” are precisely what I was after in starting this thread.

From my original post: “What would be the implications, pros and cons of designing and implementing the business logic using Integration Server?”

We got a bit side-tracked in talking about how much business logic should be in the integration layer (a good discussion to be had) but my hypothetical wasn’t about that.

Here are some possible “whys”

  • As a few people have attested to in these forums (Mark included IIRC) developing services in IS can be wicked faster than doing similar tasks in a classic programming language, once you’re over the learning curve.
  • Database connection pooling out of the box (app servers do this too and it’s a good thing).
  • Logging out of the box (similar to log4j in capability) and automatic log file rotation.
  • Modeler can be used to model some/all of the logic, providing tracking and troubleshooting via WmMonitor.
  • Web service support without needing to do anything specific (okay, you’d outgrow that pretty quickly but it’s still cool that any service can be invoked via a web service with basically no extra work).
  • Session management out of the box.

There are a bunch of why nots but I’ll yield the floor to anyone else that would like to chime in at this point…

12

Oh wow… I would just hope no one would attempt that type of thing. Unfortunately, however, as we know from posts we’ve seen here and elsewhere, there are some people out there trying it.

Why shouldn’t it be done though? I don’t know what better to say than “because it’s just wrong.” :slight_smile: The tool was simply not built for that. From a performance, infrastructure, and administration standpoint, it would just not deliver. I mean, you could also build an integration platform from scratch using POJOs or using a business app, but why?

Heck, you could play soccer with a football, just don’t expect the ball to roll straight! :slight_smile:

  • Percio
13

BEA and IBM have more or less done exactly that with their app server environments. BEA’s WLI has not been very successful but is an example of having a traditional app server acting as an integration platform.

Can you provide specifics on why using IS as an app server “is just wrong?”

Really? Why not? It can’t handle the transaction volumes? It can’t scale? If that’s the case then why the heck would we use it for integration to support high-volume integrations? Certainly the interpreted nature of FLOW adds overhead but then that’s an optimization concern in which key parts could be switched to Java to get at least the same performance profile as the classic class running on an app server (both would be Java byte code running in a JVM).

I guess I’m saying that the initial assessment of “it would just not deliver” is too anecdotal–have anything objections that are more concrete?

I suspected that the initial reactions to this thread would be “why in the world would you do that?” I’m hoping we can get beyond the initial ‘blech’ and discuss details.

14

The “sweet spot” on a tennis racket is the area of the racket’s string bed that produces the best combination of feel and power.

As I’ve mentioned here in the past, I’m a big believer in finding the “sweet spot” for a given software product (development tool, application package, etc.) and using that tool only for those things that fall within it’s sweet spot.

Just as you can hit a tennis ball with parts of the racket that are less than optimum, you can attack a technology problem using either the wrong tool or the wrong features of the right tool.

The old adage “If all you have is a hammer, everything looks like a nail” applies here. If a company or department or individual consultant only knows a single tool or technology then all problems that cross their path will look like perfect fits for their one and only tool.

Developer’s who have invested time and experience in learning a particular tool’s features, techniques and quirks can be amazingly productive with that tool. I’ve run into Perl wizards who could create incredible amounts of business logic using that tool (and its huge library of built-in functions). That doesn’t mean I would ever recommend that Perl be used by that developer to create my next business application.

So, webMethods Integration Server with the services and middleware capabilities that come out of the box plus the ability to craft processing logic using Flow or java services can be a highly efficient, highly productive development environment when used by folks who have the experience and skill to wield it properly.

In this regard IS/Developer overlaps with other app servers and development environments such as J2EE with Eclipse or some other IDE or the .Net Framework with Visual Studio. The app servers provide the middleware services and the IDE’s provide the highly productive development environments.

IS / Developer is considered a leader in the integration middleware space. However, it does not compare well with app server / IDE combinations in the area of general development environments for several reasons:

  • Complete and total lack of integration with version control systems (don’t get me started)
  • Embarrassingly weak text window used for java service development (I won’t dignify it by calling it an editor)
  • Proprietary Flow language known by a limited pool of developers
  • Flow optimized for processing hierarchical documents, not general purpose business logic
  • As mentioned earlier, license costs orders of magnitude higher than general purpose development environments

The “sweet spot” of webMethods IS is in providing an highly productive environment to build and to execute integrations to transform or move data from multiple application packages or technology platforms that could not communicate without an external integration platform. That value is what allows it to command the license fees and the productivity of creating those integrations is why it does not make sense to custom code integration logic using general purpose tools.

Mark

15

Your point is right on Mark but it’s simply more anecdotal evidence of why IS isn’t good as an app server.

Let me try a different approach:

Pretend I don’t know much about IS. Explain to me–without using sports analogies :wink: – what are the specific issues with trying to do so. We all have an intuitive feel that it is indeed wrong, but I’m looking for more than intuition–give me something concrete. (and I’ll play devil’s advocate on anything that might be brought up–just to make sure we’ve thought it through).

16

I must apologize to Mark for characterizing his post as mostly anecdotal. While there is some of that in there he has listed a few concrete reasons why using IS for application development is lacking.

I wonder what the effort would be to create an Eclipse plug-in and use that as Developer? Probably bigger than anyone would really care to tackle.

17

Rob,

No worries. I was having trouble posting that day due to some extra anti-spam filtering that my host added. I posted the bulk of that message after your initial reponse.

I keep hearing that some future version of Developer will actually BE an Eclipse plugin, but I don’t know if that is still the current path or not.

Mark

18

:bump

19

I also heard that Modeler was moving to Eclipse…

Other…

I am very interested in more discussion on this topic of why IS is or isn’t good for the application layer in n-tier. If you were to remove budget (license cost) from the equation, would that balance the scale? I am very aware that IS is capable of providing the full solution - as this is exactly what I work with today - and I work with a large team of wM developers, all of whom are convinced that we would add unnecessary complexity to our solution if we begin to depend on services and ansynchronous processes outside of our “compact” solution. Indeed, our solution to handle capacity issues has simply been to throw more CPU’s at the system… addmitingly, this is not a favourable solution anymore and we’re tasked with reigning in some of our horizontal expansion and providing a solution with more facets to scale.