Battling integration challenges in the cloud

[system]

 

Issue 1, 2013

Download pdf

The real story behind cloud integration

 

Companies adopt cloud-based software primarily for two reasons – low cost of infrastructure and low maintenance. However, what they fail to realize is that there are inherent challenges with cloud-based models. Without proper solutions, those challenges can bring down the very benefits of the “low-cost” cloud model. This article highlights the key cloud integration challenges and the necessary solutions for solving them.

 

The Gray Side of Cloud

As dreamy as a fairy tale ending would sound, IT and business managers live happily ever after once they adopt the cloud. Does that sound right? Well, not really. Very few applications on the cloud have self-contained data i.e., they do not need any integration with the rest of the enterprise. 

 

Mostly, when a company adopts the cloud by taking on a new SaaS (Software-as-a-Service) application, it needs to 
integrate that application with the other
applications within its IT landscape. For example, an application such as Salesforce.com will definitely need to be integrated with other systems / databases in the enterprise since CRM (Customer Relationship Management) data is required in other systems such as ERPs or third-party sales applications (as shown in Figure 1). And, this is exactly where the nightmare begins for the IT manager.

 

Figure 1: Adopting new SaaS applications in the Cloud requires integration with multiple systems in the enterprise.

 

If you think that cloud applications can be easily integrated by calling an API or web service, think again. There are larger problems that are not quickly visible to the eye when you deal with cloud integration. The cloud application may seem like yet another endpoint but the entire concept of integration with the cloud is a paradigm shift. We will discuss that gray area in the sections below.

Key Challenges with Cloud Integration

Cloud integration vendors will tell you that integration with cloud applications is a “walk in the park” using their software. However, as a well-informed user, you need to be able to ignore the “cloud-washing”, focus on real problems and ask real questions. In order to help you do so, here are three broad buckets of challenges with cloud integration:

 

1. API Challenges
   The most obvious way to connect with cloud applications (SaaS) is to use SOAP / REST APIs. However, in spite of standards, every vendor has their own style of API implementation. Some use a single web service that contains all the operations while some use a web service that contains referenceable schemas. There may also be a few that have nested web services making it even more complex. To top this, each of these vendor APIs have thousands of operations and hundreds of data constructs.
   
   The documentation for all this runs into thousands of pages making it extremely difficult for a developer to find and understand the relevant operations for her integration needs. To top this complexity, vendors release an update to their API every three months or so. How are developers supposed to cope with that? This can easily turn out to be the single biggest maintenance nightmare for the IT team.
   
    
2. Runtime Challenges
   The key difference between the traditional way of integration and the new cloud integration is that we have the huge notion of distance between the applications. When you integrate an on-premise application like an ERP with a cloud application like Salesforce.com, you have already been introduced to the concept of latency since the latter is outside of the DMZ (aka firewall). So, unlike connecting two on-premise applications where network latency was hardly a factor, we have to consider that when integrating with the cloud.
   
   Bandwidth constraints lead us to think more about caching and compression techniques when passing data between the cloud and on-premise. Security will also be a key factor to consider when you are passing data outside of the firewall. 
   
   Governance.  Within the realm of SaaS applications and other cloud providers, usage-based billing and subscription models are quite common. Given that, your cloud services integration solution needs to track usage of all the service consumption that happens. This helps in identifying bad logic creating costly SaaS API consumption, rogue services using unnecessary bandwidth and also in reconciling with your SaaS providers’ usage claims. This is called usage monitoring or governance.
   
   Governance also has a bigger play in providing a mediation layer for all service consumption. This layer will enforce all types of policies including but not limited to SLAs, Security, Availability, Consumer Identification, etc. Ensure that the availability of governance is your single biggest criterion for selecting a Cloud Services Integration solution.
   
   
   “...Cloud Integration is a paradigm shift compared to traditional integration…”
    
3. Data Challenges
   Since we are talking about integration, data will be of utmost importance. When two or more cloud / on-premise applications need to be integrated together, there is an inherent need for the developer to understand the various data structures that need to be defined manually so that they can be mapped from one to the other. This takes away the main promise of “quick cloud integration”.
    
   How often should I synchronize my changes with the rest of my systems? As simple as that question seems at the outset, it is actually tricky to answer. Almost all of the SaaS subscriptions charge you based on number of service invocations or at least limit you to a tier of calls that you can make within a month. Considering such limits, it is prudent for you to measure how often you access your SaaS application since each call has a cost factor.
    
   The other factor to consider is latency. Your network bandwidth will definitely play a key role on how quickly you can access your SaaS data. So, if you are thinking of synchronizing your cloud data with your on-premise data as quickly and as often as possible, cost and performance are going to hit you right in the face.

 

Software AG webMethods Cloudstreams

With a very exciting 9.0 release of the webMethods Suite coming out in June 2013, Software AG will be launching a brand new product called webMethods CloudStreams. It is a cloud integration product that provides the ability to securely integrate with common SaaS applications out of the box, rapidly build custom integrations with other cloud-based applications, and provide monitoring and management of these integrations with actionable analytics.
 

Figure 2: webMethods CloudStreams connects SaaS and on-premise applications
 

webMethods CloudStreams creates and governs connections between any combination of SaaS and on-premise applications as shown in Figure 2. That helps organizations integrate SaaS applications in days rather than months, gain insights into their usage and ensure compliance with SaaS vendor contracts as well as with other self-governing policies.
 

Most other solutions in the market either require extensive custom development, or focus solely on either data or app integration and require separate governance solutions. Since webMethods CloudStreams is built on an industry-leading integration platform, it provides quick-cloud integration capabilities in bringing together data and applications, along with a strong governance engine. No other cloud integration provider offers governance capabilities. It also provides out of the box connectors to common SaaS applications and a wizard-driven framework to easily integrate any other cloud-based application.

 

webMethods CloudStreams also offers features such as:

• Rich mapping & transformation of data structures
• Service orchestration and mediation
• Extensible cloud connector framework to connect with any SaaS application
• Enforcement of SLA policies
• Dashboard of usage analytics and metrics
• Support for various types of WSDL and REST APIs
• Connectors for Salesforce.com, Workday, SuccessFactors, Google Apps, Twitter streams etc.

 

Conclusion

When shopping for a cloud integration solution, please consider the three broad challenges discussed above. Also, remember that usage consumption governance is a key area often forgotten in conjunction with cloud integration. 

 

webMethods CloudStreams offers the right platform to integrate cloud and on-premise applications with the same ease and flexibility. It has an extensive framework to support integration with any SaaS application. webMethods CloudStreams is a very unique product that addresses all those key challenges very effectively and keeps up the real promise of the cloud.