Introduction
This article talks about the best practices while implementing the hybrid use case on webMethods.io integration.
Audience
It is assumed that readers of this article know how to do implementation on webMethods.io Integration
Why do we need Hybrid Integration?
-
With the current marketing trend neither migrating an all-cloud integration application system is the right approach for all enterprises nor is continuing with on-premise systems.
-
Hybrid cloud integration provides a best-of-both-worlds solution.
-
With Hybrid integration, we can integrate your on-premises applications (databases, warehouses, applications, and legacy systems) with cloud-based applications such as SaaS/PaaS-based applications.
-
For more details please refer to below link:
How to setup the hybrid connectivity with webMethods.io integration
• Please follow the below document for step-by-step implementation of hybrid use case
Hybrid Integration - webMethods.io Integration
-
Video tutorial to implement the hybrid use case.
Hybrid Integrations | webMethods.io Integration Tutorials - YouTube
Hybrid Integration on webMethods.io Integration
Now when we start implementing the hybrid use case on wemethods.io integration, we need to consider the multiple factors discussed below.
Multiple and dedicated instances of on-premise Integration Server
-
When we are setting up the hybrid connectivity between the on-premise server and cloud tenant then we should consider having multiple or dedicated instances. This will provide an edge to the connectivity and performance when one of the On-premise IS(integration servers) is not available. Then the request will be routed to the other server which provides high availability (HA) and also in the core business hours when the load is quite high then the on-prem system can handle it efficiently.
-
Video tutorial to implement hybrid use case with multiple on prem server.
Hybrid Connectivity with Cluster On-Prem environment | webMethods.io Integration - YouTube
Tuning of Thread pools dedicated to Hybrid Connectivity
• Customization of thread pools becomes an important aspect when we are aware of the load\ request per min coming to the on-premise server from the cloud.
• Whenever we install the on-prem server. Post-installation it is recommended to verify the configuration of server like the number of server threads on IS server, percentage(%) of threads allocated to the hybrid connectivity, and so on.
• In the below screenshot, we have a vanilla IS server installed and we can see the maximum number of threads is 75 .
• Out of these 75 threads by default only 5 % of threads are allocated for the Hybrid connection. So approximately we are allocating only 4 threads for a hybrid.
- In order to extract the best performance, we should calculate the number of threads required at the server and hybrid connection as per the request we are going to get from the cloud system.
Multiple Accounts
- Whenever we are creating the on-premise accounts we should consider to segregate the accounts with respect to the types of requests coming from upstream.
- When we have segregated requests then we can have multiple accounts. Accounts are nothing but they are the kind of listeners that can serve an incoming request.
- So if we have more listeners we can have multiple requests getting processed without any hiccups.
o With multiple accounts it will also remove the bottleneck of depending only on one account.
Tuning & Scaling of IS instance on the on-prem side
- When we have the scenario where the request is coming from an upstream system for hybrid integration, we need to do the load testing to see how the on-prem resources utilization works. If the resource utilization is high then we need to allocate more resources to the on-premise server for smooth functioning.
Latest on-prem fix related to cloud
- Our R&D team keeps on working to enhance the performance of the hybrid connections. To achieve the same R&D team provides the fixes which will not only enhance the performance but also make sure the connectivity resumes on it own whenever there is an upgrade happens on the cloud system or whenever on premise IS server restarts.
- It is always recommended to have the latest fix installed on-premise especially when the load is high and the customer is looking for optimal performance.
Aggregation of request
-
Whenever we are sending the request\payload we need to see how effectively we can use the on-prem connection.
-
Suppose we have exposed our webmethods.io integration flow service as API to the external partner and we are using the hybrid connection in the flow service then we can’t do much with respect to aggregation of requests while sending to on-premise. But if the service is not exposed in real-time then we can aggregate all the requests based on some logic and then send an aggregated request to the on-premise server. It will help us to use resources wisely.
Payload size
- When we are submitting the request from the cloud system to on-premise we should examine the request and should have minimum data that can cater to the purpose.
- Hybrid integration on webmethods.io integration is achieved using Universal messaging. We should understand we cant send huge MB’s of data from on cloud to on-premise for processing.
- And if we have to send this big amount of data then there is a glitch in the application logic.
- If the payload sent from the cloud to on-premise is in MB’s then use some compression techniques before submitting it to on-premise.
Points to remember
- Normally when we are implementing the hybrid use case we typically ignore the infra limitation. Any integration iPaaS product provides infra to integrate different systems but we have to understand even though we are using iPaas product we need to use them wisely. For example, when we are logging the incoming payload (size 5-10 MB) we should log only those fields which are essential to track the instance instead of logging the full payload. This not only bombards the system with unnecessary logging but also degrades the performance of the platform. It looks very odd that performance will degrade if we log 5 MB data but what if we are getting 1 million records per day and we are logging all those requests?
- Application design architecture should be reviewed carefully. We should try to avoid as much as we can using hybrid integration.
- Integration of 2 cloud systems will always have better performance.
What works for one team might be completely inadequate for another.