webMethods API Gateway version: 10.15 and above
Problem Statement
Ever been in a situation where one backend service starts to malfunction and eventually, its queuing in the API Gateway exhausting all the available threads, and having a knock-on effect with serving requests for the other APIs?
Solution
Restricting the number of concurrent API requests processed for a more robust and fault-tolerant API deployment
Microservice patterns can be extended to the API Gateway world to make the APIs resilient. One such pattern is Bulkhead, wherein elements of an application are isolated into pools so that if one fails, the others continue to function. API Gateway supports the implementation of the Bulkhead pattern through a policy in the Transport section of the API policy configuration.
What is Bulkhead?
The term bulkhead comes from the structure reliability engineering concept that is used in ships to create partitions in the hull. In the event of a hull breach, these isolated partitions prevent water from filling up the entire hull and sinking the ship. The aim is to limit and contain the damage to the isolated partitions.
The bulkhead pattern in software systems works similarly by isolating services to prevent them from starving the system of resources.
A ship without bulkheads sinks as the water fills the entire hull
A ship with bulkheads isolates and contains the damage
Enable Bulkhead Policy
Our approach uses a thread pool to separate the service from the caller and contain it to a subset of system resources.
Bulkhead configuration allows you to specify the maximum number of concurrent requests processed by an API. You can configure this setting individually for an API or globally for all APIs.
When the number of concurrent requests to an API exceeds the specified limit, the excess requests are rejected. In such scenarios, the policy violation events are generated to report the violations that occurred for an API. If there are 100 violations, then 100 policy violation events are generated.
You can control the response status code and the status message using the extended settings.
For detailed documentation, please refer Bulkhead Policy.
Want to try? Get started with our Free Forever Edition: Sign up - Software AG Cloud
This article is part of the TECHniques newsletter blog - technical tips and tricks for the Software AG community. Subscribe to receive our quarterly updates or read the latest issue.
Credit: openliberty.io