API Mashups in API Gateway 10.4

[Clench_Paign_James3 (Clench Paign James)]

Author: Raj, Gokul (gora@softwareag.com)
Supported Versions: 10.4 and above

Please refer API Mashups in API Gateway for version 10.3

webMethods API Gateway tutorial

Overview of the tutorial

In a typical microservices architecture, application is split into multiple services based on business functions or architectural decisions. The granularity of APIs provided by microservices is often different than what a client needs. Microservices typically provide fine-grained APIs, which means that applications need to interact with multiple services.

In such cases, API gateway can provide a single entry point for all clients. The API gateway can handle requests in one of two ways. Some requests are simply proxied/routed to the appropriate service. It handles other requests by fanning out to multiple services by composing the individual microservices/APIs in to one mashed up API. This reduces the number of requests/round trips that the client application otherwise will have to make.

Here the API Gateway will often handle a request by invoking multiple microservices and aggregating the results before providing a response.

An API mashup is an API that orchestrates multiple APIs and exposes them as a single API. API Gateway provides Chaining and Parallel invoke model for API Mashup. Based on the use case one can orchestrate the services using any one of the model.

• Chaining invoke model is chaining the API invoke in a sequence order with one api invoke at one level, where the response of an API invoke available as request for the next API invoke.
• Unlike chaining, in parallel invoke model we can configure multiple api invoke in one level of sequence order. Aggregator is used to aggregate the response of the multiple api invoke call at any given level, and the aggregated payload is available for the next level of api invoke. 

API Mashups: API mashups in API Gateway used for simple orchestration that involves multiple external callouts and response aggregation. For more complex use cases like conditional branching, looping requirements, it is recommended to use webMethods Integration Server.

In this tutorial we will go through the following steps in detail.

• Creating a Mashup API
• Adding steps and aggregator to the Mashup API
• Invoking the Mashup API

Required knowledge

The tutorial assumes that the reader has,

• a basic understanding of API Gateway and its policy enforcement
• a good knowledge on APIs and their definitions

Why?

Servers that provides APIs may expose a vast set of functionality and each of these individual functionalities or services can be exposed as APIs in API Gateway. While this is usually effective, sometimes it is useful or required to consolidate a few services and expose them as a single service. In other situations, you might want to extend a service with the functionality provided by an external API. API mashups address these requirements for grouping services and exposing them as a single service.

Lets see a sample usage scenario.

Let's take for our example an API that provides information about courses offered by different universities in a given location. This API returns a list of universities for a given course name and postal code. The URI of the API is GET /universities?course=medicine&postalcode=600001
The provider of the API wants to extend this API for use in mobile applications that have access to users’ location. As mobile applications can access a user’s location in terms of longitude and latitude, the invocation of this API involves first retrieving the postal code for the users’ current location and then passing that information to the existing API. Lets say there is a publicly available service that returns the postal code based on the longitude and latitude values. The URI of the API is GET /postalcode?latitude=333&longitude=444444

Using an API mashup, we can create and expose a single API that calls both the APIs, the API that returns the postal code by calling the external service and the existing API that provides the list of universities. The resulting mashup API would be GET /universities?course=medicine&lat=333&long=444444

Prerequisite steps

Complete the below prerequisites to make you ready to get into the details API mashup in API Gateway.

• Install API Gateway of version 10.4 or above.
• Install postman REST client of version v6.0.10 or later
• Create the REST APIs in API Gateway by importing the attached archive files (PostalCodeAPI.zip, UniversitiesAPI.zip)
• Install java 1.8 if not available. Run the java web application for the native service endpoints (java -jar apimashup.jar)

Details

An API mashup allows you to orchestrate multiple resources and methods from different Gateway APIs and expose the behavior as a single API. Mashup API contains one or more steps and aggregators. A step invokes an API and the aggregator aggregate the response of the previous steps. Before going into the use cases we will see how to construct the API mashup using chaining and parallel invoke models in below sections.

API Mashup Construction

This section contains information on how to add invoke step and aggregator and connect the same for chaining and parallel invoke. On clicking Add invoke and Add aggregator, it will create a invoke step and an aggregator without any connections. The policy action parameters respective to the step will get rendered in the right panel.

Now, the user has to manually connect the step and aggregator with the start and stop. On hovering the top and bottom of invoke step and aggregator you can see the section got highlighted which indicates the connection port.  The connection port at the top used for connection in and the bottom used for connection out. We will discuss few construction model in detail below.

Chaining Model

In this model we connect two invoke steps with one aggregator in chaining. For this we need two invoke steps and an aggregator. 

Connect: Start → Invoke step → Invoke step → Aggregator → End.

Parallel invoke Model

In this model, we connect two invoke steps in parallel with one aggregator. 

Connect: Start → Invoke step 1 and again Start → Invoke step 2. Now aggregator will automatically got created for the parallel invoke. Aggregator → End.

Complex Model

In this model, we combines Chaining and Parallel invokes to create mashup. For e.g the mashup in the below image contains six levels.

1. Two invoke step in parallel
2. Aggregator
3. Two invoke step in parallel
4. Aggregator
5. Invoke step
6. Aggregator

Delete Invoke step, Aggregator and Link

In case if there is a wrong configuration of API Mashup we can delete and re-link the control as required. To delete a invoke step there is a delete icon on the top right and also we can select a particular invoke step and delete using keyboard delete button. To delete aggregator and link select it and delete with keyboard delete button. For some delete actions you will get an warning message, which means you can't delete as this will collapse the API Mashup configuration.

API Chaining Use case

Step 1: Create a Mashup API

Go to APIs and click Create API. Provide the API name and configure the resource universities with the query parameters latitude, longitude and course as below and Save the API.

Now click Edit and select Mashups tab. The Mashups tab appears and It displays the resources of the API and their methods on the left panel and an empty Default routing step. 

In the list of resources, click the resource in which you want to include the mashup. The resource tab expands and the methods included in the resource are displayed. Click the toggle button to enable the method in which you want to create the mashup. 

After enabling the resource, you can see a start and end in the middle with Add invoke and Add aggregator at the top right position. On the top left you can view the miniature version of the mashup.

Note: If the API that does not have any mashup, the Mashup tab displays the list of resources only in the Edit mode, the tab is empty in the view mode. If you use the toggle button and disable a method that has a mashup, the mashup definition for that method is immediately cleared

Step 2: Add default step to the Mashup API

Now lets add our first step for the PostalCode API. This API returns the postalCode in the response for the given latitude and longitude query values. The below screenshot shows the invocation of PostalCode API and its response.

Click Add invoke. The Mashup Routing panel appears on the right. Enter a Mashup step name that is unique. Select the API Endpoint that you want to invoke in this mashup step by providing the API Gateway API, Resource and Method as in the below screenshot. Add query parameters latitude and longitude would be retrieved from the incoming request's context using a special syntax ${variable_name}. They can be accessed using ${request.query.latitude} and ${request.query.longitude}. A detailed coverage of how the values from the request and steps are accessed is explained in the next sub section. Now along with these query parameters add a custom pipleline variable course whose value would be retrieved from the incoming request's query parameter course. The course pipleline variable would be used in the next step Universities for retrieving the list of universities. Click Save to save the API. Now the mashup API is created for the selected resource and method. As a result of this step the request would call the PostalCode API by sending latitude and longitude query parameters and receive the postalCode in response.

Mashup step inputs

Here is the detailed explanation of all the input fields in the Mashup step.

• Should execute Outbound policies - If you want the outbound security policies of the participating API to be enforced in the context of an API mashup
• Use incoming Headers - Uses the headers in the incoming request
• Custom Headers - You can add custom headers in this section
• Query Parameters - The query parameters and values to be sent to the current invoking step
• Path Parameters - The path parameters and values to be sent to the current invoking step
• Payload - The payload to be sent to the current invoking step. We can also extract a particular value from the incoming request payload
• Advanced transformation - API Provider can define a custom implementation for request and response transformation using IS services and configure them using the Invoke webmethods IS service for an API Mashup step
• Transformation metadata - For all XPath, we can configure namespace prefix and namespace URI, which can be used throughout the policy configuration
• Custom Pipeline Variables - We can configure a pipeline variable here to hold values that would be used in the next steps of the Mashup API. Pipeline variable can be accessed by the syntax ${pipeline_variable_name}. Example: ${course}

The values such as headers, query parameters, payload, etc from the previous steps can be accessed using a special syntax of aliases. The below section describes the syntax in detail.

Variable types and aliases

In this section we can discuss how the values such as headers, query parameters, etc, from previous steps can be accessed from in a step. The below table shows the variable types and their possible values.

Object/Variable	Type Possible values
paramStage	request response
paramType	payload or body headers query path httpMethod statusCode statusMessage
queryType	xpath jsonPath regex

• ${paramStage.paramType} - You can use this syntax to access the following string variables: path, statusCode, statusMessage, httpMethod. Examples: ${request.path}, ${response.statusCode}
• ${paramStage.paramType.paramName} - You can use this syntax to access map types, such as query, headers, and path. Example: ${request.query.var1}, ${response.header.Content-Type}, ${request.path.name}.
• ${paramStage.paramType.queryType[queryValue]} - This syntax can be used to query a paramType. Examples:
• ${request.payload.xpath[//ns:emp/ns:empName]} - Where "//ns:emp/ns:empName" is the XPath to be applied on the payload if contentType is "application/xml", "text/xml", or "text/html".
• ${response.payload.jsonPath[$.cardDetails.number]} - Where "$.cardDetails.number" is the jsonPath to be applied on payload if contentType is "application/json" or "application/json/badgerfish".
• ${request.payload.regex[[0-9]+]} - Where "[0-9]+" is the regular expression to be applied on the payload if contentType is "text/plain".

While xpath and jsonPath are applicable only to payload, regEx can be used with both payload and path

• ${paramStage[stepName].paramType.queryType[queryValue]} - You can use this syntax to access data from any of the previous steps. For example, you can use the following syntax to access the payload of the step named PostalCodeAPI: ${response[PostalCodeAPI].payload.jsonPath[$.postalCode]}.

Step 3: Add second step to the Mashup API

Now lets add the step which would retrieve the universities list based on the course and postalCode query parameters. The below screenshot shows the invocation of Universities API and its response.

Both the course and postalCode parameter values would be extracted from the first step's custom pipeline variable and response respectively. Click Add invoke. Provide the values of UniversitiesAPI endpoint details and the query parameters course and postalCode as in the below screenshot. The value for retrieving the postalCode from the PostalCode step's response uses JSON path syntax in the alias.

Note: API should be in edit mode to insert, update and delete mashup steps. Select a mashup step to view its policy properties in the right panel. Selected mashup step is highlighted with shadow background and a delete icon in the top right corner. When you click Add Invoke button a mashup step will be added next to the selected step. To delete a particular step in mashup, just click the delete icon in the top right corner.

Step 4: Invoke the Mashup API

The below screenshot shows a sample invocation of the mashup API and its response. On invocation the Postal Code step would be executed first, which would invoke the PostalCodeAPI API to retrieve the postCode for the latitude and longitude of 333 and 444444 respectively. Then the second step Universities would be executed, which would invoke the UniversitiesAPI API to get the list of universities for the course value medicine and the postalCode value 600001 which is retrieved from the previous step.

The sample URL: http://localhost:5555/gateway/MashupAPI/1.0/universities?course=medicine&latitude=333&longitude=444444

Policy Execution

By default, the policies of an API that is participating in an API mashup are not enforced when it is invoked within the API mashup. However, if you select the Should execute Outbound policies option, the outbound security policies of the participating API are enforced in the context of the API mashup.

Data Flow

None of the following are automatically passed from one step to another in an API mashup:

• Headers
• Query parameters
• Path parameters
• Payload

You must add and use parameters in an API mashup step to access data from any of the previous steps or another source. An exception to this rule is the first step (the first participating API) in a mashup, which can receive all the data (the complete request) from request sent from the client.

More samples

Below section provides more samples to better understand the use cases of API Mashup chaining.

Use case 1

As a prerequisite, Import the Service Management service and Policy Management service from <Installation Folder>\IntegrationServer\instances\default\packages\WmAPIGateway\resources\apigatewayservices APIGatewayServiceManagement.zip and APIGatewayPolicyManagement.zip respectively.

When you create an API from API Gateway UI, the UI invokes a set of internal REST services which creates an API and associates default policies. You can then proceed to activate the API. API Gateway provides REST services for automating all the above steps. In this example, we will use API Mashups to provide a single service which will help create an API, add some policies (apart from the default), associate these policies to the API and activate the API using the API Gateway REST services. 

Let's see the steps in detail

• Step 1: /apis method of Service Management Service will create an API in API Gateway with the given payload
• Step 2: /policyActions method of Policy Management Service will create a Policy Action with the given payload
• Step 3: /policies method of Policy Management Service will create a Policy using the Policy Actions created in Step 2
• Step 4:  /apis/{apiId}/activate method of Service Management Service activate the API created in Step 1
• Step 5: /policies/{policyId}/activate method of Service Management Service activate the Policy created in Step 3

Create a REST API (Demo Usecase 1) from scratch with POST resource(/create/activate/secured) and add mashups to this resource. The mashups consist of 5 steps.

Step Name	Description	API	Service	Method	Payload/Path Param
1. createAPI	Create API using payload	/apis	Service Management	POST	ChainingUC1_createAPI.json
2.createPolicyAction	Create policyAction using Payload	/policyActions	Policy Management	POST	ChainingUC1_createPolicyAction.json
3. createPolicy	Create Policy with policy action created in Step 2	/policies	Policy Management	POST	ChainingUC1_createPolicy.json
4. activateAPI	Activate API created in Step 1	/apis/{apiId}/activate	Service Management	PUT	apiId -> ${response[createAPI].payload.jsonPath[$.apiResponse.api.id]}
5. activatePolicy	Activate Policy created in Step 3	/policies/{policyId}/ activate	Policy Management	PUT	policyId -> ${response[createPolicy].payload.jsonPath[$.policy.id]}

Save and Activate the API, and invoke using createAPI payload with Postman or any other rest client. API will be created with the given name in the payload and associated with a policy action as described. The above described Mashups is available as attachment Demo Usecase 1.zip.

Use case 2

In this use case, we are going to accumulate the responses obtained from all the Mashup steps using Invoke ESB service.

Import the Service Management service from APIGatewayServiceManagement.zip located at <Installation Folder>\IntegrationServer\instances\default\packages\WmAPIGateway\resources\apigatewayservices. Copy ResponseAccumulation.zip in <Installation Folder>\IntegrationServer\instances\default\replicate\inbound and go to IS → Packages → Management → Install Inbound Releases → Choose the ResponseAccumulation.zip in the drop down and click Install. Create a Custom Service API in API Gateway using the Native endpoint(http://localhost:5555/rest/Response), resource(/accumulation) and method(POST). Create a REST API (Demo Usecase 2) from scratch with POST resource(/accumulate) and add mashups to this resource. The mashups consist of 3 steps.

Step Name	Description	API	Service	Method	Payload/Path Param
1. CreateAPI	Create API using payload	/apis	Service Management	POST	ChainingUC2_createAPI.json
2. ActivateAPI	Activate API created in Step 1	/apis/{apiId}/ activate	Service Management	PUT	apiId -> ${response[createAPI].payload.jsonPath[$.apiResponse.api.id]}
3. ResAccu	Accumulate the response of Step 1 and 2	/accumulate	Inbound Service	POST	[${response[CreateAPI].payload},${response[ActivateAPI].payload}]

Save and Activate the API, and invoke using CreateAPI payload with Postman or any other rest client. API will be created with the given name in the payload. The above described Mashups is available as an attachment in Demo Usecase 2.zip.

API Parallel invoke Use case

API parallel invoke is similar to API chaining with the addition that we can add multiple invoke steps at the same level and aggregate their response using aggregators.

Use case 1

As a prerequisite, Create API Service Management service and Application service using files APIGatewayServiceManagement.json from <Installation Folder>\IntegrationServer\instances\default\packages\WmAPIGateway\resources\apigatewayservices and Application.json from <Installation Folder>\IntegrationServer\instances\default\packages\WmAPIGateway\resources\apigatewayservices respectively, ignore if already created.

In this use case, we are going to create an API and Application in parallel and then we are going to aggregate the response using aggregator. Then, we are going to associate the api and application created in the previous step. 

Let's see the steps in detail

• Step 1 Parallel invoke 1: /apis method of Service Management Service will create an API in API Gateway with the given payload
• Step 1 Parallel invoke 2: /applications method of Application Service will create an Application in API Gateway with the given payload
• Step 2: aggregate the response from Step 1 and Step 2
• Step 3 Parallel invoke 1:  /applications/{applicationId}/apis method of Application Service will associate the api and application created in Step 1
• Step 3 Parallel invoke 2: /apis/{apiId}/activate method of Application Service will activate the API created in Step 1
• Step 4: Aggregate the response from Step 1 and 3.

Create a REST API (Parallel Demo 1) from scratch with POST resource(/create/associate/api/application) and add mashups to this resource. The mashups consist of 3 steps.

Step Name	Description	API	Service	Method	Payload / Path Param
1. CreateAPI	Create API using given payload	/apis	Service Management Service	POST	ParallelUC1_createAPI.json
2. CreateApplication	Create application using given payload	/applications	Application Management Service	POST	ParallelUC1_CreateApplication.json
3. Aggregate response	Aggregate the response of Step 1 and Step 2 using Aggregator.				Create a custom pipeline variable for apiID and applicationID using variable framework  ${response[CreateAPI].payload.jsonPath[$.apiResponse.api.id]}  and  ${response[CreateApplication].payload.jsonPath[$.id]} respectively.
4. Associate	Associate the API and Application created in the Step 1 and Step 2	/applications/ {applicationId}/apis	Application Management Service	PUT	Payload: {"apiIDs":["${apiID}"]} Path Param: Use the custom pipeline variable created in the aggregator as ${applicationID}
5. ActivateAPI	Activate the API created in Step 1	/apis/{apiId}/ activate	Service Management Service	PUT	Path Param: Use the custom pipeline variable created in the aggregator as ${apiID}
6. Aggregate response	Aggregate the response of Step 1, 2, 4 and 5.				Finally you will receive the aggregated response.

Save and Activate the API, and invoke using below payload with Postman or any other rest client. API and Application will be created, activated and associated with the given name in the payload. The above described Mashups is available as an attachment as Parallel_Invoke_Demo_1.zip.

{
"apiName": "Parallel Invoke API",
"applicationName": "Parallel Invoke Application",
"apiURL": "http://petstore.swagger.io/v2/swagger.json"
}

The below screen shows the API invocation and its response.

API Mashup Aggregator

In addition to other sections, in aggregator we have Mashup Response Transaction which will take care of aggregating the responses of the previous steps. If the Aggregate response checkbox is checked then the aggregator will automatically aggregate the responses of the previous invoke steps using it's step name. If the user wants to change the aggregated response then they can provide their custom payload in the payload section with Aggregate response to false. 

Limitations

• Currently API Gateway supports API mashups for REST APIs only. Also only REST APIs can be added in the mashup steps
• API Gateway doesn't support mashups of external API Gateway APIs as well as third party APIs. It supports the API Gateway APIs in the current instance
• API Mashup through composite APIs isn't supported as of now

Tryout

If you want to play around with Mashup APIs, you can access the below cloud instance where few APIs are created for this purpose. Please find the APIs Demo Usecase 1, Demo usecase 2 and MashupAPI in the cloud instance.

To use our API Click the below link

https://single4.webmcloud.com/apicloud/login#/login

Login using the below credentials.

Username: single4

Password: Test@123

A Postman collection for this example also provided in the below git-hub page. Clone the repository into your local system.

https://github.com/Malligarjunan/apigateway/tree/master/postmanCollections/dev-tutorials

Install Postman and import the Postman collection API Mashup in API Gateway.postman_collection.json from the above git hub link. Now you are ready to invoke the APIs.

Downloadable artifacts

apimashup.jar (15.5 MB)
Demo Usecase 1.zip (71.2 KB)
Demo Usecase 2.zip (41.7 KB)
MashupAPI.zip (17.2 KB)
Parallel_Invoke_Demo_1.zip (79.9 KB)
ResponseAccumulation.zip (6.3 KB)

Learn more

• For details on usage of API Mashups in API Gateway 10.3, please refer API Mashups Developer Tutorial