webMethods API Gateway tutorial
Author: Sidduraj, Malligarjunan (Malligarjunan.Sidduraj@softwareag.com)
Supported Versions: 10.2 and above
Overview of the tutorial
As we all know, with drastic changes and immense evolution of the internet, Websockets made a breakthrough and revolution in the client-server communication by providing a bidirectional, full-duplex communication between server and client over a single TCP connection. API Gateway, to be inline with this hot growing protocol's market, provides support for proxying Websocket APIs from 10.2 on wards.
This tutorial covers the below steps in detail to help you understand how a Websocket API can be created and invoked in API Gateway.
- Installing sample IS package for native endpoint
- Creating Websocket port in API Gateway
- Creating Websocket API and policy enforcements
- Invoking Websocket API from a client
Required knowledge
The tutorial assumes that the reader has:
- a good knowledge on the Websocket protocol
- a basic understanding of API Gateway and its policy enforcements
Why?
With the tremendous support already available for SOAP and REST API proxies, API Gateway, before 10.2, was lacking support for Websockets, which is now becoming a demanding requirement for a real time server to client two way communication. Before Wesockets arrival this was instead accomplished by means of the time and memory consuming long polling using REST/Ajax calls and Server-sent events. Lets discuss a use case to better understand the scenario of why Websockets are necessary in API Gateway.
A customer would like to use Websocket protocol to access the UM (Universal Messaging) queues by the external web clients whose traffic needs to be routed through API Gateway to apply the runtime security policies instead of the direct connection of the web client to UM. Similarly the external web applications initiate subscribing to that particular queue and start listening to the events in that queue in a secured way.
The above said problem can be well explained using a case study. Lets take for our example a telematics company that provides tailored solutions for the insurance and automotive industries by collecting contextual vehicle, location and crash data as well as driver behavior information to support usage-based insurance policies. A car theft has been detected and/or notified by an on-board device. The on-board device receives a command from the device management software to activate track mode. The IoT pipeline is receiving and processing the tracking events pumped from the on board device. Now various users (the car owner, call center users, etc.) may use the customer / CRM portals and they open a page for subscribing to the tracking events. From the subscription page an API is called to generate the event subscription and the API actually creates a subscriber to the IoT pipeline tracking events (Universal Messaging) with the given filter and returns to the browser the subscription id and properties. The created subscriber begin to en-queue tracking events and wait for the Websocket to be opened on the browser side and the browser immediately opens a Websocket to receive the tracking events. Tracking events flow towards the browser and the information is presented to the user. The API Gateway is placed between the IoT pipeline to UM and UM to the browser so that all the runtime mediation policies such as security, message transformation policies provided by the API Gateway Websocket API can be leveraged.
Prerequisite steps
Complete the below prerequisites to make you ready to get into the details of the Websockets API support in API Gateway.
- Install API Gateway of version 10.2 or above
- Install the attached IS package (websocketDemoPackage.zip) for our tutorial's native Websocket endpoint
Details
Installing sample IS package for native endpoint
API Gateway is a gateway or a proxy layer through which policies can be enforced and requires a native WebSocket server. Lets use the sample WebSocket demo IS Package provided as part of this tutorial to discuss and demonstrate the steps involved in getting familiar with the Websocket API. This sample package provides couple of useful services to publish or unpublish the Echo Broadcaster Websocket API on user preferred URL context. The Echo broadcaster API functionalities are given below:
- URL context is nothing but relative URI value from host:port
- Websocket client can connect to an API using WS protocol
- Echo broadcaster API listens for client messages
- Once the Echo broadcaster API receives the client messages it broadcast the client message to all clients (Acts Like a Chat Server)
- If an API is published with URL context that contains broadcast then the server every seconds sends the Server Time as a message to all clients
Now let see how we can test these functionalities in the below sections.
Deploying WebSocket API in Integration Server
While executing the service, by default, the websocketDemoPackage publishes the Echo Broadcaster API with the /api/demo/ URL context. You can access the Websocket through ws://host:wsport/api/demo/. When the Client sends a message, the Echo Broadcaster API broadcasts the message in the format "Message Received from {user}: {message}" to all the Clients. The {user} value is retrieved from query parameter user and the {message} is the message send by the Client. The sample package comes up with a Web UI using which you can use to test this native as well as API Gateway API. You can also test APIs by accessing the http://<IS_host>:port/websocketDemoPackage/demo.html page.
Deploying the Echo API with a Different URL Context
You can access the same API with a different URL contexts by invoking the flow service startDemoAPI that is available in the demo package. For example http://localhost:5555/invoke/websocketDemo:startDemoAPI?relativeURI=/websocket/demo/. Once the API is invoked you can modify the WebSocket URL to ws://host:WSport/websocket/demo/ and test the API. To unpublish the URL context you can invoke the stopDemoAPI flow service, for e.g. http://localhost:5555/invoke/websocketDemo:stopDemoAPI?relativeURI=/websocket/demo/
Deploying the Same Echo API with a broadcast URL Context
If the relativeURI has a broadcast string then the Server keeps sending the API with server time as one second to all clients.
For example invoke http://localhost:5555/invoke/websocketDemo:startDemoAPI?relativeURI=/websocket/demo/broadcast/. The server time appears in the client UI.
Creating Websocket port in API Gateway
Go to Adminitration → Security → Ports, click Add ports and type the details for the port number and alias and save the port detail.
After creating the port enable it. Now a Websocket port is created and would be listening for Websocket connections in API Gateway.
Creating Websocket API in API Gateway
In this section lets create a Webscoket API in API Gateway.
Go to APIs → Create API and select "Create API from scratch" option. Select WebSocket and click Create button.
Provide the Name and WS Url as ws://localhost:8082/api/demo/ and save the API.
You can also define the message information like Origin, Type, etc. Origin specifies from where the message originates, Type specifies the type of the message such as Text or Binary, Sample message payload specifies the message payload schema or simply a sample message payload and Message description explains about the message.
Invoking Websocket API from a client
Now lets invoke the SampleWebsocketAPI we have created. Open the sample Webscoket client web application provided by our sample IS package websocketDemoPackage using the link http://localhost:5555/websocketDemoPackage/demo.html
Type in ws://MCCLEP02:8082/websocket/EchoWS/1.0 in WebSocket URI, provide a message and click connect button. The Websocket connection would be made and ready for accepting messages. Type in some message in the Message field and click send. The message would be sent to the API Gateway and a response would be received and displayed in the client web application UI.
Execution flow
Here is the brief outline of what happens when a Websocket API is invoked in API Gateway. Once the the Websocket API is activated, the Websocket Client opens a connection and exchanges messages through API Gateway. The following figure describes the flow of messages between the client, API Gateway, and the server in a typical Websocket communication. It also describes at which point each of the API Gateway policies are called.
Enforcing policies on Websocket API
In this section lets discus in detail how runtime polices can be enforced for a Websocket API. The below policies are supported for Websocket API.
- Identify & Authorize Application
- Invoke webMethods IS (Client & Server messaging stages)
- Straight Through Routing (added by default)
- Log Invocation
- Throttling Traffic Optimization
Lets briefly discuss the configuration each of the policies in detail.
Identify & Authorize Application
This policy handles the Upgrade request from the client and allows a successful connection only if the client's API Key or host name or IP address matches with any of the registered applications of the API. The API key is sent in the x-Gateway-APIKey query parameter. If the validation fails, the upgrade request fails and rejected with 401 status code.
Invoke webMethods IS
This policy can be enforced on client message as well as the server message. The policy in Client Message Processing is used to transform the message before forwarding it to native Websocket API and the policy in Server Message Processing is used to transform the server message before forwarding it to Websocket clients.
Go to Policies and select Invoke webMethods IS policy. Type a valid IS namespace service in the webMethods IS Service field.
A sample code snippet for such service is given below.
private static final String STR_MESSSAGE_CONTEXT = "MessageContext";
private static final String STR_MESSSAGE_TYPE = "messageType";
private static final String STR_MESSSAGE = "message";
public static final void transformClientMessage(IData pipeline) throws ServiceException {
IDataCursor csr = null;
try {
csr = pipeline.getCursor();
MessageContext context = (MessageContext) IDataUtil.get(csr, STR_MESSSAGE_CONTEXT);
String messageType = (String) context.getProperty(STR_MESSSAGE_TYPE);
if (STR_TEXT.equalsIgnoreCase(messageType)) {
String message = (String) context.getProperty(STR_MESSSAGE);
if (message != null) {
message = message.concat(" Client Transformation complete");
context.setProperty(STR_MESSSAGE, message);
}
}
} catch (Exception e) {
LOG_RES.logger().error("0001.9999", "Processing Client Message Transformation failed. Error Message: " + e.getMessage());
}
}
In above code snippet, APIGateway sends org.apache.synapse.MessageContext as value for the key "MessageContext". From MessageContext, the client message can be retrieved. If the message type is Binary then API Gateway sends the value as java.nio.ByteBuffer as the message type.
Log Invocation
This policy is used to send the transaction log messages to applicable destinations. All logs sent to API Gateway destination can be visualized in the Analytics page. The configuration detail is given below.
Store Messages Sent From Client → Adds the client message payload in transaction log
Store Messages Sent From Server → Adds the server message payload in transaction log
Compress Payload Data → Compresses the message payload
Select the Log Generation Frequency as 'Always' since there is no success or failure concept with Websocket messages
Throttling Traffic Optimization
This policy throttles the number of connections that can be active for a Websocket API. The configuration detail is given below.
- Rule Name → Only value "Total Request Count" specifies that this throttling is done based on the number of active connections
- Operator → Only value "Greater Than" a comparison operator
- Value → Specifies the number of active connections allowed
When the number of connections exceeds the specified value for the specified applications, the HTTP upgrade requests fails with HTTP 503 status code.
Limitations
Below are some of the limitations associated with the Websocket APIs:
- Authentication with JWT (JSON Web Token) is not supported
- Basic auth enforcement is not supported
- Most of the policies applicable for SOAP and REST APIs such as Authorize User, Data Masking and routing policies are not supported for Websocket API
- Mashup and Scopes are not supported
- Websocket API can be created only from scratch and not by importing a file or URL
Troubleshooting
- If the native API URL context ends with / ( e.g /api/demo/) then the same must be configured in routing policy. Otherwise an error would occur.
References
Bunch of online resources to better understand the need for Websockets.
- https://stackoverflow.com/questions/44898882/why-to-use-websocket-and-what-is-the-advantage-of-using-it
- https://stackoverflow.com/questions/19169427/how-websockets-can-be-faster-than-a-simple-http-request
- https://www.pubnub.com/blog/2015-01-05-websockets-vs-rest-api-understanding-the-difference/
- https://blog.stanko.io/do-you-really-need-websockets-343aed40aa9b
- https://stackoverflow.com/questions/28613399/websocket-vs-rest-api-for-real-time-data/28618369#28618369
Downloadable artifacts
- Sample IS package for Websocket native API - websocketDemoPackage.zip
Learn more
- For details on usage of Invoke webMethods IS policy in versions 10.2 and above, refer Invoke webMethods IS policy in API Gateway 10.2