Author: Sai Sailesh Singh (SaiSaileshSingh.Venkatagiri@softwareag.com)
Supported Versions: 10.3 and above
webMethods API Gateway tutorial
Overview of the tutorial
This tutorial explains in detail about the support of AMQP 1.0 protocol in API Gateway and how AMQP runtime policies in API Gateway can help to achieve asynchronous communication with native services which are developed in different languages and platforms.
Let us go through the following steps in detail:
- Configuring message brokers
- Configuring AMQP settings in API Gateway
- Configuring AMQP policies in a REST API
Required knowledge
This tutorial assumes that the reader has:
- Basic knowledge of API Gateway APIs and policies
- Knowledge on Messaging systems, JMS, AMQP and Message brokers
Why?
Most of Java EE developers know that JMS is a standard Java API for communicating with Message Oriented (MOM). As a matter of fact, JMS is a part of the Java EE which allows two different Java applications to communicate. Those applications could be JMS clients and yet kept being decoupled. It is considered to be robust and mature. Since JMS is part of Java EE, it is typically used when both client and servers are running in a JVM. What if client and server are using a different language or platform? AMQP comes to the rescue. AMQP - Advanced Message Queuing Protocol is an open source published standard for asynchronous messaging by wire. It enables encrypted and messaging between organizations and applications across multiple platforms. It can go P-2-P (One-2-One), publish/subscribe, and some more, in a manner of reliability and secured way. Some of the main features of AMQP are listed below.
- Platform independent wire level messaging protocol
- Consumer driven messaging
- across multiple languages and platforms
- It is the wire level protocol
- Can achieve high performance
- Supports long lived messaging
- Has support for SASL and TLS
So here is why API Gateway needs AMQP support in addition to the JMS support.
Asynchronous communication
By Enabling AMQP protocol for an API, the API is powered with asynchronous messaging capabilities across different platforms and it can scale up with the abilities to support message driven or event driven architecture.We can expose our REST/SOAP APIS over JMS/AMQP to support an asynchronous style of communication and to scale for larger data processing without downtime issues.
Protocol bridging
REST to JMS/AMQP
Exposes the REST API over JMS/AMQP
JMS/AMQP to REST
Exposes the JMS/AMQP message queue or topic as a HTTP REST API
SOAP to JMS/AMQP
Exposes the SOAP API over JMS/AMQP
JMS/AMQP to SOAP
Exposes the JMS/AMQP message queue or topic as a HTTP SOAP API
Prerequisite steps
For on-premise edition, complete the below prerequisites to make you ready to get into the details of the AMQP 1.0 protocol support in API Gateway.
- Install API Gateway of version 10.3 or higher
- Install Universal messaging broker or Active MQ broker or any other broker that supports AMQP 1.0 protocol
Details
In the following sections we will go through the AMQP 1.0 protocol support in API Gateway in detail.
Step 1: Configure message brokers
Active MQ server
In this tutorial we will use Active MQ server as the Message broker. Active MQ server supports AMQP 1.0 protocol and the default the port for AMQP is 5672.
Universal Messaging Server
In order to use UM server, enable connectivity from external parties to Universal Messaging server topics/channels using AMQP. For that, nsp or nsps interface should be created and it should be running.These ports can be created from Enterprise Manager, Comms tab.
Step 2: Configure JNDI and JMS settings in API Gateway
The following section describes the settings necessary for AMQP support in API Gateway.
JNDI provider alias
- Each JMS provider can store JMS administered objects in a standardize namespace called the Java Naming and Directory Interface (JNDI). JNDI is a Java API that provides naming and directory functionality to Java applications
- Create one or more JNDI provider aliases to specify where API Gateway can look up administered objects when it needs to create a connection to JMS/AMQP provider or specify a destination for sending or receiving messages
Go to API Gateway UI → Administration → General → Messaging. Click on Add JNDI provider alias
Type the below details.
- JNDI Alias Name : Name of JNDI alias
- Description: description of the JNDI alias
- Initial Context Factory: For example org.apache.qpid.jms.jndi.JmsInitialContextFactory for ActiveMQ
- Provider URL: This would point to a properties file location,ex: <Installation>\ IntegrationServer\amqp.properties)
Sample amqp.properties
#amqp.properties file should have below details
# Set the InitialContextFactory class to use
java.naming.factory.initial = org.apache.qpid.jms.jndi.JmsInitialContextFactory
# Define the required ConnectionFactory instances
connectionfactory.qpidConnectionFactory = amqp://<host>:<port>
Click on Add. Once JNDI alias has been created, test the JNDI by clicking on test and see the result message.
JMS connection alias
A JMS configuration specifies the information that API Gateway needs to establish an active connection between API Gateway and the JMS/AMQP provider. API Gateway uses a JMS connection alias to send messages and receive messages from the JMS/AMQP provider. To create JMS connection alias using this JNDI. Go to API Gateway UI → Administration → General → Messaging. Click on Add JMS connection alias.
Type the below details.
- Connection alias name: The name of JMS connection
- Description: JMS connection alias related description
- JNDI provider alias: The name of JNDI alias (Here we are refering to a JNDI of name JNDI_AMQP which we have created earlier)
- Connection factory lookup name: Name of the connection factory(It must be qpidConnectionFactory in case of AMQP)
Click on Add. Once JMS connection alias has been created, enable the JMS connection by clicking on enable and see the result message. Before you enable the JMS connection make sure whether Message broker is running. We have configured the Message broker details in amqp.properties while creating the JNDI, make sure the configured host name and port number is valid AMQP URL. Once the JMS connection alias is enabled, we can very well go and create the APIs to expose over JMS/AMQP.
Step 3: Create REST API with AMQP Inbound support
This section explains the steps to expose a REST API over JMS/AMQP protocol.
Create a REST API in API Gateway (This tutorial uses EmployeeManagementService API for the demo). Edit the API go to Transport policies and add Enable JMS/AMQP policy to expose API over AMQP. Add the policy to the API and activate the API. Once API has been activated, API Gateway will be listening to the configured input queues/topics for the input request. If request comes then API Gateway invokes the corresponding REST resource and put back the response message to replyTo queue/topic as mentioned in the input message.
Enabling AMQP policy configuration
The below section provides the details of the Inbound AMQP policy configuration.
Connection Alias Name
Name of the JMS connection
Input Source Name
Name of the Queue or Topic where API Gateway will listens for the input request
Input Source Type
The input source either Queue or Topic
Selector
These allow you to filter the messages that a MessageConsumer will receive. The filter is a relatively complex language that mimics the syntax of an SQL WHERE clause. The selector can use all message headers and properties for filtering, but can not use the message content. Selectors are mostly useful for Topics that broadcast a very large number of messages to its subscribers.
The way selectors work depends on the destination type:
- On Queues, only messages that match the selector will be returned. Others stay in the queue (and thus can be read by a MessageConsumer with different selector)
- On Topics, messages that do not match the selector will be ignored as if they have not been published
The following SQL elements are allowed in the selector.
Element
|
Description
|
Example Selector
|
---|---|---|
Header Fields | Any headers except JMSDestination, JMSExpiration and JMSReplyTo | JMSPriority = 9 |
Properties | Message properties that follow Java identifier naming | releaseYear = 1982 |
String Literals | String literals in single quotes, duplicate to escape | title = 'Sam''s' |
Number Literals | Numbers in Java syntax (int and double) | releaseYear = 1982 |
Boolean Literals | TRUE and FALSE | isAvailable = TRUE |
( ) | Brackets | (releaseYear < 1980) OR (releaseYear > 1989) |
AND, OR, NOT | Logical operators | (releaseYear < 1980) AND NOT (title = 'Thriller') |
=, <>, <, <=, >, >= | Comparison operators | (releaseYear < 1980) AND (title <>'Thriller') |
LIKE | String comparison with wildcards '_' and '%' | title LIKE 'Thrill%' |
IN | Find value in set of strings (more) | title IN ('Off the wall', 'Thriller', 'Bad') |
BETWEEN | Check whether number is in range (both numbers inclusive) (more) | releaseYear BETWEEN 1980 AND 1989 |
IS NULL, IS NOT NULL | Check whether value is null or not null. | releaseYear IS NOT NULL |
*, +, -, / | Arithmetic operators | releaseYear * 2 > 2000 - 20 |
Resource
Resource name of the API
HTTP Method
HTTP method to invoke (GET, POST, PUT, DELETE)
Content-Type
Content type of the request body can be set here
Request Transformation policy changes (optional)
The usage of Request Transformation Policy is totally optional and based on the need. Use jms.* parameters to apply Request Transformation policy for the request message coming through message queue/topic. If Request transformation policy defined using request.* parameters then it is only applicable for the input HTTP requests. An example configuration is given below.
As per the above Request transformation policy, path parameter will be replaced by the requestId value. Here requestId is the JMS property of the input JMS message.
API Invocation
A sample JMS client program to invoke this API is given below.
HelloWorld.java
package org.apache.qpid.jms.example;
import javax.jms.*;
import javax.naming.Context;
import javax.naming.InitialContext;
public class HelloWorld {
public static void main(String[] args) throws Exception {
try {
// The configuration for the Qpid InitialContextFactory has been supplied in
// a jndi.properties file in the classpath, which results in it being picked
// up automatically by the InitialContext constructor.
Context context = new InitialContext();
ConnectionFactory factory = (ConnectionFactory) context.lookup("myFactoryLookup");
Destination queue = (Destination) context.lookup("myQueueLookup");
Connection connection = factory.createConnection(System.getProperty("USER"), System.getProperty("PASSWORD"));
connection.start();
Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
MessageProducer messageProducer = session.createProducer(queue);
TextMessage message = session.createTextMessage("Hello world!");
//message.setStringProperty("API_KEY","API_KEY");
message.setJMSReplyTo(new Queue() {
@Override
public String getQueueName() throws JMSException {
return "replyQueue";
}
});
messageProducer.send(message, DeliveryMode.NON_PERSISTENT, Message.DEFAULT_PRIORITY, Message.DEFAULT_TIME_TO_LIVE);
connection.close();
} catch (Exception exp) {
System.out.println("Caught exception, exiting.");
exp.printStackTrace(System.out);
System.exit(1);
}
}
}
jndi.properties
# Set the InitialContextFactory class to use
java.naming.factory.initial = org.apache.qpid.jms.jndi.JmsInitialContextFactory
# Define the required ConnectionFactory instances
# connectionfactory.<JNDI-lookup-name> = <URI>
connectionfactory.myFactoryLookup =amqp://localhost:5672
# Configure the necessary Queue and Topic objects
# queue.<JNDI-lookup-name> = <queue-name>
# topic.<JNDI-lookup-name> = <topic-name>
queue.myQueueLookup =apiInputQueue
topic.myTopicLookup =topic1
Please find the attached client program under download artifacts for your reference (the client program used Apache qpid libraries for JMS/AMQP support).
Step 4: Create REST API with AMQP Outbound support
This section explains the steps to expose native AMQP Message queue/topic to be accessible over HTTP protocol.
Create a REST API in API Gateway (This tutorial uses EmployeeManagementService API for the demo). Edit the API go to Routing policies and add JMS/AMQP Routing Policy to route request message to the Message queue/topic. Add the policy to the API and activate the API. When the API is invoked from the REST client, the API Gateway will route/post the input message to a Message Queue/Topic and look for the response in the ReplyTo Queue/Topic.
JMS/AMQP Routing policy configuration
The below section provides the details of the Inbound AMQP policy configuration.
Connection Alias Name: Name of the JMS connection
Destionation Name: Name of the Queue or Topic to post/route the request message
Destination Type: Type of the destination, Either Queue or Topic
ReplyTo Name: Name of the Queue or Topic to which the API Gateway look for the response message for the earlier posted request to the Destination Name
ReplyToType: Either Queue or Topic
Time to Live: It sets the time to live of a message put onto a queue/topic, in milliseconds. A value of means live indefinitely
Time to Wait: It sets the waiting time to the API Gateway for the response message
Delivery Mode: The delivery modes, persistent or non persistent. The Persistent delivery mode, instructs the JMS provider to take extra care to ensure that a message is not lost in transit in case of a JMS provider failure. A message sent with this delivery mode is logged to stable storage when it is sent.
Limitations
- Durable Topic is not supported
References
The below links provides a good collection of the resources on AMQP.
- https://en.wikipedia.org/wiki/Advanced_Message_Queuing_Protocol
- http://www.wmrichards.com/amqp.pdf
- https://github.com/apache/qpid-proton-j
- https://www.oasis-open.org/committees/download.php/56418/amqp-bindmap-jms-v1.0-wd06.pdf
- https://access.redhat.com/documentation/en-us/red_hat_enterprise_mrg/3/html/messaging_programming_reference/differences_between_amqp_0-10_and_amqp_1.0
- http://techcommunity.softwareag.com/pwiki/-/wiki/Main/Using+webMethods+Integration+Server+with+Apache+Qpid
- https://timjansen.github.io/jarfiller/guide/jms/selectors.xhtml
Downloadable artifacts
Learn more
- Please refer SOAP over JMS in API Gateway for a detailed view on configuring JMS policies in API Gateway