Event Driven APIs with API Gateway and AMQP

Knowledge base
, ,
1

Author: Sai Sailesh Singh (SaiSaileshSingh.Venkatagiri@softwareag.com) 
Supported Versions: 10.3 and above

webMethods API Gateway tutorial

Overview of the tutorial

This tutorial explains in detail about the support of AMQP 1.0 protocol in API Gateway and how AMQP runtime policies in API Gateway can help to achieve asynchronous communication with native services which are developed in different languages and platforms.

Let us go through the following steps in detail:

  • Configuring message brokers
  • Configuring AMQP settings in API Gateway
  • Configuring AMQP policies in a REST API

Required knowledge

This tutorial assumes that the reader has:

  • Basic knowledge of API Gateway APIs and policies
  • Knowledge on Messaging systems, JMS, AMQP and Message brokers

Why?

Most of Java EE developers know that JMS is a standard Java API for communicating with Message Oriented  (MOM). As a matter of fact, JMS is a part of the Java EE which allows two different Java applications to communicate. Those applications could be JMS clients and yet kept being decoupled. It is considered to be robust and mature. Since JMS is part of Java EE, it is typically used when both client and servers are running in a JVM. What if client and server are using a different language or platform? AMQP comes to the rescue. AMQP - Advanced Message Queuing Protocol is an open source published standard for asynchronous messaging by wire. It enables encrypted and messaging between organizations and applications across multiple platforms. It can go P-2-P (One-2-One), publish/subscribe, and some more, in a manner of reliability and secured way. Some of the main features of AMQP are listed below.

  • Platform independent wire level messaging protocol
  • Consumer driven messaging
  • across multiple languages and platforms
  • It is the wire level protocol
  • Can achieve high performance
  • Supports long lived messaging
  • Has support for SASL and TLS

So here is why API Gateway needs AMQP support in addition to the JMS support.

Asynchronous communication

By Enabling AMQP protocol for an API,  the API is powered with asynchronous messaging capabilities across different platforms and it can scale up with the abilities to support message driven or event driven architecture.We can expose our REST/SOAP APIS over JMS/AMQP to support an asynchronous style of communication and to scale for larger data processing without downtime issues.

Protocol bridging

REST to JMS/AMQP

Exposes the REST API over JMS/AMQP 

JMS/AMQP to REST

Exposes the JMS/AMQP message queue or topic as a HTTP REST API

SOAP to JMS/AMQP

Exposes the SOAP API over JMS/AMQP

JMS/AMQP to SOAP

Exposes the JMS/AMQP message queue or topic as a HTTP SOAP API

Prerequisite steps

For on-premise edition, complete the below prerequisites to make you ready to get into the details of the AMQP 1.0 protocol support in API Gateway.

  • Install API Gateway of version 10.3 or higher
  • Install Universal messaging broker or Active MQ broker or any other broker that supports AMQP 1.0 protocol

Details

In the following sections we will go through the AMQP 1.0 protocol support in API Gateway in detail.

Step 1: Configure message brokers

Active MQ server

In this tutorial we will use Active MQ server as the Message broker. Active MQ server supports AMQP 1.0 protocol and the default the port for AMQP is 5672.

API Gateway allows to use any Message broker which supports AMQP 1.0 protocol

Universal Messaging Server

In order to use UM server, enable connectivity from external parties to Universal Messaging  server topics/channels using AMQP. For that, nsp or nsps interface should be created and it should be running.These ports can be created from Enterprise Manager, Comms tab.

1266×156 77.2 KB

Step 2: Configure JNDI and JMS settings in API Gateway

The following section describes the settings necessary for AMQP support in API Gateway.

Only mandatory configurations like creating JNDI alias and JMS alias will be explained in this tutorial. Please refer API Gateway administration guide's JMS configurations section for complete details.

JNDI provider alias

  • Each JMS provider can store JMS administered objects in a standardize namespace called the Java Naming and Directory Interface (JNDI). JNDI is a Java API that provides naming and directory functionality to Java applications
  • Create one or more JNDI provider aliases to specify where API Gateway can look up administered objects when it needs to create a connection to JMS/AMQP provider or specify a destination for sending or receiving messages

Go to API Gateway UI → Administration → General → Messaging. Click on Add JNDI provider alias

1292×507 18.6 KB

Type the below details.

  • JNDI Alias Name :  Name of JNDI alias
  • Description: description of the JNDI alias
  • Initial Context Factory: For example org.apache.qpid.jms.jndi.JmsInitialContextFactory for ActiveMQ
  • Provider URL:  This would point to a properties file location,ex: <Installation>\ IntegrationServer\amqp.properties)

Sample amqp.properties

#amqp.properties file should have below details
# Set the InitialContextFactory class to use
java.naming.factory.initial = org.apache.qpid.jms.jndi.JmsInitialContextFactory
# Define the required ConnectionFactory instances
connectionfactory.qpidConnectionFactory = amqp://<host>:<port>

Click on Add. Once JNDI alias has been created, test the JNDI by clicking on test and see the result message.

1263×233 14.3 KB

JMS connection alias

A JMS configuration specifies the information that API Gateway needs to establish an active connection between API Gateway and the JMS/AMQP provider. API Gateway uses a JMS connection alias to send messages and receive messages from the JMS/AMQP provider. To create JMS connection alias using this JNDI. Go to API Gateway UI → Administration → General → Messaging. Click on Add JMS connection alias.

1274×516 24.3 KB

Type the below details.

  • Connection alias name: The name of JMS connection
  • Description: JMS connection alias related description
  • JNDI provider alias: The name of JNDI alias (Here we are refering to a JNDI of name JNDI_AMQP which we have created earlier)
  • Connection factory lookup name: Name of the connection factory(It must be qpidConnectionFactory in case of AMQP)

Click on Add. Once JMS connection alias has been created, enable the JMS connection by clicking on enable and see the result message. Before you enable the JMS connection make sure whether Message broker is running. We have configured the Message broker details in amqp.properties while creating the JNDI, make sure the configured host name and port number is valid AMQP URL. Once the JMS connection alias is enabled, we can very well go and create the APIs to expose over JMS/AMQP.

Step 3: Create REST API with AMQP Inbound support

Note: Please refer SOAP over JMS in API Gateway for the configuring AMQP policies in SOAP APIs

This section explains the steps to expose a REST API over JMS/AMQP protocol.

Create a REST API in API Gateway (This tutorial uses EmployeeManagementService API for the demo). Edit the API go to Transport policies and add Enable JMS/AMQP policy to expose API over AMQP.  Add the policy to the API and activate the API. Once API has been activated, API Gateway will be listening to the configured input queues/topics for the input request. If request comes then API Gateway invokes the corresponding REST resource and put back the response message to replyTo queue/topic as mentioned in the input message.

1823×435 22.4 KB

Enabling AMQP policy configuration

The below section provides the details of the Inbound AMQP policy configuration.

Connection Alias Name

Name of the JMS connection

Input Source Name

Name of the Queue or Topic where API Gateway will listens for the input request

Input Source Type

The input source either Queue or Topic

Selector

These allow you to filter the messages that a MessageConsumer will receive. The filter is a relatively complex language that mimics the syntax of an SQL WHERE clause. The selector can use all message headers and properties for filtering, but can not use the message content. Selectors are mostly useful for Topics that broadcast a very large number of messages to its subscribers.

The way selectors work depends on the destination type:

  • On Queues, only messages that match the selector will be returned. Others stay in the queue (and thus can be read by a MessageConsumer with different selector)
  • On Topics, messages that do not match the selector will be ignored as if they have not been published


The following SQL elements are allowed in the selector.

Element
Description
Example Selector
Header Fields Any headers except JMSDestination, JMSExpiration and JMSReplyTo JMSPriority = 9
Properties Message properties that follow Java identifier naming releaseYear = 1982
String Literals String literals in single quotes, duplicate to escape title = 'Sam''s'
Number Literals Numbers in Java syntax (int and double) releaseYear = 1982
Boolean Literals TRUE and FALSE isAvailable = TRUE
( ) Brackets (releaseYear < 1980) OR (releaseYear > 1989)
AND, OR, NOT Logical operators (releaseYear < 1980) AND NOT (title = 'Thriller')
=, <>, <, <=, >, >= Comparison operators (releaseYear < 1980) AND (title <>'Thriller')
LIKE String comparison with wildcards '_' and '%' title LIKE 'Thrill%'
IN Find value in set of strings (more) title IN ('Off the wall', 'Thriller', 'Bad')
BETWEEN Check whether number is in range (both numbers inclusive) (more) releaseYear BETWEEN 1980 AND 1989
IS NULL, IS NOT NULL Check whether value is null or not null. releaseYear IS NOT NULL
*, +, -, / Arithmetic operators releaseYear * 2 > 2000 - 20

Resource

Resource name of the API

HTTP Method

HTTP method to invoke (GET, POST, PUT, DELETE)

Content-Type

Content type of the request body can be set here

Request Transformation policy changes (optional)

The usage of Request Transformation Policy is totally optional and based on the need. Use jms.* parameters to apply Request Transformation policy for the request message coming through message queue/topic. If Request transformation policy defined using request.* parameters then it is only applicable for the input HTTP requests. An example configuration is given below.

1806×449 19.5 KB

As per the above Request transformation policy, path parameter will be replaced by the requestId value. Here requestId is the JMS property of the input JMS message.

API Invocation

A sample JMS client program to invoke this API is given below.

HelloWorld.java

package org.apache.qpid.jms.example;


import javax.jms.*;
import javax.naming.Context;
import javax.naming.InitialContext;


public class HelloWorld {
    public static void main(String[] args) throws Exception {
        try {
            // The configuration for the Qpid InitialContextFactory has been supplied in
            // a jndi.properties file in the classpath, which results in it being picked
            // up automatically by the InitialContext constructor.
            Context context = new InitialContext();

            ConnectionFactory factory = (ConnectionFactory) context.lookup("myFactoryLookup");
            Destination queue = (Destination) context.lookup("myQueueLookup");

            Connection connection = factory.createConnection(System.getProperty("USER"), System.getProperty("PASSWORD"));
            connection.start();

            Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);

            MessageProducer messageProducer = session.createProducer(queue);

            TextMessage message = session.createTextMessage("Hello world!");
            //message.setStringProperty("API_KEY","API_KEY");
            message.setJMSReplyTo(new Queue() {
                @Override
                public String getQueueName() throws JMSException {
                    return "replyQueue";
                }
            });

            messageProducer.send(message, DeliveryMode.NON_PERSISTENT, Message.DEFAULT_PRIORITY, Message.DEFAULT_TIME_TO_LIVE);
            connection.close();
        } catch (Exception exp) {
            System.out.println("Caught exception, exiting.");
            exp.printStackTrace(System.out);
            System.exit(1);
        }
    }
}

jndi.properties

# Set the InitialContextFactory class to use
java.naming.factory.initial = org.apache.qpid.jms.jndi.JmsInitialContextFactory

# Define the required ConnectionFactory instances
# connectionfactory.<JNDI-lookup-name> = <URI>
connectionfactory.myFactoryLookup =amqp://localhost:5672

# Configure the necessary Queue and Topic objects
# queue.<JNDI-lookup-name> = <queue-name>
# topic.<JNDI-lookup-name> = <topic-name>
queue.myQueueLookup =apiInputQueue
topic.myTopicLookup =topic1

Please find the attached client program under download artifacts for your reference (the client program used Apache qpid libraries for JMS/AMQP support).

Step 4: Create REST API with AMQP Outbound support

This section explains the steps to expose native AMQP Message queue/topic to be accessible over HTTP protocol.

Create a REST API in API Gateway (This tutorial uses EmployeeManagementService API for the demo). Edit the API go to Routing policies and add JMS/AMQP Routing Policy to route request message to the Message queue/topic. Add the policy to the API and activate the API. When the API is invoked from the REST client, the API Gateway will route/post the input message to a Message Queue/Topic and look for the response in the ReplyTo Queue/Topic.

566×709 17.4 KB

JMS/AMQP Routing policy configuration

The below section provides the details of the Inbound AMQP policy configuration.

Connection Alias Name: Name of the JMS connection

Destionation Name: Name of the Queue or Topic to post/route the request message

Destination Type: Type of the destination, Either Queue or Topic

ReplyTo Name: Name of the Queue or Topic to which the API Gateway look for the response message for the earlier posted request to the Destination Name

ReplyToType: Either Queue or Topic

Time to Live: It sets the time to live of a message put onto a queue/topic, in milliseconds. A value of  means live indefinitely

Time to Wait: It sets the waiting time to the API Gateway for the response message

Delivery Mode: The delivery modes, persistent or non persistent. The Persistent delivery mode, instructs the JMS provider to take extra care to ensure that a message is not lost in transit in case of a JMS provider failure. A message sent with this delivery mode is logged to stable storage when it is sent.

Limitations

  • Durable Topic is not supported

References

The below links provides a good collection of the resources on AMQP.

Downloadable artifacts

Learn more