New in 10.1 – HTTP Server Connectivity Plugin

Knowledge base
1

In Apama 10.1, we introduced a connectivity plug-in to allow Apama to act as a simple HTTP server, such as within the community edition or on edge devices. For more fully featured HTTP server use cases, we recommend that Apama be used in conjunction with other Software AG products, such as the webMethods Integration Server or Cumulocity.

The intent of this plug-in is to provide a light-weight HTTP server alternative that doesn’t require a JVM, as is required by the existing Java sample.

The server supports:

  • HTTP 1.1
  • Any HTTP verb (GET, POST, PUT, DELETE)
  • Automatic success and response messages
  • Serving static files from disk (for example, a Swagger definition)
  • HTTPS, using TLS 1.2 or higher
  • Authentication via BASIC from a list of hashed passwords in a file
  • Persistent connections

We currently don’t support:

  • Websockets
  • Form-handling codec (for POST)
  • EPL being able to supply responses
  • IPv6

We also provide a tool to generate the password file for use with BASIC authentication.

Assumed Knowledge

  • Correlator configuration using a YAML file.
  • Setting up connectivity plug-ins using the above YAML file.

Constructing a ‘Hello World’ Example

The simplest way of using the HTTP server is to have a fixed, known API, no security and no authentication. This obviously isn’t recommended for a production environment but provides a demonstration of how quickly the server can be set up for use. First we need to tell the correlator that we want to use the HTTP server.

Using the HTTP Server Plug-in

Within Designer you can add the plug-in to your project by adding the HTTP Server connectivity bundle to the project. This will open the properties file of the default instance for editing.

Alternatively, you can load the transport with the following connectivityPlugins section within your correlator YAML configuration file:


# Define a chain manager. Connection details.
dynamicChainManagers:
    httpServer:
        transport: httpServer
        managerConfig:
            # Alter these to suitable values for your set-up.
            port: 443
            bindAddress: 10.13.23.125
# Define the chain. Mapping and codecs.
dynamicChains:
    httpServer:
        - apama.eventMap
            defaultEventType: HelloWorld
            defaultChannel: HelloWorldChannel
        # There are no mapper rules required here as there is a one-to-one mapping
        - jsonCodec
        - stringCodec
        - httpServer:
            automaticResponses: true
            allowedMethods: [PUT]

The EPL

Finally, we need the EPL to do something on receipt of the event generated from HTTP request:

event HelloWorld {}
 
monitor HelloWorldMonitor
{
    action onload()
    {
        monitor.subscribe(“HelloWorldChannel”);
        on all HelloWorld ()
        {
            log "Hello World" at INFO;
        }
   
        // signal that we are ready to receive events
        ConnectivityPlugins.onApplicationInitialized()
    }
}

Running the Example

Run the project from within Designer or otherwise start up your correlator with a —config argument specifying the YAML configuration file. Then we need to send in a request. This could be done using another correlator using our HTTP client connectivity plug-in or with another application, such as curl.

curl -X PUT 'http://host:port/’ -d ' {}'

You should see the INFO line ‘Hello World’ appear in the correlator log for each request you send.

To run using the HTTP client, you will need to create a complementary YAML file to that used by the HTTP server, along with a short EPL file to create the request events and receive the responses. An example of EPL sending a request every second, is:

event HelloWorld {}
 
event HTTPRequest
{
    integer id;
    string path;
    HelloWorld data;
    string method;
}
 
event HTTPResponse
{
    integer id;
    string body;
    string code;
    string message;
}
 
event HTTPError
{
    integer id;
    integer code;
    string message;
}
   
monitor HelloWorldSender
{
    action onload()
    {
        on all wait(1.0)
        {
            integer requestId := integer.incrementCounter("requestId");
   
            on HTTPResponse(id=requestId) as resp and not HTTPError(id=requestId)
            {
                log "Received response from HTTP server: " + resp.toString() at INFO;
            }
   
            on HTTPError(id=requestId) as err and not HTTPResponse(id=requestId)
            {
                log "Received error from HTTP server: " + err.toString() at ERROR;
            }
   
            send HTTPRequest(requestId, "/sample", HelloWorld(), "PUT") to "httpClient";
        }
  
        ConnectivityPlugins.onApplicationInitialized();
    }
}

The chain in the client’s yaml file will be:


startChains:
  httpClient:
    - apama.eventMap
    - mapperCodec:
        apamax.httpserversample.HTTPRequest:
          towardsTransport:
            mapFrom:
              - metadata.http.path: payload.path
              - metadata.requestId: payload.id
              - metadata.http.method: payload.method
              - payload: payload.data
            defaultValue:
              - metadata.contentType: application/json
              - metadata.sag.type: HelloWorld
        HTTPResponse:
          towardsHost:
            mapFrom:
              - payload.body: payload
              - payload.id: metadata.requestId
              - payload.code: metadata.http.statusCode
              - payload.message: metadata.http.statusReason
        HTTPError:
          towardsHost:
            mapFrom:
              - payload.id: metadata.requestId
              - payload.code: metadata.http.statusCode
              - payload.message: metadata.http.statusReason
    - classifierCodec:
        rules:
          - HTTPResponse:
            - metadata.http.statusCode: 202
          - HTTPError:
            - metadata.http.statusCode:
    - jsonCodec:
        filterOnContentType: true
    - stringCodec
    - HTTPClient:
        host: ${CORRELATOR_HOST}
        port: ${CORRELATOR_PORT}

Constructing a More Complex Example

Now let’s try a more complex example utilizing more of the features available within the HTTP server. We can imagine an IoT scenario where we are using this plug-in to receive events from a temperature sensor. We’ll construct an example that takes us as far as receiving the data ready for normalization and processing.

The Event and Request

The event we’ll be dealing with this time will be a bit more complex. The EPL for the event will be:

event Temperature
{
    string sensorId;
    float value;
    float timestamp;
}

For this example we will be wanting to dynamically specify the event type to use as we can imagine other types of data being received, such as pressure or humidity. This can be done in a number of ways including specifying the event type and channel in the request header or as a query. For this example we will be supplying the event type and channel as part of the payload, to process the following curl command:

curl -X PUT
     -u user:password 'https://host:port/'
     -d '{"type":"Temperature", "channel":"TemperatureChannel", "data":{"sensorId":"sensor001", "value":31.41, "timestamp":1549385584}}'

Authentication

Here we are using the BASIC authentication (via the -u argument). For this we require a password file containing a list of users and their passwords, compatible with Apache’s htpasswd -B. We provide a small application to accomplish this, httpserver_passman , which allows us to create this file. For this example we’ll use the command line:

httpserver_passman IoT_users.txt -c user password

Now we can add the authentication to the dynamicChains part of our correlator configuration:

dynamicChains:
    httpServer:
    - apama.eventMap
    # mapping rules to be added
    - httpServer:
        authentication:
            authenticationType: HTTP_BASIC
            allowedUsersFile: IoT_users.txt

TLS

We are also using TLS (Transport Layer Security) in this example, and hence must provide certificate and key files. These are specified in the dynamicChainManagers section of the correlator configuration:

dynamicChainManagers:
    httpServer:
        transport: httpServer
        managerConfig:
            port: 443
            bindAddress: 10.13.23.125
            tls: true
            tlsKeyFile: ${PARENT_DIR}/servername.key.pem
            tlsCertificateFile: ${PARENT_DIR}/servername.cert.pem

Mapping

Then we need to tell the plug-in how to map the incoming request to the event we need in EPL:

dynamicChains:
    httpServer:
        - apama.eventMap
        - mapperCodec:
            "*":
                towardsHost:
                    mapFrom:
                        - metadata.sag.type: payload.eventType
                        - metadata.sag.channel: payload.channel
                        - payload: payload.data
        # Authentication as above.

EPL

And finally, in addition to the event we specified at the start of this example, we need an EPL monitor to process the events:

monitor TemperatureMonitor
{
    action onload()
    {
        monitor.subscribe("Temperature Channel");
        on all Temperature() as t
        {
            log "Received: " + t.toString() at INFO;
            # Process t, for example, by normalizing for use in the Industry Analytics Kit.
        }
    }
}

Swagger

So far we have assumed that we know the API for the request. But what if we don’t, or we want to keep open the possibility of changing it? For this we can use the ability to GET static files to retrieve a Swagger file from the server.

Swagger or OpenAPI 2.0 is an open standard for defining REST APIs. A Swagger definition allows you to define the transport mechanism, data, requests and responses, and to document the required RESTful API in a human and machine readable format, allowing for easy discovery and integration with the server. For further information, take a look at https://swagger.io.

This GET request is also a special case for the HTTP server. In the case of retrieving a static file, the GET is not passed on to the correlator, but processed by the plug-in itself. As such, if you are only using GET for static files, it does not have to be specified in the allowedMethods member of the configuration.

Each static file that might be retrieved must be specified individually within the configuration, so to add this to the previous example we would have:

dynamicChains:
    httpServer:
        - apama.eventMap
        - mapperCodec:
            ###
        - httpServer:
            ###
        staticFiles:
            /swagger.json:
                file: ${PARENT_DIR}/swagger.json
                contentType: application/json
                charset: utf-8

The contents of the Swagger file then specify the API:


{
  "swagger": "2.0",
  "info":
  {
    "version": "1",
    "title": "Temperature API",
    "description": "A simple IoT Temperature API."
  },
  "schemes": ["http"],
  "consumes": ["application/json"],
  "produces": ["application/json"],
  "definitions":
  {
    "data": {"type": "object"},
    "eventAndPayload":
    {
      "type": "object",
      "required": ["eventType", "channel", "data"],
      "properties":
      {
        "eventType": {"type": "string"},
        "channel": {"type": "string"},
        "data": {"$ref": "#/definitions/data"}
      }
    }
  },
  "paths":
  {
    "/":
    {
      "put":
      {
        "summary": "Send event to a named channel",
        "description": "The JSON payload contains an event consisting of fields.",
        "parameters":
        [{
          "name": "data",
          "description": "Event fields",
          "in": "body",
          "schema": {"$ref": "#/definitions/eventAndPayload"}
        }],
        "responses":
        {
          "202": {"description": "Accepted"},
          "400": {"description": "Bad Request"},
          "401": {"description": "Unauthorized"},
          "405": {"description": "Method Not Allowed"},
          "429": {"description": "Too Many Requests"},
          "500": {"description": "Internal Server Error"}
        }
      }
    }
  }
}

Full Documentation

Full documentation on the HTTP server can be found in the online documentation for Apama 10.1 under “The HTTP Server Transport Connectivity Plug-in“.

Feel free to post on our forums with any questions you may have about this, or any other, topic.

Disclaimer

Utilities and samples shown here are not official parts of the Software AG products. These utilities and samples are not eligible for technical support through Software AG Customer Care. Software AG makes no guarantees pertaining to the functionality, scalability, robustness, or degree of testing of these utilities and samples. Customers are strongly advised to consider these utilities and samples as “working examples” from which they should build and test their own solutions.