Best practices for Cumulocity real-time APIs

Overview

Cumulocity clients, such as devices, microservices, web applications or even external systems, often need to run certain logic when new data arrives or existing data is modified. They need to be notified by Cumulocity to do this.

This article discusses different mechanisms for notifying Cumulocity clients. It introduces the technology behind the mechanisms briefly, reviews their trade-offs and concludes with a summary. The mechanisms are:

  • MQTT.
  • cometd.
    • Long polling.
    • Websockets.
  • Complex event processing.
  • Polling.

Note: The article focuses on the Cumulocity standard APIs. If you would like to provide a notification API in a custom microservice of yours, we recommend to either use the Cumulocity APIs or REST hooks. Exposure of custom publish/subscribe APIs through the Cumulocity microservice proxy is not supported.

MQTT

MQTT is a TCP-based publish-subscribe protocol. While MQTT cannot handle request-response communication well, it has been purpose-built for publish-subscribe communication and notifications. To get notified of new or modified data, a client opens a permanent connection to the server, subscribes to topics and gets notifications sent back on the connection.

Cumulocity implements MQTT for efficient device communication as described in the Device SDK. It can currently only be used for device communication and you can only be notified of commands and errors. The main purpose of MQTT in Cumulocity is to stream data from devices and to deliver commands to devices.  To do this in a very reliable fashion, MQTT includes quality of service (QoS) guarantees. For example, "QoS 1" means that commands are guaranteed to be delivered at least once.

cometd

cometd is an HTTP-based publish-subscribe protocol. It builds on the fact that HTTP is ubiquitous and supported by just about any programming language and device. While other protocols are often blocked in enterprises, HTTP is usually available everywhere.

HTTP itself is a request-response protocol: A client sends a request (like a "GET" request to retrieve information) and receives a response (the requested information). The basic HTTP protocol has no built-in functionality to connect from server to client, or to notify clients. It is stateless and typically not aware of its clients outside of the immediate request.

Hence, a protocol on top of HTTP is required to implement this functionality: cometd. On a high-level, cometd works like this:

  • The client sends a "handshake" HTTP request to initiate the communication and its parameters. The server responds with a client ID to identify the communication channel (HTTP is stateless).
  • The client sends subscribe requests containing the topics to be notified of and the client ID.
  • The server sends notifications about these topics. 

The last step can be implemented in several different ways. Cumulocity supports long polling and websockets. For more details on the Cumulocity cometd implementation and the payload format, please refer to the Reference Guide.

Long polling

Long polling is a trivial extension of the basic HTTP protocol for very simple HTTP clients and low traffic volumes. On a high-level, it works like this:

  • The client sends an HTTP request including the client ID, indicating that it wants notifications to be sent. 
  • The server keep the HTTP request open until a notification needs to be sent. It then sends the notification as a response.
  • The client connects again.

Essentially, the connection from the client to the server looks like a normal HTTP request, but it is "hanging" until there is something to respond. (There is a lot more to the protocol that is covered in the cometd documentation.)

Long polling works fine under the following conditions:

  • The HTTP client can only handle basic HTTP communication.
  • Notifications occur irregularly and infrequently, and you need to react immediately.

Long polling is not suitable for high volume traffic. If notifications occur frequently, a long polling client would constantly connect, return immediately and connect again. This is very inefficient and results in high bandwidth usage. We recommend using Websockets over long polling.

Websockets

Websockets is a standard part of the HTTP protocol defined in RFC6455 and available in modern HTTP implementations. It permits full bidirectional communication on top of HTTP by opening a stable TCP connection from client to server. It is most commonly used in web browsers to dynamically update a web application. 

On a high-level, it works like this:

  • The client sends a GET request to the server, requesting a connection upgrade to web sockets. The server indicates that it can switch and keeps a TCP connection open to the client.
  • Client and server can bidirectionally exchange data (in Cumulocity's case, in cometd format with Cumulocity payload).

This is a much more efficient form of communication similar, as the communication channel remains open all the time. You can actually check yourself how this mechanism works by opening your browser's developer tools on, e.g., Cumulocity Cockpit, and by searching for a request with the URL "realtime". In Chrome, you can follow up on the messages passed back and forth in the tab "Messages".

cometd notifications over websockets are supported by the Cumulocity Web SDK. There are also many implementations of cometd for other programming languages.

Websockets is a high-performance notification mechanism, but it currently only supports simple filtering options in Cumulocity. Depending on your connectivity, there is a limit on the maximum number of notifications that you can transmit through a single websocket connection. You can open several websocket connections for different devices and data types, but partitioning is limited to these options.

Please also note that you will not receive change notifications between connection attempts to the server. So if your connection breaks, depending on your use case, you may have to query missed data through the REST APIs.

Complex event processing

If you are subscribed to the complex event processing feature, your complex event processing rules will receive every change in your tenant. If you need to do calculations and data transformations, using complex event processing is certainly the easiest way to do so.

Depending on the complexity of the logic and the amount of events that you need to process, there are different "T-shirt sizes" of event processing engines that you can subscribe to (s/m/l), which provide different number of CPU cores and memory limits. If Apama cannot process events fast enough, Cumulocity will show an alarm in your tenant ("Real-time event process is currently overloaded") signalling that events have been lost. This may be resolved by increasing capacity, depending on the nature of application (e.g. whether the logic is making use of multiple contexts, large amount of memory, etc).

Polling

If you are expecting data to change regularly and/or in high volumes, polling is an efficient protocol. For example, you know that your devices send data once per hour in a reliable fashion. In that case, the most efficient method is to query the data once per hour. Notifications would only add protocol overhead here.

Summary

In this article, we have reviewed the currently available mechanisms for getting new data from Cumulocity. In summary:

  • Use MQTT for devices.
  • Use Websockets for web applications and other medium traffic clients that need to be updated immediately.
  • Use complex event processing if you need post process data along the way.
  • Poll the REST APIs for regular data extraction, or when your network does not permit you to use the long-standing connections that the other protocols require.

You can certainly also mix and match the methods. For example, listen to alarms through websockets, because alarms are low volume and you want to react quickly, and extract measurements through querying.