Block SDK describes how to write, test and deploy custom analytics builder blocks
By Kusuma Kumari, QA Specialist IoT & Analytics and Mounika Konathala, Senior Software Engineer IoT & Analytics, Software AG
Prerequisites
- Apama installation 10.5.0 or above
- Apama Analytics Builder Block SDK
- A Cumulocity cloud tenant (with admin user), or a Cumulocity Edge installation (both 10.5.0 or later)
Note: Custom block, and hence the Block SDK, are not supported with Apama Starter, you need a full version.
Introduction
Analytics Builder allows you to build analytic models using blocks that are capable of processing data in real time. Blocks are small bits of logic wired together to form models in model manager. Analytics Builder provides a set of pre-built blocks like finding the average, or the difference between inputs, etc. (Pre-built blocks). In the case where you want to have your own custom logic to be implemented and use them in models, this article gives you detailed information on how to add custom blocks to Analytics Builder using the Block Software Development Kit, which is called the Block SDK. The Block SDK 10.5.0 is introduced in Analytics Builder 10.5.0. You will need to git clone the Block SDK from github.
In this article let's use one of the samples provided with the Block SDK and upload it to a tenant that hosts Analytics Builder.
Blocks can be tested using the PySys test framework that is included in an Apama installation. To develop, test and package blocks, you will need a full installation of Apama.
Blocks are implemented in Apama's Event Processing Language (EPL). This guide assumes a working knowledge of EPL. Refer to the Apama documentation on developing EPL applications and the API Reference for EPL (ApamaDoc). This guide also assumes a working knowledge of the Analytics Builder data model. Refer to the Apama Analytics Builder documentation.
Using Software AG Designer
While blocks may be written in any text editor, Software AG Designer provides syntax highlighting, completion proposals and error reporting. For the Designer to provide completion proposals, it needs to be configured with the location of the block SDK. You can do this with the analytics_builder configure designer command line script, which is included in the Block SDK. Open an Apama Command Prompt window, change to the directory where you have cloned the Block SDK repository, and run the command line:
analytics_builder configure designerAfter the above command has run, restart Software AG Designer. Create an Apama project using Designer and add the bundles Analytics Builder and Cumulocity Block Helpers to the project.
Figure 1: ‘Add Bundle’ dialog showing Analytics Builder bundles after configuring with Block SDK
Create an EPL file Offset.mon under the Apama project as shown below:
Figure 2: Right-click on the project and select EPL Event Definition menu item
Figure 3: New EPL Event Definition dialog with default option selected
Figure 4: Creation of Offset block with package name
Figure 5: EPL editor
Writing a block
Let's look at one sample block, Offset, that adds a fixed offset of 100 to the supplied value. This block is available in the 'samples' directory of the Block SDK.
package apamax.analyticsbuilder.samples;
using apama.analyticsbuilder.BlockBase;
using apama.analyticsbuilder.Activation;
/**
* Offset.
*
* Add a fixed offset of 100 to the supplied value.
*
* @$blockCategory Calculations
*/
event Offset {
/**BlockBase object.
*
*/
BlockBase $base;
/**
* This action receives the input value and contains the logic of the block.
*
* @$inputName value Value
*/
action $process(Activation $activation, float $input_value) {
$setOutput_output($activation, $input_value + 100.0 );
}
/**
* Output.
*
* The output value.
*/
action<Activation, float> $setOutput_output;
}
$base is a field of type apama.analyticsbuilder.BlockBase that provides contextual information about the model the block is within. All blocks must have a BlockBase $base field.
$process in an action is where the main processing of the block will take place.
$activation is a parameter to that action to provide contextual information required when generating a block output.
Block inputs are specified as parameters whose name is prefixed with $input_ and the $process action may contain multiple input parameters.
Block outputs are each specified as separate action variables whose name is prefixed with $setOutput_, and again a block may contain multiple outputs.
When a model containing this block is activated, an instance of the block is created, initializing $base and $setOutput_* and calling the $process action when the block receives input.
If you create a model containing this block and the 'value' input wired to a source block and then activate the model, the model will call the $process action of our sample block every time a new value is available on the 'value' input.
Any calls this block makes to $setOutput_output will generate a new value on the block's output.
Adding the ApamaDoc tag @$blockCategory followed by a category name shows a block under the category in Analytics Builder's model editor.
For more detailed information refer to the "Basic Blocks" section of the Block SDK documentation.
Testing a block
You can test your custom blocks using PySys testing framework shipped with an Apama installation. You also need to set an environment variable, ANALYTICS_BUILDER_SDK, to the location of the Block SDK.
Below is a sample PySys test for the Offset block (run.py):
from pysys.constants import *
from apamax.analyticsbuilder.basetest import AnalyticsBuilderBaseTest
class PySysTest(AnalyticsBuilderBaseTest):
def execute(self):
correlator = self.startAnalyticsBuilderCorrelator(
blockSourceDir=f'{self.project.SOURCE}/blocks/')
modelId = self.createTestModel('apamax.analyticsbuilder.samples.Offset')
self.sendEventStrings(correlator,
self.timestamp(1),
self.inputEvent('value', 100.75, id=modelId),
self.timestamp(2),
self.inputEvent('value', 10.50, id=modelId),
self.timestamp(5),
)
def validate(self):
self.assertGrep('output.evt', expr=self.outputExpr('output', 200.75))
self.assertGrep('output.evt', expr=self.outputExpr('output', 110.50))
Each PySys test should contain pysystest.xml which describes the test purpose and run.py which checks the block's functionality.
To elaborate, run.py starts the correlator and injects necessary EPL files into the correlator. It then creates a test model that connects all inputs and outputs of the block under test, so you can send inputs to the block and see the block's outputs. that testing of the block is done in the local Apama installation, not within Analytics Builder running inside Cumulocity.
For more detailed information, refer to the "Testing blocks" section of the Block SDK documentation.
For more examples of tests, refer to Apama Analytics Builder Block SDK github repository.
Uploading a block to Analytics Builder
Once blocks are written and tested, you can upload by packaging them into "extensions" (which are .zip files).
From an Apama Command Prompt, run the below command to upload a new extension named "Offset". Note that extension names must be unique.
analytics_builder build extension --input <location of samples> --cumulocity_url <cumulocityURL> --username <user> --password <password> --name <extensionName> --restarte.g.
analytics_builder build extension --input apama-analytics-builder-block-sdk-master\samples\blocks\temp --cumulocity_url https://tenant.c8y.io --username testUser --password testPassword --name Offset --restartNote: --input should be a valid path to a directory containing extension files.
If you are using Cumulocity Edge then username should be specified along with the tenant ID, for example --username <tenantID/user>
This command will build and upload this custom block as an extension to a Cumulocity tenant. These extensions are only read when the apama-ctrl microservice inside Cumulocity is started, so the service must be restarted to make use of this new block (note the --restart in the above command). The web application must also be reloaded (as a simple browser reload) to reload the block catalog after uploading an extension.
Open Analytics Builder by clicking on Analytics Builder in the app switcher and then when opening/creating a model in the editor you can see the newly uploaded block, Offset, under the Calculation category in the left panel.
Figure 6: Shows Apama Analytics Builder pane before uploading block and after uploading Offset block
Figure 7: Offset block being used in the model and its documentation
The above picture depicts our custom block, Offset, being used in the model connecting input and output blocks and you can see its inline ApamaDoc in the right-side documentation pane of the model editor part of Analytics Builder.
For more examples, refer to Apama Analytics Builder Block SDK github repository.
Delete a block from Analytics Builder
Similarly, if you do not want to use any custom block in Analytics Builder, you can delete the specified block by running the below command followed by restarting Apama Controller (--restart in the command sends restart request of Apama Controller).
analytics_builder build extension --delete --name <extensionName> --cumulocity_url <cumulocityURL> --username <user> --password <password> --restarte.g
analytics_builder build extension --delete --name Offset --cumulocity_url https://tenant.c8y.io --username testUser --password Adding parameters to a block
The Offset block adds a fixed value of 100 to the input value. You can also pass this value as a parameter to the block by adding a <BlockName>_$Parameters event with a member for each parameter and add a member to the block of this type named $parameters. Supplied parameters can be validated by implementing $validate action on either the <BlockName>_$Parameters event or the block itself.
/**
* Parameters for Offset block.
*/
event Offset_$Parameters {
/**
* Offset.
*
* The offset as a float.
*
*/
float offset;
action $validate() {
if offset < 0.0 or not offset.isFinite() {
throw Exception("block_msg_apamax.sampleblock.Offset_outOfRange", offset.toString());
}
}
}
/**
* Offset.
*
* Adds offset to the supplied value.
*
* @$blockCategory Calculations
*/
event Offset {
BlockBase $base;
Offset_$Parameters $parameters;
action $process(Activation $activation, float $input_value) {
$setOutput_output($activation, $input_value + $parameters.offset);
}
action <Activation, float> $setOutput_output;
}
ApamaDoc comments in the snippet will be shown in model editor's documentation pane.
Figure 8: Offset block after modification showing parameters and its documentation
Note: Re-uploading the same extension name will replace the old extension. You must not have two extensions with different names but the same block package and type name.
Summary
Block SDK allows us to create, test and deploy custom Analytics Builder Blocks and you have seen creating a simple Offset block and adding parameters to it. You can create even more sophisticated blocks with timers, blocks with state, listeners that listen to or sends to external events, etc. For more details, refer to Block SDK Documentation.
To learn more, visit:
API Reference for Analytics Builder (ApamaDoc)
API Reference for EPL (ApamaDoc)