The Cumulocity IoT platform has transitioned to using Apama Streaming Analytics for event processing and real-time analytics. Historically, the Cumulocity Event Language (CEL), based on Esper, was provided in the Cumulocity IoT platform to execute streaming analytics. Apama Streaming Analytics and the new architecture bring many additional capabilities including:
- Analytics Builder – a drag and drop, web-based model building environment for domain experts
- Web-based code editing – a new web-based editor for Apama EPL code with syntax highlighting, error reporting and a library of sample apps
- Advanced desktop tooling for developers – tools such as inline debugging, data replay, code coverage analysis and memory/CPU profiling
- Improved performance and predictable scalability – per-tenant streaming analytics ensures that demanding tenants do not negatively impact the performance of streaming analytics on other tenants
Software AG terminated support for using CEL (Esper) in Cumulocity IoT on 31 Dec 2020
Streaming analytics in Cumulocity IoT using CEL (Esper) was deprecated in 2018 and this message is included in the online documentation.
“Important: Support for streaming analytics using CEL (Esper) has ended. All new Cumulocity IoT subscriptions use the Apama CEP engine. Software AG terminated support for using CEL (Esper) in Cumulocity IoT on 31 Dec 2020 following its deprecation in 2018.
For documentation on using the unsupported CEL functionality based on Esper, refer to the CEL analytics guide .
For details on migration, see Migrating from CEL (Esper) to Apama in the Streaming Analytics guide.”
Customers still using CEL are expected to migrate to Apama. Software AG will work with customers to provide guidance and make this migration as straightforward as possible.
Cumulocity IoT tenants should use Apama for running smart rules and custom rules after the end of 2020
Each tenant in Cumulocity IoT which is using smart rules or custom rules will require an instance of Apama.
Apama Starter was made available in Oct 2019 to allow customers to execute an unlimited number of smart rules. This is available as a free of charge microservice for all tenants (except enterprise sub-tenants hosted by Software AG). Refer to the Apama Starter section in this document.
Rationale for migration to Apama in Cumulocity IoT
The rationale for the migration from Esper to Apama and the subsequent End-of-Support notice for Esper is summarized by 2 points:
- Apama is better for Cumulocity IoT customers:
- Apama handles more data, is faster and has more capabilities
- Analytics Builder drag & drop, web-based, model building environment for non-coders
- Web-based code editing with sample app library
- Advanced desktop tooling for developers
- Apama is a Software AG product:
- Other solutions are separately licensed by Software AG for Cumulocity IoT
- Cumulocity IoT requirements not key on for a licensed 3rd party product
- Cumulocity IoT R&D and Apama R&D work together and have aligned strategy and roadmap
- Analytics Builder not possible without Apama
Streaming Analytics capabilities in Cumulocity IoT
Esper was used for smart rules and for simple custom code applications (using CEL). Development of code applications was performed in the web browser using a simple text editor.
Apama can be used for smart rules, Analytics Builder models, simple custom code applications, full applications running in their own custom microservice and full applications running externally and integrating with Cumulocity IoT. Code applications can be developed either in the web-browser or using Apama’s comprehensive developer tools running on the desktop including a full IDE with debugger and tools such as data replay, code coverage analysis and memory/CPU profiling.
Apama allows use cases to evolve from smart rules, through drag & drop Analytics Builder models, to simple and advanced applications running inside or outside of Cumulocity IoT.
Hosting - shared instances vs per-tenant instances
Esper was made available primarily as a shared instance, meaning that multiple Cumulocity IoT tenants could execute streaming analytics using a single shared instance.
- The main advantage of running in a shared instance is reduced hosting costs - since many tenants can execute streaming analytics in a single hosted instance.
- The main disadvantage of running in a shared instance is the lack of guaranteed performance - since an individual tenant can monopolize the processing of streaming analytics algorithms at the expense of other tenants.
Esper was also made available as per-tenant instances, primarily to help try and guarantee a satisfactory level of performance.
To help deliver predictable performance for tenants, Apama is only available in per-tenant instances. It is not available as a shared instance.
Cumulocity IoT permits only one streaming analytics engine per tenant
Cumulocity IoT only allows a tenant to have one microservice that subscribes to its streaming data interface. It is not possible for a tenant to be running both Apama and Esper.
Therefore, a tenant can only be configured for Apama Starter, Apama or Esper.
Cumulocity IoT permits only one streaming analytics engine per tenant and therefore piecemeal migration of a single tenant from Esper to Apama is not possible.
When migrating from Esper to Apama, customers should be aware of this limitation as it prevents a piecemeal transition. Instead it is likely that all streaming analytics apps must be written and tested in Apama before the streaming analytics engine can be switched from Esper to Apama.
Migrating from CEL (Esper) to Apama
If assistance is required getting started with Apama EPL code then the Apama guide available on the Cumulocity IoT site is a good starting point. Refer to the migration topic in the Analytics guide.
Customers migrating from CEL (Esper) to Apama in Cumulocity IoT can migrate by following these guidelines.
- Lock down the CEP custom rules on the existing tenant to prevent change.
- Make available a new tenant on which Apama has been enabled.
- Manually convert all old custom rules from the existing tenant into equivalent Apama EPL apps on the new tenant. Refer to the rest of this guide, in particular Best practices and guidelines. This includes smart rules where the CEL has been modified.
- Test the behavior of the new EPL apps by sending, for example, measurements or events into the new tenant and verifying that the new EPL apps respond appropriately.
- When all new EPL apps have been developed and tested, move your production tenant from CEP to Apama, that is: subscribe the new tenant to the Apama-ctrl microservice (and unsubscribe it from CEP).
- Any unmodified smart rules will migrate automatically.
- Delete any smart rules where the CEL version had been modified and a new EPL app has been implemented.
- Activate your newly developed EPL apps in the production tenant.
Customers can also choose to work with Software AG Professional Services to help ensure the migration is as smooth as possible. Software AG Professional Services can provide training on using Apama in Cumulocity IoT and can even help migrate CEL code into Apama code.