The stream analytics model unable to process the bulk of events that are ingested cloud in no real time

What product/components do you use and which version/fix level are you on?

Latest

Is your question related to the free trial, or to a production (customer) instance?

Production

What are you trying to achieve? Please describe it in detail.

The events counter model has not processed bulk input events.
Below is the screenshot of the model.
The configuration of event input block is the “New events” and the ignore timestamp.


The IoT device has sent bulk of events to the stream analytics model. The time difference between the real time events happend and ingestion time is 3 hours.

Do you get any error messages? Please provide a full error message screenshot and log file.

Have you installed all the latest fixes for the products and systems you are using?

yes

Can you attach an export of the model?

when you say the device has sent bulk events - how exactly is it doing this?

The iot edge gateway saves the events on the device when there is no internet connection. The unpublished events are sent to the cloud one by one through thinedge mqtt topic when the internet connection comes back.

Test-Count.docx (19.3 KB)

I tested the model you sent, with events sent with MQTT and found no problems, the count was continually updated.

The events are generated at 15:19:49 but uploaded at 18:21:30 to the cloud through thinedge.io.
The above stream analytics model is unable to take this input for computation.

Does the stream analytics model cannot take the late updated events for caluclation?

Can you reproduce the problem, then provide an enhanced diagnostics zip (so it will contain information relevant to the error/anomaly if any)? There’s a link called ‘Enhanced’ at the bottom of the screen that will do the download for you.

From your initial description, it sounded like “Ignore Timestamp” was enabled. I loaded you model and it is actually disabled. Can you please re-test with “Ignore Timestamp” enabled?

Without it, Analytics Builder will discard events that are older than a couple of seconds:
https://documentation.softwareag.com/apama/Analytics_Builder/pab10-15-0/webhelp/pab-webhelp/index.html#page/pab-webhelp%2Fco-AnaBui_common_block_inputs_and_parameters.html%23wwID0EUSBO

What is difference between the Notification Modes New event, Updated only and All events in the input Event block?

It’s configuring when an event will actually be received by the input block (and sent into the model).

In the Cumulocity IoT data model, an event can be modified in place, not just created. If an event is modified in place, then ‘Updates Only’ or ‘All’ will send that event with its latest state into the model each time it’s modified. With ‘New events only’ or ‘All’ you will also get newly created events going into the model.

What does creation time and last updated time here means?


What time the event input block considers when ignore time stamp is used?

creationTime is the time when the object has been initially created in the database.
`lastUpdated? is the time when the object has been last updated.

For events the time is used to reorder the events. When selecting ignore Timestamp the events are not re-ordered and will be processed as occurred in the platform.

In your case when not checked: an event has been processed which has a newer timestamp time so any other event with an older timestamp, even occured later, will be ignored. When checked you get this event, no matter if the time is in the past or future.

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.