APAMA Streaming analytics

Hello,

in one of the use-cases, remote site may experience some internet outages that is why location events messages from this remote site may be buffered in a temporary DB before being resent to cumulocity. for the particular use-case, we measure the time between 2 alarms (Ex: prodzone_in and prodzone_out). Those alarms get generated by on the initial location events (Expected to be received with a timestamp). The question is: the duration between the 2 alarms is being counted by cumulocity based on the time elapsed? or somehow based on the time difference between the 2 location updates events timestamps?

Note: we already have async mode active on the corresponding analytics builder model

kind reminder, to clarify more the site gateway will be buffering those packets, and those packets have their original measurement time. once internet connection is back those buffered packets on gateway will reach the positioning engine where positioning is occurring. in between positioning engine and cumulocity platform we have a python middleware and we are selecting the measurement time as the time which will be sent to cumulocity. On cumulocity, we measure using analytics builder the time between 2 location update events (using a duration block in analytics builder) so we need to know if:

  1. optionA: this duration relies on some internal time counter which starts when the first event is received and it stops when the second event is received
  2. OptionB: this duration looks for the corresponding time in each location update packet and compares it to the time in the next location update packet
    to simplifiy it, lets give an example: Location update1 has occured on 2pm and location update2 occured at 2.30pm so practically the difference in time in 30 minutes, but this information was buffered for some time on the gateway because internet was down and when internet was back to normal those 2 packets reached cumulocity with for example 2 seconds delay, it matched an analytics builder model which contains a duration block. Does the duration provide an output of 30 minutes? or 2 seconds?

It depends :slight_smile:

An Analytics Builder model has its own clock that can run with a slight delay to the actual clock. This is the case if “Ignore Timestamps” is set to false. In that case, time in the model will match the time of the received data. In your case, you could use the Duration block and the difference will be the difference between when the two alarms actually happened. Downside is that if the delay is too large (I think by default the limit is 10s) the alarms would get dropped. Also you would process data always with that configured limit as a delay as Analytics Builder has to wait for latecomers.

If “Ignore TImestamps” is true (I think that is what you mean by “async mode active”), the situation is different. Data is received by the Analytics Builder model when it is received in C8Y. Time between the model and the received data is not synchronized. Thus the Duration block will give you the difference between when the two alarms were received. Not when they happened. For your scenario, there should be a relatively simple solution: You can use the “Extract Property” block to extract the “time” property from the alarms. You could then use the “Delta” block to calculate the difference for this property for subsequently received alarms.

It is not easy since we are not using duration for a simple use-case which can be replaced by Delta, In fact we are measuring the duration between prodzonein event and prodzoneout event so prodzonein would initiate the ‘start’ of the duration block and prodzoneout will initiate the ‘measure’ of the duration block and it will also reset the block after it (since we dont want to measure the difference between prodzoneout and the next prodzonein) and once a new prodzonein is received we start again the block etc… so it is a little bit advanced, what do you suggest?

Btw increasing the delay from default 10 seconds is not an option since sometimes messages might be buffered for hours, so we cannot delay the processing of all messages for hours

so in summary if there is a way to avoid dropping the location update if its time is different than cumulocity time but in the same time while still considering the timestamp in the duration calculation thats is exactly what we need, we dont want to completely ignore time stamps, we just want to avoid dropping the location update packet because of some time difference, is it possible somehow?

Hi,

in that case you could still use the solution I described for “Ignore Timestamp” even if you are not ignoring timestamps. This would look something like this:

This assumes that you always have “prodzonein” followed by “prodzoneout”. The Gate block ensures that only the differences between entry and exit and not between exit and entry are being recorded. You can store the gated values in a measurement.

ok, 1 more detail: I can see in the alarm that there is something called “DEVICE TIME” and another time which is when the alarm was generated by the platform (which are different here since the original location update from the device was delayed by some time due to buffering when internet was down in this particular site), we are interested in “DEVICE TIME” , how can we extract it using ‘Extract property’
image

device time is the “time” property I used in the example above. “creationTime” is when it was created in Cumulocity, “lastUpdated” is the last change to the object in Cumulocity.

ok great so all i have to do is to set the ‘extract property’ this way with property path: “time” and that’s it right?
image

You need to switch the type to float:

image

ok and the output of difference would be seconds? or minutes? or hours?

It is in seconds.