Managed object lookup via external ID, best practice

,
1

Let assume i have a simple managed object (device):

{
 "id": "123",
 "type": "myDevice",
 "c8y_IsDevice": {}
}

What would be the best aproach to model the managed object and identiy to lookup this device, via a custom UI component, using an external ID and show some of th custom fragments in the UI?

Option 1

The standard way, to lookup managed object is using identity API:

{{url}}/identity/externalIds/{{externalIdType}}/{{externalId}}

Example:
{{url}}/identity/externalIds/my_ExternalId/23

the response contains the managed object internal ID and with another call on inventory API i can fetch the complete managed object.

{{url}}/inventory/managedObjects/123

Option 2

Onother way is to set the external ID as custom fragment directly at the managed object:

{
 "id": "123",
 "type": "myDevice",
 "c8y_IsDevice": {},
 "my_ExternalId": "23"
}

here i could query with only one query the managed object:

{{url}}/inventory/managedObjects?query=\$filter=my_ExternalId+eq+%2723%27

My question is, what will be the better aproach?

  1. Regarding performance by assuming, having millions of devices, how does the inxe look like etc… Memory consumption on Mongo DB…
2

Hi Alex,

one of the main differences is that using the identity API ensures that the used external ID is unique - and not already used by another device.
Hence multiple results are in theory possible in case the same external ID is used on multiple devices.

Regards Kai

3

In my experience a best practice is to use the identity API with a memory cache to retrieve the objects. Especially when you have a high amount of requests per second for same devices it doesn’t make sense to query the identity API all the time. My benchmark also showed that the identity API request slowed down the overall response time. So what I did is using a In-Memory Cache containing all external IDs (as key, value=MO ID) I already have processed while the microservice is up & running.
Downside of this approach. On each restart of the microservice the response time is bad because cache is empty. On deletion of MO or not-found responses you need to clean-up your Cache. The cache itself can be limited to a number of entries e.g. 10.000 ids, not consuming too much memory.

I wouldn’t use the query param with a custom fragment to retrieve the MO. No index is used so you basically doing a whole collection scan and if you have a lot of managed objects, like you assumed, the response time might be really bad.

4

Btw. it would be a good feature request to have this caching of identities handled by the platform :slight_smile:

5

Yes, i have also developed such microservice using a cache that you have explained. Even provided as template: cumulocity-microservice-templates/ehcache at main · SoftwareAG/cumulocity-microservice-templates · GitHub . But i have never really measured the lookup time and i am wondering if Mongo DB is handling this already. I found following statement in Mongo DB forum:

“Does MongoDB handle caching? Yes. MongoDB keeps most recently used data in RAM. If you have created indexes for your queries and your working data set fits in RAM, MongoDB serves all queries from memory.”

Of course, this is really general and i am not sure if this covered my specific use case. I mean we don’t connect directly to Mongo we use the c8y REST API, what happens there, i don’t know (Black Box).

This caching topic something, as you already wrote, to discover with Cumulocity R&D.

6

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.