The remote-data approach, or demystifying the
As pointed out in chapter 1 and chapter 2 of the
c8y-data-grid tutorial series, the easiest and potentially dirtiest way to set up a grid is to use the
rows attribute. As this doesn’t scale at all if all the data needs to be fetched and held on the client side, we want to use the remote-data approach by setting up a
serverSideDataCallback and generating queries whenever the user triggers actions on the grid.
The secret sauce that will drastically improve the performance is the combination of using queries and pagination. That way we will always show data matching the user’s sort and filter criteria, but drastically reduced to just a small dataset being fetched and shown.
The server-side data “mode” requires a lot more effort to be set up and knowledge in many different areas as you need to understand:
- the data source and as such the API you want to query
- the query parameters and/or query language that this API offers
- the abstraction layer of @c8y/client
o how to setup filter objects for list queries
o how to setup query json which is then parsed to a query string by
- how to create own custom filter view components
o how to persist a query in the column
o how to read and translate the query back to be reflected in the filter UI
In the following example, we are going to display a list of devices (by using the Inventory API). This implies that we target a Cumulocity IoT Tenant with our queries and can use the query language to our advantage.
You can already check out the example of the remote-data-example from the GitHub repository and follow along while reading the code there.
Our tasks are as follows:
- Wire up the
- Create queries based on the grid state
Wire up the
The first task you need to do is to wire up a method that is being bound to the
serverSideDataCallback property of the grid. It is recommended to move the logic of that to its own service so that you can reuse that service for different c8y-data-grids.
Inside of your components html (Example)
Inside of the service you’re wiring up (Example)
By using the
bind-method, we can bind a method call as if it was a property binding. Make sure to provide and inject that service into your component containing the
c8y-data-grid. Now that the
serverSideDataCallback is wired up, the method
onDataSourceModifier is called, whenever the grid needs to reload. This happens whenever:
- the grid loads for the first time
- the user (re-)sets a filter or sortation
- the user clicks the reload button
- the user changes the current page
datasourceModifier parameter will then contain the current state of the grid, such as:
- current state of all columns (whether filter or sortation is set)
- searchText (if
showSearchwas set to
trueon the grid and the user entered text or cleared the search-field)
- pagination information
We will now need to generate queries based on that information. This task will be taken care of by the InventoryDatasourceService on which we call
reload (see line 24 of the previous code-example).
As we intend to always just fetch devices, meaning Managed Objects with the
c8y_IsDevice fragment, we have a base filter that always needs to be applied. We pass the state of the grid and this base query to the
reload method of the
These two services are decoupled by intention, as that way you could also create a
GroupDataSourceService having the same
InventoryDataSourceService as its foundation, just passing a different base-query.
Create queries based on the grid state
Based on the information from the
datasourceModifier parameter, we will now need to create 3 queries:
Usually, just one query would be sufficient, which is the query where all filters etc. are considered and the data of the current page would be queried and displayed. Unless you configure the grid to hide its header (by setting the
gridHeader property of the grid’s
displayOptions to false) and disable pagination (by configuring a load more mode) the grid will show how many items match your current filter/ search and an overall count of items if no filter/ search was applied. This information must be queried too, adding 2 more queries as a result.
We will now have a look at how to implement these 3 queries by starting from the top of the call hierarchy and going down more and more into the details of how this is implemented.
Have a look at the implemented reload method:
You can see these 3 queries being declared on lines 21 to 23:
- actual dataset matching the filters (line 21)
- count of dataset matching the filters (line 22)
- count of the dataset without filters (line 23)
Before sending the queries, we first need to create the query strings (lines 18-19). Inside of the fetch calls, these query strings are added as
query-attribute to the filter object that is passed to the
list-method of the
InventoryService. One query (here
filterQuery in line 18) will contain the actual query information, while the other query just contains the base query (
allQuery in line 19). The results of the queries are then returned combined as
ServerSideDataResult which the grid will then consume to update its view.
Next, let’s have a detailed look at the methods
fetchManagedObjectsCount which are called by the
The fetch methods:
In the method
fetchManagedObjectsForPage we just combine the query string, pagination information (
currentPage which are destructured from paging) to a filter object that we then pass to the
If you want more information about parent-managed objects, you can set
true. It’s important to set
false as the number of pages is calculated via the
fetchManagedObjectsCount calls. Unnecessarily setting
true would drain the performance.
list method then retrieves and returns all items matching the query parameter for the current page, cut into little chunks depending on the size of the
pageSize attribute that was configured in the
fetchManagedObjectCount doesn’t use a count endpoint of the API, but instead also uses the
list method, but with adds a little trick: By setting the
totalPages information of the retrieved response matches the count of items in our dataset. The method returns this count wrapped in a Promise (lines 59-61).
Now that we have covered how the requests are sent, let’s now dive even deeper and check out how the queries for these requests are generated. Remember – the
allQuery strings were the first things we created in the
reload method of the
createQueryFilter generates a query string based on the information it gets from the
-array and the
baseQuery parameter. If you want to extend your query also by the search, you could also add it as a third parameter here (by extracting it from the
datasourceModifier parameter of the
The approach is to create a query JSON object which then the QueriesUtil (which is part of @c8y/client) converts to a string. You can find examples of how such query strings look in the OpenAPI documentation. If you need help with how to build the JSON query object, hover over the
buildQuery method and check out the comprehensive JSDoc guide there.
Make sure to check out the comprehensive JSDoc of the
This query string is then used in the filter parameter that is given to the
list function of the
InventoryService. You could basically also set query strings directly on the
filter object in a static way, but in this context, we need to dynamically create it, depending on the state of the columns and whether or not filters or sortations are set or not.
This dynamic part is done in the
reduce function, which starts with a very basic JSON structure, containing the
__filter attribute with the base query or just an empty object. As previously mentioned – the grid supports sorting multiple columns and this also reflects in the
__orderby attribute being initialized with an empty array.
For every column, we extend the
queryJSON object in case filter or sortation information is available in the column.
In case the user has set a standard string filter on a column in the UI, it would implicitly mean that the
filterable attribute was set to
true for that column and we can expect the
filterPredicate to contain a string with the value that the user entered. We convert that to an equals-query by setting the path as key and the
filterPredicate-content as the value (line 86) of
More complex queries, and especially custom-filter queries, are usually written into the
externalFilterQuery attribute of the column. We expect the custom filter view to already set an appropriate query JSON and just extend
__filter with whatever is stored in the
Next, we need to check whether a column has been sorted or not. As columns are sortable by default, we do not check for
sortable to be true, but instead just check if the
sortOrder got set for that column as it changes whenever the user clicks on the sort button.
sortOrder was set, and thus the column is sorted ascending or descending, we add a key-value pair to the
__orderby array, where the key is the path of the column, and the value is either
1 for ascending or
-1 for descending order (line 94).
That’s it for a basic setup if you want to use the remote-data “mode” of the c8y-data-grid. We have covered how to wire the grid up to a datasource-service, why to use a base-query, and how the
InventoryDataSourceService translates the state of the grid’s columns into list queries going against the Cumulocity IoT tenant.
If you want to see the complete code working in action, please check out the open-source project, run it locally and have a look at the remote-data-example.
The result doesn’t look that different from the local data approach, right? But under the hood, we are now using a way more powerful approach that will guarantee good performance even if large datasets are shown.