How to migrate data from external database to Cumulocity

,
1

What product/components do you use and which version/fix level are you on?

Free Trail

Is your question related to the free trial, or to a production (customer) instance?

Production Instance

What are you trying to achieve? Please describe it in detail.

How we can migrate data (user data,device data,telematery data) from a database which is deployed in another cloud platform to cumulocity platform?

Can we create custom device schema in cumulocity operational store?

Do you get any error messages? Please provide a full error message screenshot and log file.

Have you installed all the latest fixes for the products and systems you are using?

2

Hi Punam,

you can add custom schema (we call it fragments) to the cumulocity IoT domain model but to effectively use the platform you need to map it to the pre-defined structure / domain model.

I fear you need to implement some logic the fetch that data from your database, map it into Cumulocity IoT domain model and send it to Cumulocity IoT using the API.
I would prefer to either implement in a microservice or in a script running externally.

3

Hi Stefan,

We need to check feasibility of a requirement where we need to migrate an application from one cloud platform to cumulocity.

The application has UI layers, Backend and Database layer. We want to migrate it to cumulocity.

UI and backend of the application can be deployed in cumulocity as Web application and microservice respectively.

we want to migrate the database layer as well to cumulocity. Provided database schema remains same so that we need to do minimal changes in backend application in the beginning.

Since, cumulocity doesn’t provide database access so export and import of database schema is not possible. Because here we can interact to database using REST API’s.

Can you please tell me how we can add custom database schema to cumulocity IoT domain model?

4

Hi @punam90singh,

I don’t think it makes much sense just migrating an existing IoT solution with minimal effort to Cumulocity IoT.
Cumulocity IoT is not an IoT Platform like others where you just host stuff and use some specific services like databases etc.

You definitely need to adapt the backend services to microservices which leverages the functionality of Cumulocity IoT. This means also adapting the existing schema to the Cumulocity IoT Domain model. The schema cannot remain same, it needs to be adapted. You should also consider using stuff like streaming analytics or device management functionality.

For the UI it can be re-used when the API of the microservices remains stable, but still it is not using any features of Cumulocity IoT. Also here it would make sense to adapt the UI using services of Cumulocity IoT like dashboarding, real-time visualization, data grids etc. Also maybe use default components so you don’t need to implement & maintain all of your own.

Again: Moving one IoT solution to Cumulocity IoT as is does not make much sense as Cumulocity IoT is not an application hosting platform only. Real benefit comes when you use the tons of available services of the platform so you just implement your custom logic on top. This would be a valuable migration in the end.

5

Hi Poonam, Migrating data from an external database to Cumulocity can be done using various methods. Here are some steps to help you get started:

1. Identify the source database: The first step is to identify the database that you want to migrate data from. This could be any database that holds data that you want to move into Cumulocity.
2. Choose a migration method: There are various ways to migrate data from a database to Cumulocity, such as using Cumulocity’s REST APIs, Cumulocity’s microservice architecture, or using third-party migration tools.
3. Map data fields: Once you have identified the database and chosen a migration method, you will need to map the data fields from the source database to the Cumulocity data model. This ensures that the data is correctly aligned and transferred to Cumulocity.
4. Transform the data: Data in the source database might not always match the Cumulocity data model, and transformation may be needed to ensure that data is correctly transferred. You may need to write scripts or use third-party migration tools to transform the data before migrating it.
5. Test the migration: Before migrating large amounts of data, it’s important to test the migration process to ensure that it works as expected. You can start by migrating a small set of data and verifying that it appears correctly in Cumulocity.
6. Execute the migration: Once you have tested the migration and verified that it works correctly, you can start migrating the rest of the data.

It’s important to note that the migration process can be complex, and you may need to engage with Cumulocity support or third-party migration experts to help you migrate data. Additionally, you may need to adjust the migration process depending on the complexity of the data and the size of the migration.

6

Moreover, if you want to avoid to insert years of history into the Cumulocity Platform, you could also store the older data from your database to a data lake. Cumulocity IoT supports offloading data to a data lake with the product Cumulocity IoT DataHub, and we have customers who use this successfully to keep track of older data from both the Cumulocity platform and systems they used before.

1 Like
7

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.