How to get started with Cloud Remote Access for Cumulocity IoT

Stefan_Witschel · July 4, 2022, 9:30am

Introduction

Update Jan 2024: I’ve updated some links and added a chapter to tunnel a local HTTP proxy directly to the C8Y UI.

Last year, I published an article roughly describing the Cloud Remote Access Feature of Cumulocity IoT.

Since then I received a lot of requests on how to use it in detail and decided to write another knowledge base article going into detail and covering all of the requests I received. So the following questions should be addressed in this article:

What is the Cloud Remote Access Feature?
Is VPN required for that?
Which agents support it / How can an agent support cloud remote access?
How can I have a Web SSH / VNC Connection my Device?
How can I make use of the Passthrough configuration?
What are the benefits of using the Passthrough configuration instead of Web SSH / VNC?
How can I connect to a device using Passthrough configuration and the local proxy?
Is there a simple way to bridge a local HTTP Server to make it accessible via Cumulocity IoT?

Let’s get started with an overview about the Cloud Remote Access Feature!

Cloud Remote Access

The Cloud Remote Access Feature of Cumulocity IoT allows you to simply connect to your devices using SSH, Telnet, VNC or any other TCP-based protocol. The good thing about that is you don’t need any VPN tunnel by getting the same grade of security a VPN tunnel would provide. This is achieved by having a microservice running in Cumulocity IoT tunneling all protocols through a secure web socket connection and taking care of the authentication.

When using the SSH, Telnet or VNC configuration of the Cloud Remote Access, the tunneled connection is terminated within the Cloud Remote Access Microservice, and the output is visualized in a web terminal in a browser using xTerm.js

Here is an example how this looks like in the browser:

Sometimes it is not sufficient that we are limited to a web terminal and that the Cloud Remote Access Microservice terminates the connection. Therefore we have introduced the Passthrough Configuration.

As you can see from the sketch above in Passthrough configurations, no clients of the microservice are used. Instead it forwards the packets to another web socket channel where a local proxy is running. The local proxy connects to the web socket provided by the microservice, provides TCP sockets where native clients can be connected and of course tunnels the packets receiving from the clients through the web socket and vice versa.

The logic on the device side is always the same.

So let’s sum this up:

If you want to have a web terminal using SSH, telnet or VNC, you can achieve that with the according provided configuration within the Cumulocity IoT platform
If this is not sufficient and you want to use native clients, you can make use of the Passthrough configuration in combination of a local proxy.
In both scenarios, you just need an agent implementing the Cloud Remote Access feature

Cloud Remote Access enabled agents

There are some agents available which are already implementing the Cloud Remote Access feature on device side.

Cloud Remote Access Agent

If you want to try out the Cloud Remote Access Feature and nothing else, the Cloud Remote Access Agent is the one you should go for. The good thing is it also contains a python module, which can by easily integrated into your python agent, enabling the cloud remote access feature only in a few steps. Have a look here for more details! The python package is also available at pypi

thin-edge.io

Also the thin-edge.io implements the cloud remote access feature fully as one of many Device Management Features Cumulocity IoT is providing. So if you are looking for a production grade full working agent you should go for the thin-edge.io

Device Management Reference Agent

The device management reference agent has been explained in detail in one of my last articles: https://tech.forums.softwareag.com/t/getting-started-with-cumulocity-iot-device-management

The cloud remote access module from the agent is part of this agent as it leverages docker containers to quickly create new devices and it comes with a preconfigured SSH & VNC Server. It is one of the fastest options to try out Cloud Remote Access as it provides a docker image on docker hub and a package on pypi.

Cumulocity Linux Agent

The Linux Agent is a lightweight agent implemented in C for linux operated devices. It also has implemented the Cloud Remote Access Feature.

c8yMQTT Raspberry Pi Sense Hat Agent

This agent is very similar to the Device Management Reference Agent. It is implemented in Python and supports the Cloud Remote Access Feature. It is mainly developed for the raspberry pi in combination with a sense hat.

Steps to establish a Web SSH/VNC connection

As the Device Management Reference Agent has everything built in, in this guide I will use this one. Using the other agent requires additional steps like setting up a SSH and VNC Server on the Device.

NOTE: This is only used for demonstration purposes. Of course you can use any agent of the list above or custom agent to follow the steps below but need to set up additional components manually.

Prerequisites

Docker must be installed
Git or any other git client of your choice
A Cumulocity (Trial) tenant with Cloud Remote Access Feature enabled.

NOTE: The Cloud Remote Access Feature is an additional feature and must be assigned to your Tenant. Please ask your contact or administrator to add it to your tenant

1. Start the DM-Agent

This is only needed if you don’t have any agent running already. Please go on with step 2 if you already have an agent running supporting Cloud Remote Access.

First of all we have to have the agent started. Normally this is done by just running ./start.sh or start.bat once you’ve cloned the repo and registering the device using the container id in your tenant. For more details check this guide. Getting Started with Cumulocity IoT Device Management

2. Assign Cloud Remote Access roles to users

To use the Cloud Remote Access functionality, we have to add the existing roles to the users who should be allowed to use this feature. As this feature is pretty powerful (and sometimes destructive), it should be only assigned to experts who exactly know what they are doing.

Go to Administration → Roles → Add global role (on top right).
Give it the name “Cloud Remote Access Role” and make sure to check the “Admin” permission for “Remote Access”.

Next go to Users → select your user and assign the just created role to it. Don’t forget to save your user.

NOTE: If you don’t see the “Remote Access” Tab in Device Management most probably it is because you haven’t assigned the role to your user

3. Configure Web SSH Remote Connection

Now that we have the required permission to use the Cloud Remote Access, we can switch to Device Management Application. Go to your device. You should see now the “Remote Access” Tab. Click on Add Endpoint.

The following details must be provided:

Name of endpoint: Local SSH Server (in my case)
Protocol: SSH
Sign-in method: Username and password
Host: 127.0.0.1 (the SSH Server is running in the same host where the device agent is running)
Port: 22
Username: root (preconfigured in dm-agent docker file, must be changed of course in different environments)
Password: test123# (preconfigured in dm-agent docker file, must be changed of course in different environments)

The host key is optional and will be accepted and stored on first connect.

Please click on Save to store the configuration.

4. Start Web SSH Remote Connection

As we have configured our first SSH connection, we can now test it by clicking on “Connect”

A pop-up will be opened with the web ssh client session. After some seconds, you should see the terminal of your device.

Now you can enter any command of your choice e.g. “df -h” to check available disk space.

NOTE: If you don’t see the terminal but are getting a time-out or other error message please check if the device is online and available. Also check the provided credentials and connection details you’ve provided in step 3.

5. Configure and start the VNC remote connection

Let’s repeat step 3 and 4 for a VNC remote connection.
Click an “Add Endpoint.”

The following details must be provided:

Name: Local VNC
Protocol: VNC
Sign-in method: Password only
Host: 127.0.0.1
Port: 5901 (notice the change to 5901 here)
Password: test456# (preconfigured in dm-agent docker file, must be changed of course in different environments)

Click on “Save.”
Finally click on “Connect” to start the VNC Remote connection. You should see the desktop screen now in a browser window.

NOTE: If you don’t see the desktop and are getting an error, try different ports e.g. 5902. Sometimes the vnc server increases the port automatically especially when the agent is restarted multiple times.

Steps to setup a passthrough connection

As described above, the passthrough is the most advanced way to use cloud remote access. You might wonder why there is no “Passthrough” option in the drop-down box when creating a new Remote Access Endpoint. This is a systemwide option that is disabled by default on all our public instances (cumulocity.com / eu-latest.cumulocity.com and so on). On dedicated instances, this option can be set to enabled, and you will see an option in the drop-down box when creating a new endpoint. Still you can try the passthrough option on public instances. Follow the steps below to do so.

NOTE: This guide only helps to demonstrate the passthrough capabilities. If you want to use this in production and on a mass scale, please get in contact with your responsible Software AG contact or Software AG support to make sure not running to any issues.

1. Create a passthrough endpoint

First of all, we need to create a passthrough endpoint. The easiest way would be using the UI. So if you can see the passthrough option in the UI, use that to create an endpoint. You just need to provide a name, a URL and a port. No credentials are needed as they are provided by the client.
If you don’t see the passthrough option in the selection box, you can create an endpoint using the REST API.
Use Postman or a similar REST Client to execute the following call:

POST <host>/service/remoteaccess/devices/<InternalDeviceID>/configurations

Headers: 
Basic Auth: <base64 encoded (yourC8YUser : yourPassword)>
Content-Type: application/json
Accept: application/json


Payload:
{
    "name": "Passthrough",
    "hostname": "<hostname of local server>",
    "port": <port of local server>,
    "protocol": "PASSTHROUGH",
    "credentialsType": "NONE"
}

Example to connect to the local SSH Server:

POST https://switschel.eu-latest.cumulocity.com/service/remoteaccess/devices/97282922/configurations

Headers:
<see above>

Payload:
{
    "name": "Passthrough",
    "hostname": "127.0.0.1",
    "port": 22,
    "protocol": "PASSTHROUGH",
    "credentialsType": "NONE"
}

As response we get back:

{
    "attrs": {},
    "id": "2",
    "name": "Passthrough",
    "hostname": "127.0.0.1",
    "port": 22,
    "protocol": "PASSTHROUGH",
    "credentials": {
        "type": "NONE"
    }
}

Also in the UI, when we refresh the Cloud Remote Access view on our Device, we should see the endpoint now:

2. Install local proxy

As we’ve seen in the introduction section for passthrough, we need an agent supporting remote access; and a local proxy running on the client side or on a server, we can connect to with our local clients. A local proxy is provided as an open-source project:

So if needed, you can clone / fork it and adapt it to your needs. For now we focus on getting it running on our local computer / computer where we want to run the local proxy. We make sure that python3 is installed e.g. python --version or python3 --version. Also make sure that pip is installed. If not, install it to your computer.

As a next step, we install the local proxy via:

pip install c8ylp

3. Connecting to device

Afterwards we can run the local proxy. The local proxy supports multiple modes. For us, two modes are of interest:

Server Mode - which opens a local server listening on a port a a local client can connect
connect ssh Mode - directly use local installed ssh client to establish a ssh session

First of all we need to login to our tenant:

swit@SAG-34TVR73:~$ c8ylp login --env-file mytenant.env
Loading env-file: mytenant.env
Host: switschel.eu-latest.cumulocity.com
Enter your Cumulocity Username: stefan.witschel@softwareag.com
Enter your Cumulocity Password [input hidden]:
Env file was updated: mytenant.env

The c8ylp login command will create an env file which stores the required information, so we don’t have to enter them on any other request again.

3.1 Using connect ssh mode

Next we can already establish a ssh connection. Make sure that you’ve open ssh locall installed on your computer by entering “ssh” in your terminal.

Now we can run the connect mode of the local proxy:

c8ylp connect ssh <external ID of device> --ssh-user root --env-file mytenant.env

Example:

c8ylp connect ssh cc47511bb809 --ssh-user root --env-file mytenant.env

It will prompt you for the password for the user root and directly connects you to the device.

3.2 Using server mode

As an alternative, we can use the server mode, so we don’t have the requirement to have ssh installed but can use any other client like putty to connect to our local proxy instance.

c8ylp server cc47511bb809 --env-file mytenant.env

This command opens a random port (39941 in my example) which we can use to connect with a client of our choice:

In my example I use putty:

Connection is established successfully:

4. Further use cases for using Passthrough

There are much more use cases for using the Passthrough endpoint but not only SSH.

For example you can bridge a local accessible Web UI using the Passthrough configuration. To do so, you just need to configure the internal URL and port in the Passthrough config. If you now connect the local proxy in server mode you can enter localhost and the generated port in your browser to access the local User Interface.

Basically any TCP-Port can be bridged that way, not only SSH, VNC or HTTP, see next chapter for an example bridging HTTP Server to the cloud.

Setup a cloud http proxy

Using the Passthrough configuration you can also easily tunnel a local HTTP server which is only accessible for the device. Here this open-source repo provides a microservice, who takes care about the connection handling. and a UI Plugin that let you visualize any User Interface of that local HTTP Server directly on the device in Device Management.

See this example where a local node-red server is tunneled and visualized on a C8Y device directly from the cloud.

Summary

In this guide, I demonstrated almost all possibilities you have when using the Cloud Remote Access Feature including SSH, VNC and Passthrough endpoints on a step-by-step approach and using the DM reference agent. It should help to understand the basic concepts and demonstrate on an example of how to perform the steps using Cumulocity IoT.

If you have any questions or feedback feel free to approach me! In case of issues with used components like dm-agent or local proxy, feel free to open an issue directly in the Github projects.

This article is part of the TECHniques newsletter blog - technical tips and tricks for the Software AG community. Subscribe to receive our quarterly updates or read the latest issue.