Error while device to Cumulocity IoT using thin-edge.io

Product/components used and version/fix level:

Cumulocity IoT > Device Management > Management > Trusted Certificates
BACKEND: 1018.0.229
UI: 1018.0.151

Detailed explanation of the problem:

My user has global roles “admins” and “tenant manager”.
When I tried to connect the device using thin-edge.io getting an error,
when I tried the following command
sudo tedge connect c8y
I already uploaded the certificate in Cumulocity tenant

Output of tedge config list --all command

device.id=my-device_123
device.key_path=/etc/tedge/device-certs/tedge-private-key.pem
device.cert_path=/etc/tedge/device-certs/tedge-certificate.pem
device.type=thin-edge.io
c8y.url=**iot.eu-latest.cumulocity.com
c8y.root_cert_path=Zscaler.crt
c8y.smartrest.templates=
c8y.http=**iot.eu-latest.cumulocity.com
c8y.mqtt=**iot.eu-latest.cumulocity.com
c8y.topics=[“te/+/+/+/+”, “te/+/+/+/+/twin/+”, “te/+/+/+/+/m/+”, “te/+/+/+/+/e/+”, “te/+/+/+/+/a/+”, “te/+/+/+/+/status/health”]
c8y.enable.log_upload=true
c8y.enable.config_snapshot=true
c8y.enable.config_update=true
c8y.proxy.bind.address=127.0.0.1
c8y.proxy.bind.port=8001
c8y.bridge.include.local_cleansession=auto
az.root_cert_path=/etc/ssl/certs
az.mapper.timestamp=true
az.topics=[“te/+/+/+/+/m/+”, “te/+/+/+/+/e/+”, “te/+/+/+/+/a/+”, “te/+/+/+/+/status/health”]
aws.root_cert_path=/etc/ssl/certs
aws.mapper.timestamp=true
aws.topics=[“te/+/+/+/+/m/+”, “te/+/+/+/+/e/+”, “te/+/+/+/+/a/+”, “te/+/+/+/+/status/health”]
mqtt.topic_root=te
mqtt.device_topic_id=device/main//
mqtt.bind.address=127.0.0.1
mqtt.bind.port=1883
mqtt.client.host=localhost
mqtt.client.port=1883
http.bind.port=8000
http.bind.address=127.0.0.1
software.plugin.max_packages=1000
run.path=/run
run.lock_files=true
logs.path=/var/log
tmp.path=/tmp
data.path=/var/tedge
firmware.child.update.timeout=3600
service.type=service

Error messages / full error message screenshot / log file:

The system config file ‘/etc/tedge/system.toml’ doesn’t exist. Use ‘/bin/systemctl’ as a service manager.

Detected mosquitto version < 2.0.0
Checking if systemd is available.

Checking if configuration for requested bridge already exists.

Validating the bridge certificates.

Creating the device in Cumulocity cloud.

ERROR: Tls(Io(Os { code: 104, kind: ConnectionReset, message: “Connection reset by peer” }))
Error: failed to connect Cumulocity cloud.

any suggestions about it ?

It suspect that there is a connectivity problem on the device.

How do you upload the device’s certificate? using tedge cert upload c8y? Or did you do this manually via the Cumulocity IoT UI?

Also can you check if the device has connectivity to Cumulocity IoT by using the following curl command?

curl "https://$(tedge config get c8y.url)/tenant/loginOptions"

Actually I think I found the error…you are using a relative path for the c8y.root_cert_path setting. Currently you are setting it to:

c8y.root_cert_path=Zscaler.crt

This should be something like:

sudo tedge config set c8y.root_cert_path '/mypath/to/the/file/Zscaler.crt'

I get the following output when I ran the command
curl “https://$(tedge config get c8y.url)/tenant/loginOptions”

{"self":"https://t1415107691.eu-latest.cumulocity.com/tenant/loginOptions/","loginOptions":[{"initRequest":"https://t1415107691.eu-latest.cumulocity.com/tenant/oauth?tenant_id=t1415107691","userManagementSource":"INTERNAL","visibleOnLoginPage":true,"type":"OAUTH2_INTERNAL","sessionConfiguration":{"absoluteTimeoutMillis":1209600000,"renewalTimeoutMillis":86400000,"userAgentValidationRequired":false,"maximumNumberOfParallelSessions":5},"self":"https://t1415107691.eu-latest.cumulocity.com/tenant/loginOptions/afe30a8b-e030-4d43-95ef-5b2253a2f033","id":"afe30a8b-e030-4d43-95ef-5b2253a2f033","grantType":"PASSWORD","tfaStrategy":"SMS","greenMinLength":null,"loginRedirectDomain":"persistentiot.eu-latest.cumulocity.com","enforceStrength":true,"_type":"OAuth2Config","strengthValidity":false},{"userManagementSource":"INTERNAL","visibleOnLoginPage":false,"type":"BASIC","authenticationRestrictions":{"trustedUserAgents":[],"forbiddenClients":["WEB_BROWSERS"],"forbiddenUserAgents":[]},"self":"https://t1415107691.eu-latest.cumulocity.com/tenant/loginOptions/ff596dd9-190a-42a3-b98f-1eff7d9ec2ca","id":"ff596dd9-190a-42a3-b98f-1eff7d9ec2ca","tfaStrategy":"SMS","greenMinLength":null,"enforceStrength":true,"_type":"BasicAuthConfig","strengthValidity":false}]}[

I have changed the path as well still getting the same error

I have installed
thin edge version: 0.13
mosquitto version: 1.6.10

Can you print out the tedge configuration again (after you updated the path to the c8y.root_cert_path)?

And also did you upload the device certificate manually to Cumulocity or using a cli command?

I just saw your other post (Error while running opcua gateway locally) where you mention that you are behind a corporate proxy…is it possible that this device is also behind the proxy? If so I suspect that the proxy is handling http/s traffic ok, but not mqtt, which is the cause of the connection error.

Yeah this device is also behind corporate proxy Zscaler

Can you check if the Zscaler is configured to allow traffic via the 8883 port (used by the MQTT mosquitto bridge to connect to Cumulocity IoT)? I suspect the proxy is closing down the connection.

To provide some more connection info, can you also run the following command on the device to see what certificates are being presented to a client trying to connect to the mqtt address/port of Cumulocity IoT:

openssl s_client -connect $(tedge config get c8y.url):8883

It is giving the following output when I ran the above command:

CONNECTED(00000003)
write:errno=104
no peer certificate available
No client certificate CA names sent
SSL handshake has read 0 bytes and written 344 bytes
Verification: OK
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)

Ok this looks like that the Zscaler proxy is blocking communication when connecting to an outgoing port 8883. (I can tell that because the response does not include any of the Cumulocity SSL certificates which should be returned if the proxy was actually proxying requests)

You will have to request the Zscaler to allow TCP traffic on the outgoing port 8883 so that the MQTT connection can function.

1 Like

Thanks @Reuben_Miller2 for helping me through this. I will request Zscaler to allow traffic through port 8883

Hi @Reuben_Miller2 I was able to fix the allow traffic through the port 8883.I am able to create a device in Cumulocity, but the measurements are not getting created when I tried
using the cli.
tedge mqtt pub ‘c8y/s/us’ ‘211,20’

The measurement isn’t being created because you are sending an invalid mqtt message. However you trying to send a measurement directly to the Cumulocity bridge and not via thin-edge.io…and I’m a bit surprised where you even got that command from…

Anyway, I would recommend you using the thin-edge.io topics instead as this is more generic and let’s the cloud specific mappers worry about the cloud specific messages. Best place to get started is to use the “Getting Started” section of the thin-edge.io docs:

For more details you can also read more details about the thin-edge.io MQTT API here:
MQTT API | Thin-edge

But an example to publish a measurement for the main device using the thin-edge.io MQTT API would be:

tedge mqtt pub 'te/device/main///m/environment' '{
  "temperature": 23.4
}'

Hi @Reuben_Miller2, I am getting a follwing error when I ran the following command

sudo tedge connect c8y

output of the command:

The system config file ‘/etc/tedge/system.toml’ doesn’t exist. Use ‘/bin/systemctl’ as a service manager.
Detected mosquitto version < 2.0.0
Checking if systemd is available.
Checking if configuration for requested bridge already exists.
Validating the bridge certificates.
Creating the device in Cumulocity cloud.
Saving configuration for requested bridge.
Restarting mosquitto service.
Awaiting mosquitto to start. This may take up to 5 seconds.
Enabling mosquitto service on reboots.
Successfully created bridge connection!
Sending packets to check connection. This may take up to 2 seconds.
ERROR: Local MQTT publish has timed out.
Warning: Bridge has been configured, but Cumulocity connection check failed.
Checking if tedge-mapper is installed.
Starting tedge-mapper-c8y service.
Persisting tedge-mapper-c8y on reboot.
tedge-mapper-c8y service successfully started and enabled!
Local MQTT publish has timed out.
The request has been sent, however, no response.
Failed to get the connected tenant URL from Cumulocity.
Enabling software management.
Checking if tedge-agent is installed.
Starting tedge-agent service.
Persisting tedge-agent on reboot.
tedge-agent service successfully started and enabled!

I think due to this error, the measurements are not getting created

any idea what error it might be and any solutions for it

Can you provide the logs from mosquitto?

You can enable mosquitto logs via standard mosquitto functionality (e.g. mosquitto.conf man page | Eclipse Mosquitto)

But you can enabled details mosquitto logs using the following:

  1. Edit the /etc/mosquitto/mosquitto.conf file and add the settings to the top of the file (leave the other config):

    log_dest file /var/log/mosquitto/mosquitto.log
    log_type all
    
  2. Restart mosquitto

    sudo systemctl restart mosquitto
    
  3. Try connecting again using tedge (to see more entries in the mosquitto log) and enable more detailed logging in the command:

    sudo RUST_LOG=trace tedge connect c8y
    
  4. Leave mosquitto running for about 30 seconds (to gather more logs)

  5. Collect the logs from /var/log/mosquitto/mosquitto.log and paste here (use a code block when posting the info)

  6. Also provide the Cumulocity IoT bridge configuration file if it exists (use a code block when posting the info)

    cat /etc/tedge/mosquitto-conf/c8y-bridge.conf