kubectl
installed (Install Tools).kubectl
configured with the current context pointing to your target cluster (Configure Access to Multiple Clusters).Add the Cybus connectware-helm
repository to your local Helm installation to use the connectware-agent
Helm chart to install Connectware agents in Kubernetes:
helm repo add cybus https://repository.cybus.io/repository/connectware-helm
Code-Sprache: YAML (yaml)
To verify that the Helm chart is available you can execute a Helm search:
helm search repo connectware-agent
Code-Sprache: YAML (yaml)
NAME | CHART VERSION | APP VERSION | DESCRIPTION |
cybus/connectware-agent standalone agents | 1.0.0 | 1.1.5 | Cybus Connectware |
As with all Helm charts, the connectware-agent
chart is configured using a YAML file. This file can have any name, however we will refer to it as the values.yaml file.
Create this file to start configuring your agent installation by using your preferred editor:
vi values.yaml
Code-Sprache: YAML (yaml)
To quickly install a single agent you only need to add your Connectware license key to your values.yaml file as the Helm value licenseKey
:
licenseKey: <your-connectware-license-key>
Code-Sprache: YAML (yaml)
You can now use this file to deploy your Connectware agent in a Kubernetes namespace of your choice:
helm upgrade -i connectware-agent cybus/connectware-agent -f values.yaml -n <namespace>
Code-Sprache: YAML (yaml)
Output
Release "connectware-agent" does not exist. Installing it now.
NAME: connectware-agent
LAST DEPLOYED: Mon Mar 13 14:31:39 2023
NAMESPACE: connectware
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
Thank you for using the Cybus Connectware agent Helm chart!
For additional information visit https://cybus.io/
Number of agents: 1
--------------------
- agent
If any of these agents are new, please remember to visit Connectware's client registry to set up the connection to Connectware.
Hint: If you have agents stuck in a status other than "Running", you need to delete the stuck pods before a pod with your new configuration will be created.
Code-Sprache: YAML (yaml)
This will start a single Connectware agent named “agent”, which will connect to a Connectware installation deployed in the same namespace. Unlock the Client Registry in your Connectware admin UI to connect this agent. Refer to Client Registry — Connectware documentation to learn how to use the Client Registry to connect agents.
You can repeat the same command to apply any changes to your values.yaml file configuration in the future.
If you are not deploying the agent in the same Kubernetes namespace, or even inside the same Kubernetes cluster, you need to specify the hostname under which Connectware is reachable for this agent.
In the default configuration, the following network ports on Connectware must be reachable for the agent:
Specify the hostname of Connectware to which the agent connects to by setting the Helm value connectwareHost
inside the protocolMapperAgentDefaults
context of your values.yaml file. For Connectware deployments in a different Kubernetes namespace this is “connectware.<namespace>”.
Example
licenseKey: <your-connectware-license-key>
protocolMapperAgentDefaults:
connectwareHost: connectware.cybus # adjust to actual hostname of Connectware
Code-Sprache: YAML (yaml)
To connect to a Connectware that uses the separate control-plane-broker, you need to set the Helm value controlPlaneBrokerEnabled
to true
inside the protocolMapperAgentDefaults
section of your values.yaml file.
Example
licenseKey: <your-connectware-license-key>
protocolMapperAgentDefaults:
connectwareHost: connectware.cybus # adjust to actual hostname of Connectware
controlPlaneBrokerEnabled: true
Code-Sprache: YAML (yaml)
Note: This adds TCP/1884 to required network ports.
You can use the agent chart to install multiple Connectware agents. Every agent you configure needs to be named using the Helm value name
in a collection entry inside the context protocolMapperAgents
. This way, the default name “agent” will be replaced by the name you give the agent.
Example
licenseKey: <your-connectware-license-key>
protocolMapperAgentDefaults:
connectwareHost: connectware.cybus # adjust to actual hostname of Connectware
protocolMapperAgents:
- name: bender-robots
- name: welder-robots
Code-Sprache: YAML (yaml)
This will deploy two Connectware agents, named “bender-robots” and “welder-robots”, both of which will contact the Client Registry of Connectware inside the Kubernetes namespace “cybus”, as described in Client Registry — Connectware documentation
This article describes how to integrate your Azure IoT Hub with Connectware. It will help you configure the necessary Connectware service commissioning file and provide examples of mapping data from Connectware to Azure IoT Hub and vice versa. In addition, the article links to helpful tools to help you work with Azure IoT Hub and implement your use cases faster.
As described in the official Azure documentation, Azure IoT Hub supports communication via MQTT. There are two ways to communicate with the IoT Hub device endpoints:
In this article, we will focus on connecting via TCP port 8883, which is the standard secure communication port for MQTT and is the preferred integration of Azure IoT Hub with Connectware.
To access Azure IoT Hub, we need some connection properties that we add as definitions (i.e. constant values) to our commissioning file.
For now, do not worry about copying the commissioning file snippets together into one, we provide you with a link to the complete example file at the end.
definitions:
iotHubHostname: <full CName of your Azure IoT Hub>
mqttPort: 8883
deviceId: <Your deviceID>
sasToken: <Your generated SAS Token>
Code-Sprache: YAML (yaml)
To connect to Azure IoT Hub, we set up a Cybus::Connection resource in the resources section. The connection uses the general MQTT connector from Connectware. For an overview of the connection properties, refer to MQTT (Cybus documentation).
With the !ref
tag we reference the definitions from our previous step. The username is a string composed of the iotHubHostname
and the deviceId
, to concatenate strings we need the !sub
tag. With this tag in place, we can include the definitions within the string by enclosing them in curly brackets and a preceding $
.
resources:
mqttConnection:
type: Cybus::Connection
properties:
protocol: Mqtt
connection:
host: !ref iotHubHostname
port: !ref mqttPort
username: !sub "${iotHubHostname}/${deviceId}/?api-version=2021-04-12"
password: !ref sasToken
clientId: !ref deviceId
scheme: tls
keepalive: 3600
Code-Sprache: YAML (yaml)
This is all we need to establish the initial connection to Azure IoT Hub. Now let’s define our read endpoint.
If the Connectware host system does not have access to root CAs, you may need to add the Azure root certificate to your configuration using the caCert property. For more information on Azure root certificates, refer to the Azure documentation.
You can connect to a specific endpoint on Azure IoT Hub. To write data from Connectware to Azure IoT Hub, the topic is defined by Azure IoT Hub. For more information on this, refer to the Azure documentation.
# Device to Cloud
d2cEndpoint:
type: Cybus::Endpoint
properties:
protocol: Mqtt
connection: !ref mqttConnection
topic: d2cEndpoint
qos: 0
write:
topic: !sub "devices/${deviceId}/messages/events/"
Code-Sprache: YAML (yaml)
To read data from Azure IoT Hub, we need another endpoint. In this case, we subscribe to a wildcard topic to receive all data for the device ID we are connected to. Note that this topic is already defined by Azure IoT Hub.
# Cloud to Device
c2dEndpoint:
type: Cybus::Endpoint
properties:
protocol: Mqtt
connection: !ref mqttConnection
topic: c2dEndpoint
subscribe:
topic: !sub "devices/${deviceId}/messages/devicebound/#"
Code-Sprache: YAML (yaml)
Here are two example mappings that route topics from Connectware to the endpoints we configured before. Replace the topic Azure/IoTHub/Write
with the topic where you want to publish the data to be sent to Azure IoT Hub. In the second mapping, replace Azure/IoTHub/Read
with the topic where you want to access the data that comes from Azure IoT Hub.
deviceToCloudMapping:
type: Cybus::Mapping
properties:
mappings:
- subscribe:
topic: Azure/IoTHub/Write
publish:
endpoint: !ref d2cEndpoint
cloudToDeviceMapping:
type: Cybus::Mapping
properties:
mappings:
- subscribe:
endpoint: !ref c2dEndpoint
publish:
topic: Azure/IoTHub/Read
Code-Sprache: YAML (yaml)
There are some helpful tools that are suitable for prototyping or exploring the data on your Azure IoT Hub within Visual Studio Code. These tools should help you to implement your use cases faster.
The Workbench service that comes with Connectware is a Node-RED instance that runs securely inside Connectware as a service. This allows you to install any Node-RED nodes within the service container for quick prototyping.
Important: We do not recommend using Node-RED in production instances as we cannot guarantee reliability. This should only be considered as a rapid-prototyping tool.
node-red-contrib-azure-iot-hub is a Node-RED module that allows you to send messages and register devices with Azure IoT Hub. It includes a total of four Node-RED cloud nodes: Azure IoT Hub, Azure IoT Registry, Azure IoT Hub Receiver, and Azure IoT Hub Device Twin. For more information on the module, refer to Node-Red.
Azure IoT Tools is a collection of Visual Studio Code extensions for working with Azure IoT Hub. With these extensions, you can interact with an Azure IoT Hub instance, manage connected devices, and enable distributed tracing for your IoT Hub applications. You can also subscribe to telemetry messages sent to the IoT Hub for quick testing.
For more information on installing and using Azure IoT tools, refer to the Visual Studio Marketplace.
Azure IoT Explorer is an open source cross-platform user interface for interacting with Azure IoT Hub without logging into the Azure portal. This tool can be used to perform tasks like creating, deleting, and querying devices within the IoT Hub. Device functions such as sending and receiving telemetry, and editing device and module twin configuration are also possible with this tool.
For more information on Azure IoT Explorer, refer to GitHub.
In this lesson, we will send data from the Connectware MQTT Broker to AWS IoT.
It is required to set up a Connectware instance and at least one AWS IoT Device. In case of using AWS IoT at the edge, an AWS IoT Greengrass Core has to be set up.
We assume you are already familiar with Connectware and its service concept. If not, we recommend reading the articles Connectware Technical Overview and Service Basics for a quick introduction. Furthermore, this lesson requires basic understanding of MQTT and how to publish data on an MQTT topic. If you want to refresh your MQTT knowledge, we recommend the lessons MQTT Basics and How to connect an MQTT client to publish and subscribe data.
This article is divided into three parts.
First, it provides general information about AWS IoT services and their differences. Feel free to skip this section if you are familiar with AWS IoT and the differences between AWS IoT Core and IoT Greengrass.
Then, the current integration mechanisms between Connectware and the AWS IoT are explained through a hands-on approach.
Finally, the article describes the tools to work with your MQTT use case to prototype, review and monitor the integration scenario.
AWS IoT is a managed cloud platform that lets connected devices interact easily and securely with cloud applications and other devices. AWS IoT practically supports a nearly unlimited number of devices and messages, and can process and route those messages to AWS endpoints and to other devices reliably and securely.
For AWS IoT, Amazon offers a software development kit available for most popular programming languages and platforms.
AWS IoT Core is the main component to manage devices, their certificates, shadows, Greengrass resources and integration rules to subsequent AWS resources like IoT Analytics. It also offers ways to audit and test your IoT use cases.
AWS IoT Greengrass extends AWS Cloud resources to edge devices, so they can act locally on the generated data, while still using the cloud for management, analytics, and durable storage. It is possible for connected devices to interact with AWS Lambda functions and Docker containers, execute predictions based on machine learning models, keep device data in sync, and communicate with other devices – even when not connected to the Internet.
Greengrass has the following advantages:
Although in many scenarios these advantages are very significant, one could also mention some drawbacks to make the picture more complete:
Before proceeding further, first set up AWS IoT Core (and AWS IoT Greengrass for an edge deployment) by following the respective instructions:
To integrate AWS IoT with Cybus Connectware, the built-in MQTT connector with TLS support is the simplest, most reliable and secure way of communication. For a successful AWS IoT integration, Connectware does not require more than that. As an additional advantage, the Connectware MQTT connector has also data buffering built-in, so that data is stored locally when there is a temporary connection failure with AWS IoT Core or Greengrass Core.
There can be two integration scenarios.
In the first integration scenario, the Connectware connects directly to the AWS cloud:
In the second integration scenario, the Connectware is connected to Greengrass Core, which is meant to be deployed as a gateway to the AWS cloud next to the Connectware IIoT Edge Gateway:
For AWS IoT connections using the Connectware, the following has to be configured:
For details on how to get this information, see the article How to connect AWS IoT and Greengrass. Use the example below to implement a simple AWS IoT service transmitting any data structure in the selected MQTT topic.
The definitions
part requires PEM formatted certificates:
You may then configure Endpoint and Mapping resources following the Cybus resource documentation.
The commissioning file below sends any data published on topics ${Cybus::MqttRoot}/test/#topic
to AWS IoT into topics TestDevice/$topic
with a simple transformation rule.
Make sure you are publishing data on the Connectware broker on the respective topic. The placeholder ${Cybus::MqttRoot}
represents the root topic defined as services/<serviceId>
after the service is successfully started. The notation #topic
/$topic
represents a wildcard mapping from any topic name used in subscribe
to the same topic name in publish
, which has the effect of an MQTT bridge with applied rules like the transformation in the example.
Further details on MQTT topic transformations can be found in the article How to connect an MQTT client to publish and subscribe data.
description: >
Cybus Connectware to AWS IoT Core
metadata:
name: AWS IoT Core Test
version: 1.0.0
provider: cybus
homepage: https://www.cybus.io
parameters:
Aws_IoT_Endpoint_Address:
type: string
description: The ATS endpoint to reach your AWS account's AWS IoT Core
default: <your-aws-account-endpoint-id>-ats.iot.eu-central-1.amazonaws.com
definitions:
# The root CA certificate as PEM format (AmazonRootCA1.pem)
caCert: |
-----BEGIN CERTIFICATE-----
-----END CERTIFICATE-----
# The device certificate in PEM CRT format
clientCert: |
-----BEGIN CERTIFICATE-----
-----END CERTIFICATE-----
# The device private key in PEM format
clientPrivateKey: |
-----BEGIN RSA PRIVATE KEY-----
-----END RSA PRIVATE KEY-----
resources:
awsMqttConnection:
type: Cybus::Connection
properties:
protocol: Mqtt
connection:
host: !ref Aws_IoT_Endpoint_Address
port: 8883
scheme: mqtts
clientId: !sub "${Cybus::ServiceId}-awsMqttConnection"
mutualAuthentication: true
caCert: !ref caCert
clientCert: !ref clientCert
clientPrivateKey: !ref clientPrivateKey
sourceTargetMapping:
type: Cybus::Mapping
properties:
mappings:
- subscribe:
topic: !sub "${Cybus::MqttRoot}/test/#topic"
publish:
connection: !ref awsMqttConnection
topic: TestDevice/$topic
rules:
- transform:
expression: |
(
{
"deviceId": "TestDevice",
"payload": $
}
)
Code-Sprache: YAML (yaml)
In order to connect to a Greengrass Core, the example service commissioning file needs several changes:
See the article How to connect AWS IoT and Greengrass about how to get the Greengrass Group Certificate Authority.
parameters:
...
awsGreengrassClientId:
type: string
default: TestDeviceEdge
...
resources:
greengrassTestDeviceEdgeMqttConnection:
type: Cybus::Connection
properties:
protocol: Mqtt
connection:
host: !ref Greengrass_Core_Endpoint_Address
port: 8883
scheme: mqtts
clientId: !ref awsGreengrassClientId
mutualAuthentication: true
caCert: !ref caCert
clientCert: !ref clientCert
clientPrivateKey: !ref clientPrivateKey
...
Code-Sprache: YAML (yaml)
To implement or maintain a new IIoT Edge integration use case as fast and reliable as possible, there are suitable tools for working with MQTT, Connectware and AWS IoT.
The AWS CLI generally helps with any task on AWS. In this case we have at least two tasks being most efficiently completed using the CLI:
1) Find out the AWS IoT ATS endpoint defined for your AWS account:
aws iot describe-endpoint --endpoint-type iot:Data-ATS
Code-Sprache: YAML (yaml)
The response contains the AWS account specific ATS (Amazon Trust Services) endpoint address to be used as the MQTT hostname:
{
"endpointAddress": "a7t9...1pi-ats.iot.eu-central-1.amazonaws.com"
}
Code-Sprache: YAML (yaml)
2) Get the Greengrass Group Certificate Authority certificate in case of using AWS IoT Greengrass. You then need the following for the caCert
setting in the service commissioning file instead of the Amazon Root CA:
aws greengrass list-groups
aws greengrass list-group-certificate-authorities --group-id "4824ea5c-f042-42be-addc-fcbde34587e7"
aws greengrass get-group-certificate-authority --group-id "4824ea5c-f042-42be-addc-fcbde34587e7"
--certificate-authority-id "3e60c373ee3ab10b039ea4a99eaf667746849e3fd87940cb3afd3e1c8de054af"
Code-Sprache: YAML (yaml)
The JSON Output of the latter call has a field PemEncodedCertificate
containing the requested information which needs to be set as the caCert
parameter similar to this:
-----BEGIN CERTIFICATE-----
MIIC1TCCAb2gAwIBAgIJANXVxedsqvdKMA0GCSqGSIb3DQEBBQUAMBoxGDAWBgNVBAMTD3d3dy5leGFtcGxlLmNvbTAeFw0yMDEwMDUwNTM4MzRaFw0zMDEwMDMwNTM4MzRaMBoxGDAWBgNVBAMTD3d3dy5leGFtcGxlLmNvbTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAM/0NrS45cm0ovF3+8q8TUzj+E3UH8ldnJJPCQFGMaL+7PoxbO0fYf3ETkEW+dijIZOfus9dSPX7qBDbfilz/HtNppGDem4IjgC52iQl3B1R7TvU8yLNliv43uDDUd+PkzW1cWbUuykr5QPG2sIDSANukosvRdFKO4ydP0Hr9iUdOfbg4k6hMFCrzJubKQqhcBTSsxGtl78abx0Q49shuWr9RRjzqE6mRFa4h0DrKBstgAfmsDRGm4ySBCM7lwxphSsoejb6l39WI/MNU7/U7cGj26ghWHAWp8VCksBOqma8tmr/0BuqcCgKJYaDr1tf4SVxlwU20K+jz0pphdEwSj0CAwEAAaMeMBwwGgYDVR0RBBMwEYIPd3d3LmV4YW1wbGUuY29tMA0GCSqGSIb3DQEBBQUAA4IBAQBkcKC3cgHJGna6OxA5QM3dGM5pEiSXyZt5HWoW8z6wUlYtir6U+mWIb9yg7zaSy9nUOqU4sizQh1HG/Mq9K2WbflGafvfN0wW16uyINdjcfGYDh43UDkXHr5Xzky5XIgt0Fx4BWmjgbLYsza7qpbeIg5ekUYPYQw1Ic2sNpyncmS0eutg4tAO7uzDu1x84WPcZzUjDHKYfupuDXkWroPnHTAxlJ6vtgW976c3Z5rQ5l8bUysWhLBEM8q2OP/zmGDo7fpUHYOKo5qU4h7vGD3t0Pb4ufPOd7XtHuY6HsI2cAPV3tpuetHH6wyAQTG9luhdYrZjAp+ZvlwBm+9nXYp/Y
-----END CERTIFICATE-----
Code-Sprache: YAML (yaml)
The Workbench service is basically a Node-RED application running securely on the Connectware as a service. This opens up the possibility to install any Node-RED nodes within the service container for quick prototyping as well as for the production environment. If your use-case cannot be achieved with the above service commissioning file, using the workbench will give you some flexibility and additional tools to prototype your solution using Node-RED modules.
In case of AWS IoT, MQTT connection is enough for most integration scenarios. You may use simple injection nodes and some random value generator in order to implement and test the use northbound to AWS IoT:
If there are other requirements such as working with shadow devices and other AWS resources, e.g. as part of the IoT Greengrass Core deployment, you may want to use additional Node-RED modules supporting AWS.
If it comes to more complex data management and handling, you may want to use the AWS IoT Device SDK to create a specific Connector Service for Connectware to cover your requirements.
In most cases, it is enough to process any kind of device data and apply rules to them on the Connectware as the most powerful edge gateway tool. Similar capabilities can be used on the near-cloud gateway AWS IoT Greengrass or AWS IoT Core itself to manage rules and transformations near the shadow devices definitions.
What works best depends on your business strategy and technical constraints.
Now that we are successfully sending the data to the IoT Core, we can monitor the transmitted data using various AWS resources.
The obvious tool is the AWS IoT Core MQTT Client offered on the AWS IoT console. With this tool you regularly subscribe to your topic defined in the service commissioning file for outgoing data:
In order to make use of AWS resources, you define AWS IoT rules and define actions appropriately, e.g. transmission to IoT Analytics and a DynamoDB table:
The AWS IoT Console helps to quickly implement data transfer to these endpoints.
An example of how to work with these resources could be a change to the transformation mentioned above to better meet the requirements using the fast and easy mapping support of the Connectware. Given a requirement to flatten an original data object injected into the internal topic, you can easily transform that data using a Connectware transformation rule using Jsonata:
Given a structured object:
"DeviceData": {
"Temperature": <decimal>,
"Position": {
"X": <decimal>,
"Y": <decimal>,
"Z": <decimal>
}
}
Code-Sprache: YAML (yaml)
As an example, the above mentioned mapping could be then enhanced for flattening the elements and adding a timestamp:
sourceTargetMapping:
...
rules:
- transform:
expression: |
(
{
"deviceId": "TestDeviceEdge",
"payload": $
}
)
- transform:
expression: |
(
{
"deviceId": "TestDeviceEdge",
"timestamp": $now(),
"temperature": $.payload.DeviceData.Temperature,
"position_x": $.payload.DeviceData.Position.X,
"position_y": $.payload.DeviceData.Position.Y,
"position_z": $.payload.DeviceData.Position.Z
}
)
Code-Sprache: YAML (yaml)
After implementing the use case, you may see the options to shorten things a bit. Connectware then plays its strength with fast integration processes near the connected devices, where most of the data pre-processing can be realized with low latency and fewer costs before transmitting it to the cloud.
The enhanced transformation rule within Connectware mentioned above may be inspired by a requirement to write the data in a well-structured database:
Or the requirement was to create some graph with Amazon Quicksight:
If it comes to the AWS Cloud, there is a vast amount of resources that can be useful to create your IoT Application. You should especially have a look at lambda functions that could be deployed to your IoT Greengrass Core instance.
Other new tools like AWS IoT SiteWise or AWS IoT Things Graph may be useful to build your IoT applications faster with easier management and monitoring.
This lesson first offered a brief introduction to AWS IoT and its components available for integration with other services. Then it explained how to send data from the Connectware MQTT Broker to AWS IoT Core or Greengrass Core with a simple commissioning file using the built-in MQTT connector of Connectware. Furthermore, the Cybus workbench service for prototyping more advanced scenarios was presented. The lesson finished with a description of some basic and advanced tools used to monitor data flow between AWS IoT and Connectware.