Protocol translator development

The protocol translator (PT) is the most important component for connected devices because it:

• Handles device connectivity to Izuma Device Management.
• Maps device resources to LwM2M-compliant Objects and Resources.
• Maps Device Management operations (read, write, and execute) into a connected device protocol.

Protocol translator development with the JSON-RPC API

Refer to the JSON-RPC API specification.

Protocol translator development with the C API

Refer to the GitHub example repository for detailed examples.

Device Management, its Client, and the protocol translators have their own event loop threads. Therefore, PT operations must not take a long time. If the PT blocks an event, Device Management does not operate normally. If you have lengthy operations (for example, write to the translated device) implemented in your protocol translator, transfer them to a worker thread (similar to what operating systems ask you to do in an interrupt context).

Edge Core PT client prints an alert in the logs for long operations. Pay attention to the log entries:

Callback processing took more than 500 milliseconds to run, actual call took NNNN ms

If you see this in the logs, you need to delegate the operations to worker threads.

Responsibilities of protocol translator developer

Device connection

The protocol translator acts as a bridge between Edge Core and the device. Devices connect to Edge through protocols not known in advance, and therefore, you need to handle the connecting and disconnecting.

Neither the Edge Core API nor the protocol translator API currently track the connected devices or their lifetimes. Therefore, they do not automatically clean up disconnected devices from their internal lists.

Data format translation

The role of the protocol translator is to bridge any arbitrary data format to an LwM2M-compatible format. The protocol translator API provides an interface to interact with Edge Core and expose Resources to Device Management. You need to translate the incoming data from devices to Objects, Object Instances, and Resources using this API.

Protocol translator functions

This document describes protocol translator functions. Read the API documentation for detailed parameters and return values.

Connecting to Edge Core

When connecting the protocol translator to Edge Core, the protocol translator client registers itself with Edge Core. Success or failure is communicated to the protocol translator by calling the supplied success or failure callback function. The protocol translator connects, and you can register devices when the success callback function is called.

Adding devices

Once you have set up the PT, you can start adding devices. To add devices, use the pt_device_create() or pt_device_create_with_userdata() functions. These functions prepare devices in the protocol translator client, but do not register the devices yet to Edge Core.

Adding Resources to the devices

To add Resources, call pt_device_add_resource() or pt_device_add_resource_with_callback() functions. Once you have added all necessary Resources to the device, you can register it to Edge Core.

Registering the devices

To register a device to Edge Core and Device Management, call pt_device_register().

The information flow between Edge and Device Management is not visible for protocol translators.

For example, if the registration of the protocol translated device fails in Device Management, the error is not returned to the protocol translator. The protocol translator only sees Edge-related operations and responses.

Note: All connecting devices must have a unique and persistent ID.

Device writes a value

When a device updates a value, it uses one of the following mechanisms:

• The device pushes the value to the PT. The protocol indicates this through an interrupt or wake-up mechanism.
• The PT polls the device periodically for new data. The protocol does not have interrupt or wake-up capability.

In either case, the PT must call the pt_device_set_resource_value() function to set the value for a Resource. The function must be called with the right path, indicating the whole Object, Object Instance, or Resource path it wants to write to. When all values are set, the PT must call the pt_device_write_values() function to write the values to Edge Core. Both functions operate on a protocol translated device identified by the device ID.

Refer to the protocol translator API documentation for data types and function arguments.

Depending on your use case, we recommend that you avoid updating Resource values all the time to limit network traffic to the device and Device Management. This also allows the device to conserve power by sleeping.

Notifications from Device Management Client to Device Management can generate an error. This error is not returned to the protocol translator.

The protocol translator only sees Edge-related operations and responses.

Cloud-initiated read operation

A read operation occurs when Device Management asks for a value from the connected device. The read operation never arrives to the connected device itself. Instead, the device pushes its values to Edge Core as needed, which in turn provides the values stored in its memory to Device Management.

Cloud-initiated write operation

Write operations are Device Management-initiated actions to write a value to the connected device. This chain of events is complex, because the connected device may be sleeping and the PT can only write the value when the device is awake and connected. Therefore, there are two possible scenarios:

• Write to devices always online and with fast write capability.
• Write to sleepy or slow devices.

The protocol translator API client forwards the write operation to the callback function given in the Resource add operation. Refer to adding resources.

Write to an always-online device with fast write

1. A write request arrives from Device Management to Edge Core.
2. Edge Core checks that the write request is valid.
3. If the request is valid:
   1. Edge Core passes the request with OPERATION_WRITE as an operation to the PT.
   2. Edge Core returns ACK to Device Management. This tells Device Management that Edge has started processing the request.
   3. Device Management replies to the web application with ACCEPTED response. See PUT /v2/device-requests/{device-id}.
   4. PT returns success if the write operation to the connected device succeeded.
      1. PT returns OK to Edge Core.
      2. Edge Core sets the value to the mediated LwM2M resource.
      3. Edge returns CHANGED to Device Management.
   5. PT returns failure if the write operation to the connected device failed.
      1. PT returns ERROR to Edge Core.
      2. Edge Core does not set the value to the mediated LwM2M resource.
      3. Edge Core returns INTERNAL SERVER ERROR to Device Managemnt.
4. If the request is invalid:
   1. See Write request error handling.

When Edge Core checks the incoming write request, the following error scenarios are possible. In these cases the protocol translator does not receive the request.

1. If the request is invalid (for example, invalid data format for the Resource type):
   • Edge Core responds BAD REQUEST to Device Management.
2. If the request is invalid (for example, the write operation is not allowed for the Resource):
   • Edge Core responds METHOD NOT ALLOWED to Device Management.
3. If the request is invalid (for example, the Resource doesn't exist):
   • Edge Core responds NOT FOUND to Device Management.
4. If there is already a request going on:
   • Edge Core responds PRECONDITION FAILED to Device Management.

To prevent PRECONDITION FAILED error, the protocol translator needs to be designed so that it always returns either OK or ERROR response.

Device Management has a timer (currently two minutes) to respond to the web application using Device Management. If there is no reponse to the write request within the timeout, Device Management responds GATEWAY TIMEOUT to the web application.

Write to a sleepy or slow write device

This scenario is for devices that may not be online or reachable by Device Management. The protocol translator developer needs to know that the value can be written to the device.

The devices can sleep for a very long time, for example, one hour. Therefore, there is no time to wait for them to wake up as the incoming request is received. The protocol translator must implement a work queue in which it stores the incoming requests and processes them when the devices communicate with it.

1. A write request arrives from Device Management to Edge Core.
2. Edge Core checks that the request is valid.
3. If the request is valid:
   1. Edge Core passes the request with OPERATION_WRITE as an operation to the PT.
   2. Edge Core returns ACK to Device Management. This tells Device Management that Edge has started processing the request.
   3. Device Management replies to the web application with ACCEPTED response. See PUT /v2/device-requests/{device-id}.
   4. PT stores the incoming request content in the work queue.
   5. PT knows about the sleepy device. For example, it knows if you can expect it to wake up normally.
   6. PT returns success if the sleepy device is expected wake up so that you can communicate with it.
      1. PT returns OK to Edge Core.
      2. Edge Core sets the value to the mediated LwM2M resource.
      3. Edge Core returns CHANGED to Device Management.
   7. PT returns a failure if the queue is full or in an error condition.
      1. PT returns ERROR to Edge Core.
      2. Edge Core does not set the value to the mediated LwM2M resource.
      3. Edge Core returns INTERNAL SERVER ERROR to Device Management.
   8. The device comes back online.
   9. PT writes the value to the connected device.
4. If the request is invalid:
   1. See Write request error handling.

Generic write operation sequence

This sequence diagram describes the asynchronous aspect of write operations.

Execute operation

The execute operation is similar to the write operation. There are two possible flows depending on device connectivity and speed of the operation. The flows are the same, except the operation type is execute instead of write.

The protocol translator API client forwards the execute operation to the callback function given in the Resource add operation. See to adding resources.

Execute, device always online with fast operation

1. An execute request arrives from Device Management to Edge Core.
2. Edge Core checks that the request is valid.
3. If the request is valid:
   1. Edge Core returns ACK to Device Management. This tells Device Management that Edge has started processing the request.
   2. Device Management replies to the web application with ACCEPTED response. See POST /v2/device-requests/{device-id}.
   3. PT executes the value to the connected device if it is awake. Otherwise, PT caches the operation and dispatches it when the device reconnects later.
   4. PT returns success if the execute operation to the connected device succeeded.
      1. PT returns OK to Edge Core.
      2. Edge Core sets the value to the mediated LwM2M resource.
      3. Edge returns CHANGED to Device Management.
   5. PT returns failure if the write operation to the connected device failed.
      1. PT returns ERROR to Edge Core.
      2. Edge Core does not set the value to the mediated LwM2M resource.
      3. Edge Core returns INTERNAL SERVER ERROR to Device Managemnt.
4. If the request is invalid:
   1. See Write request error handling.

Execute, sleepy devices or slow operation

The devices can sleep for very long time, for example for one hour. Therefore there's no time to wait for them to wake up as the incoming request is received. The protocol translator must implement a work queue in which it stores the incoming requests and processes them when the devices communicate with it.

1. An execute request arrives from Device Management to Edge Core.
2. Edge Core checks that the request is valid.
3. If the request is valid:
   1. Edge Core passes the request with OPERATION_EXECUTE as an operation to the PT.
   2. Edge Core returns ACK to Device Management. This tells Device Management that Edge has started processing the request.
   3. Device Management replies to the web application with ACCEPTED response. See POST /v2/device-requests/{device-id}.
   4. PT executes the value to the connected device if it is awake. Otherwise, PT caches the operation and dispatches it when the device reconnects later.
   5. PT returns success if the execute operation to the connected device succeeded.
      1. PT returns OK to Edge Core.
      2. Edge Core sets the value to the mediated LwM2M resource.
      3. Edge returns CHANGED to Device Management.
   6. PT returns failure if the execute operation to the connected device failed.
      1. PT returns ERROR to Edge Core.
      2. Edge Core does not set the value to the mediated LwM2M resource.
      3. Edge Core returns INTERNAL SERVER ERROR to Device Managemnt.
   7. The device comes back online.
   8. PT writes the value to the connected device.
4. If the request is invalid:
   1. See Write request error handling.

Execution always occurs when the proprietary device is awake and ready. Device Management does not currently show the execute completion status through Edge. Future Device Management implementations may incorporate a status notification.

Deregistering a device from Device Management

When you remove a connected device from Device Management, the PT should call pt_device_unregister() to Edge Core with the device ID to deregister.

The deregistration event is communicated to Device Management. When the event has been processed, the device is no longer associated with Edge. The device is registered until the registration expires.

Device removal relies on the expiration mechanism of Device Management. The device lifetime is common to Device Management and all devices it hosts. You can set the device lifetime in the mbed_cloud_client_user_config.h file:

#define MBED_CLOUD_CLIENT_LIFETIME              3600

Device certificate enrollment

The protocol translator provides APIs to perform certificate enrollment or certificate renewal. To enable certificate enrollment, use the PT_DEVICE_FEATURE_CERTIFICATE_RENEWAL feature flag when registering the device. The protocol translator initiates a certificate enrollment by calling the pt_device_certificate_renew API. To learn how to request certificate enrollment from Device Management, see Certificate renewal tutorial.

The protocol translator API client calls the pt_device_certificate_renew_request_handler callback function. To complete a Device Management-initiated certificate enrollment call the pt_device_certificate_renew_request_finish API. Refer to the MQTT PT example in the examples repository for an example implementation.

Device-initiated certificate enrollment

Device Management-initiated certificate enrollment

Moving devices - special considerations

Note the following when moving devices:

• Currently, there are corner cases that fail with moving devices. Therefore, support for moving devices is limited.
• Due to latencies in adding a device to Edge or registering a device to Device Management, you must avoid situations where the device connects to or disconnects from Edge repeatedly (hysteresis).
  • A quickly repeating connect - register - disconnect - deregister sequence leads to undefined and random behavior.
• If there are unhandled operations (such as write or execute) pending while the device moves to another Edge instance, the events remain in the queue forever unless the PT implements a solution for this.
  • Either you agree to lose the events by a timing filter (if the device has not been seen within its lifetime, has moved, or run out of battery), or one PT can sync the events with another PT.

Protocol translator reference implementations

Protocol translator example

The example demonstrates the bare minimum to implement a protocol translator without the protocol-specific implementation and how to use the pt-client APIs. It creates three simulated devices, cpu-temperature, thermometer and thermostat, each demonstrating some combination of readable, writable and executable resources. See the application source code.

MQTT protocol translator example

The example demonstrates how to translate devices from an MQTT broker. It shows how to interface with the MQTT broker using the Mosquitto library and how to implement a translation based on the MQTT topics and payloads. See the application source code.

Bluetooth Low Energy (BLE) protocol translator example

The example demonstrates how to translate BLE devices. It shows how to interface with the BlueZ Bluetooth daemon over DBus and how to implement a translation of BLE Generic Attributes (GATT) server services and characteristics. See the application source code.

JavaScript examples

Instead of using the provided pt-client library like the other examples, these JavaScript examples use the Edge Core protocol translator and management API directly. See the application source code.