Bridging OpenDDS® and MQTT Messaging

Introduction

There are numerous standards-based messaging frameworks available in this increasingly interconnected world. Among the most popular standards is a publish-subscribe system called MQTT.

MQTT first appeared in 1999 and was standardized in 2013 by OASIS. It originally stood for the Message Queuing Telemetry Transport, but it now has no official meaning. MQTT is popular in constrained computing environments, such as microcontrollers. For example, it is extensively used in Tasmota, which is an open source firmware for home automation devices based on the Espressif ESP-series chipsets.

DDS (Data Distribution Service) is another publish-subscribe message system standardized in 2004 by OMG. The goal of this article is to explore how to bridge MQTT and DDS using Tasmota as an example.

MQTT and DDS Capabilities

First, let's compare MQTT and DDS to establish their respective capabilities. For the comparison, we will assume a DDS system using RTPS over UDP and a MQTT system using TCP, as they represent conventional configurations.

Use of a Message Broker

Perhaps one of the greatest differences between the DDS over RTPS and MQTT is how messages are delivered.

• MQTT follows a client/server model where MQTT clients send messages to and receive messages from a broker.
• In a DDS system using RTPS, messages are delivered directly between DDS participants.

Both approaches have their pros and cons.

For reliable message delivery in DDS, the work of ensuring that a message made it to all interested recipients is performed by the participant sending the message. In MQTT the broker takes up this work, so the sender only has to send it to the broker. By the nature of being a single running service, this makes it easy to configure and monitor the messages being passed around. However, it also becomes a single point of failure.

Network Protocols

Next let's look at what is actually being sent across the network to deliver these messages.

MQTT systems use TCP, while DDS/RTPS systems use a custom protocol built on top of UDP. TCP tries to make sure data makes it to its destination by automatically resending it if necessary, whereas UDP just sends data once. RTPS optionally implements the same level of reliability as TCP, so there isn't much of a functional difference from a user's perspective.

Message Encoding

The way the contents of the message are usually created is notably different though.

In a traditional DDS system, a user is expected to write an interface definition language (IDL) file that defines the message format. This IDL file is used to generate code that includes two parts needed to send and receive messages. The first is a generated type in the user's programming language that mirrors the type in the IDL file; the second is code to convert the generated type to and from a Common Data Representation (CDR) binary format that makes up the message payload.

DDS takes advantage of the fact that it can read the fields of the user’s type to provide certain features. These features include DDS instances, which are keys corresponding to fields marked by the user in the IDL, and message filtering based on contents using an SQL-like language.

MQTT doesn't prescribe any sort of format or encoding for messages. Plain text, including JSON, is common. The user is free to use a more space-efficient binary payload format is desired, but one is not provided by MQTT itself. Google's protocol buffers have been used before, and it's theoretically possible to use the same IDL and CDR format as DDS.

Likewise in DDS, as we will see shortly, it's trivial to emulate MQTT's simple message payload in IDL using a struct with an octet sequence or string field.

So both systems are flexible with respect to the contents of messages; however DDS includes a structured binary encoding that can be leveraged for certain features, while MQTT doesn't.

API

Finally for our comparison, let's look at the API used to send and receive messages.

The DDS API is defined by the main DDS specification in IDL and covers a wide range of functionality. This includes a hierarchy of entities that allow fine-grain control over behavior. The mains ones are:

• DomainParticipant
• Topic
• Publisher
• Subscriber
• DataWriter
• DataReader

At least one of each has to be created and configured to send and receive messages. The last two are the only two that directly deal with data, and generally only for a single Topic that is fixed to a single data type.

Messages that have been received by a DataReader can be retrieved manually using the read or take methods or by callback via a DataReaderListener that can also handle other events.

Quality-of-service (QoS) can be set for each entity to configure a number of behaviors. This is a simplified overview, and there are additional features and aspects of the API, such as the aforementioned instances and content filtering. See the OpenDDS Developers Guide for more information.

The MQTT specification doesn't directly describe an API like DDS does, but MQTT client implementations tend to follow a similar API. This consists of a client object that can manually send or wait to receive a message after connecting to a broker. A callback can also be set for receiving a message.

The messages that are received depend on the topic filter that is set on the client. Topics work the same way they do in DDS, where messages have an associated topic, but because there's no strict type system, a client can receive multiple or even all messages a broker has depending on the filter.

There is also QoS like in DDS, but it only guarantees reliability of getting the message to and from the broker. Specific implements may offer additional features and settings. The simplicity of the MQTT API is achieved by foregoing additional features.

Setting up and Exploring Tasmota

Now, let's talk briefly about setting up an MQTT broker and Tasmota. Setting up Tasmota will heavily depend on the hardware being used, so this will only be a high-level overview.

Setting up an MQTT broker is fairly easy. The author used Eclipse Mosquitto, but any compatible MQTT broker should be sufficient.

Tasmota can be flashed to an ESP-series microcontroller, but it also comes pre-installed on some commercially available devices, including light switches, plugs, bulbs, and others. The author is using a set of smart plugs with energy monitoring capabilities, and the examples reflect this.

Once the device is connected to WiFi, MQTT has to be enabled, and the broker address has to be provided.

Tasmota presents the following model for discovering and exchanging data with devices:

First, there are topics for discovering devices, which can accessed be using tasmota/discovery/+/config in MQTT topic notation. The plus is a wild card for a single level of the topic hierarchy which, like a UNIX-like file path, is separated by /. In this case the wild card matches the device name.

The messages of these config topics consist of JSON that describes some of the devices' basic information, such as the name and the topics to use to access the device. These topics fall into three categories:

1. cmnd for sending commands for changing the device state and requesting information
2. stat for getting status and requested information back
3. tele for periodic information

The simple commands and responses are usually short plain text strings. For example, if there is a device with the official name tasmota_257EAF, the power can be toggled by publishing toggle to the cmnd/tasmota_257EAF/Power topic. If it's off, then it will respond on stat/tasmota_257EAF/POWER (all the response topics seem to be capitalized) with OFF. It will also respond on stat/tasmota_257EAF/RESULT with {"POWER":"OFF"}.

More complicated responses are in JSON, such as the config topics. If publishing to topic cmnd/tasmota_257EAF/Status with message 10, the device will respond with detailed energy consumption information, like:

{
  "StatusSNS":{
    "Time":"2022-05-03T17:29:43",
    "ENERGY":{
      "TotalStartTime":"2022-01-13T22:50:06",
      "Total":405.686,
      "Yesterday":0.612,
      "Today":0.410,
      "Power":427,
      "ApparentPower":428,
      "ReactivePower": 0,
      "Factor":1.00,
      "Voltage":118,
      "Current":3.623
    }
  }
}

The device is drawing 427 watts because it’s a kotatsu, a short Japanese table with a blanket skirt and a heater that was actively heating at that moment. The kotatsu itself isn’t relevant; however the amount of power it can draw will be relevant later.

Bridging MQTT and DDS

If one wants to control an existing collection of MQTT devices using DDS, there are two main challenges to address:

1. First, at least one process must speak to both the MQTT broker and the other DDS participants. For the examples, we will assume a single process acts as a relay between DDS participants and MQTT clients.
2. Second, and more significant, is the translation of messages between DDS and MQTT.

The message format will heavily depend on the situation. If DDS is already being used, then it might make sense to use DDS’s CDR format in MQTT. In the examples, it’s not practical to change the format that Tasmota is using, so there are two choices with the DDS side:

1. The MQTT message content can be passed through the relay without translation
2. The message can be translated into a more “proper” DDS message.

The example programs corresponding to both of these choices are available on GitHub.

In both of these approaches, RapidJSON is used to convert Tasmota JSON to and from IDL-generated types. Depending on the JSON, it could be possible to do this with OpenDDS technology’s to_json and from_json methods. Those methods were not used in the examples because the IDL types were simple and OpenDDS middleware can’t convert some of the JSON that Tasmota produces at the moment. If it’s an option and there’s a significant amount of IDL, then to_json and from_json would make it easier to convert to and from JSON.

Generic Approach

The first example uses a generic approach where each messages contains the MQTT topics and message content as is. One DDS topic is used to send to the MQTT broker and another DDS topic is used to receive from the MQTT broker.

@topic
struct MqttMessage {
  @key string topic;
  string message;
};

The generic-relay program receives MQTT messages and writes them to the DDS topic and vice versa. It has no knowledge of how Tasmota works, and the contents are interpreted only by the tasmota-toggle program. tasmota-toggle gets the on/off status of each Tasmota device it discovers, then sends a message to toggle it.

This approach, as opposed to just sending a toggle command as described previously, was used to illustrate a back-and-forth data exchange. Because it doesn’t need to know how Tasmota works, the generic-relay could be used for any MQTT application without changes, although this would make sense only if the DDS recipients can use the messages.

Here is an excerpt of an example of running tasmota-toggle with a lot of irrelevant details in the JSON taken out:

Received on tasmota/discovery/98CDAC257EAF/config: {dn":"lr.kotatsu","fn":["Kotatsu"],"t":"tasmota_257EAF"}
Received on cmnd/tasmota_257EAF/Power:
Received on stat/tasmota_257EAF/POWER: OFF
Toggling Kotatsu from 0 to 1
Received on cmnd/tasmota_257EAF/Power: 1
Received on stat/tasmota_257EAF/RESULT: {"POWER":"ON"}
Received on stat/tasmota_257EAF/POWER: ON

Message Mapping Approach

The second example shown by the idl-relay and tasmota-power programs uses more complex mapping between MQTT messages and DDS messages. This approach also uses two topics, but the types are specific to how the topics will be used. These types are simplified IDL versions of the on/off and energy consumption topics shown before:

@topic
struct Power {
  @key string device_name;
  string display_name;
  boolean on;
};
 
@topic
struct Wattage {
  @key string device_name;
  string display_name;
  int32 watts;
};

idl-relay is similar to the tasmota-toggle application in that it is trying to discover Tasmota devices, but it's doing this using MQTT directly and then getting and writing a Wattage for the energy consumption of a device to a DDS topic. It will send power on/off commands to the device if another DDS topic with the Power type calls for it.

tasmota-power takes a wattage limit and reads the Wattage DDS topic for wattage usage. For every device that pushes the total wattage over the argument, tasmota-power shuts off the device using the Power DDS topic.

Here is an example output of tasmota-power:

./build/idl-relay/tasmota-limit 100
LR Light 1: 15 watts
Kotatsu: 0 watts
LR Light 2: 10 watts

The numeral 100 represents the limit in watts. The sum was below 100 watts, so nothing happened. If the kotatsu thermostat turns the heater on and tomata-power is run again, the total power will exceed the limit and the kotatsu will be shut off.

LR Light 1: 15 watts
Kotatsu: 429 watts
LR Light 2: 10 watts
Kotatsu exceeds limit by 329 watts, powering off

Conclusions

DDS and MQTT are distinctly different and reflect the needs of the environments that produced them. DDS emphasizes a robust set of features. MQTT focuses on simplicity. However, the examples show that they’re not so different, to the point that it is not impossible to bridge them. This interoperability can be used to extend one or the other to gain the advantages of both.