OMG DDS for eXtremely Resource-Constrained Environments (DDS-XRCE)

Introduction

The task of designing integrated solutions is frequently met with challenges when unusual architectural constraints from system and non-system sources are introduced into the equation. Normal integration patterns are desirable, but what happens when one or more actors in your deployment introduces unique requirements?

• "My equipment is located in extremely remote locations (with no power sources)."
• "My solution must integrate OEM equipment with only 64KB of flash memory on it."
• "Intermittent network connection will be the norm for devices in my solution."

Extending creative solutions into environments with these unique requirements drives our integration solutions further toward non-traditional architectural patterns.

This article discusses how industry standards are addressing such challenges while maintaining architectural integrity (e.g., non-functional requirements) along the way and bringing the benefits of risk reduction and extension to long-term supportability. In particular, we will look at the OMG's specification, DDS for eXtremely Resource Constrained Environments (DDS-XRCE), to understand what role implementations of this specification play in solving complex problems.

DDS Standards: A Terminology Check

When considering the use of Data Distribution Service (DDS) technology overall (including DDS-XRCE),  awareness of the inventory of DDS standards and, in the case of this article, how DDS-XRCE fits is a good way to level-set. 

The table below is a quick list of those standards.

Figure 1. Family of DDS Specifications

The DDS standards are created and managed by the Object Management Group (OMG), where many distributed computing standards have been created by industry consortium members since 1989. You can find more information about the OMG at http://www.omg.org.

Below is an architectural concept of where DDS-XRCE fits in an overall application scenario. For an XRCE type of application, three specifications out of the DDS family should apply:

1. DDS-XRCE
2. DDS Core
3. DDS-RTPS

Figure 2. DDS-XRCE Concept

Here are some resources for more detailed information about those three standards:

1. DDS (https://www.omg.org/spec/DDS/). This is the core DDS specification that implementers use to produce their DDS solutions.
2. DDS Interoperability Wire Protocol (DDSI-RTPS) (https://www.omg.org/spec/DDSI-RTPS/). Implementations of this specification layers on top of the core DDS implementation and provides Real Time Publish and Subscribe (RTPS) capability in an interoperable way. In other words, Implementer A's RTPS solution will publish/subscribe with Implementer B's RTPS solution seamlessly.
3. DDS-XRCE (https://www.omg.org/spec/DDS-XRCE/1.0/Beta2/PDF). Implementations of this specification extend the minimal necessary capability of DDS to very constrained environments and extend the reach of larger systems already using DDS technology to disseminate data into enterprise systems.

The figure below gives a little more insight into how each supports the overall demand for eXtremely Resource-Constrained Environments (XRCE).

Figure 3. Coordination of DDS Specifications for XRCE Capability

DDS-XRCE Motivation

When using terms like "extremely-constrained resource" to name a standard like DDS-XRCE, it becomes obvious that there are driving criteria to define it. Let's take a look at what the specification's authors used to motivate the details of the DDS-XRCE specification itself:

A DDS-XRCE solution would:

1. Operate over networks with bandwidth limited to 40-100Kbps
2. Work with devices that undergo sleep cycles; these devices may be active once every few minutes, days, months, or even years
3. Be simple and programming-language independent, supporting devices that are programmed in a highly specialized language or frameworks
4. Support a minimal discovery protocol, allowing plug-and-play deployments where the Agent location is dynamically discovered
5. Support accessing the full capabilities of DDS; any data type can be published or subscribed to with any DDS QoS
6. Support sending updates to multiple data-times on the same or multiple DDS Topics efficiently.
7. Support receiving information both reliably and in a best-effort manner, even if the information was sent while the (remote XRCE) Client was undergoing a sleep cycle
8. Support secure communication at the transport level
9. Provide full read/write access to any data in the DDS Global Data Space (subject to access control limits)
10. Provide a full implementation requiring less than 100KB of code

So immediately we see some pretty constrained expectations. But why would you try to impose the richness of DDS into such compact environments? The goal is not to embed all of DDS into such an environment, but rather to allow such limited processing to act as a participant in a larger DDS ecosystem.

In summary, the DDS-XRCE specification gives industry solution providers of DDS technology a common way of bringing a very disparate family of technology (small, limited equipment) into a larger system such that standard interoperability among different suppliers is maintained and one-off, proprietary solutions for each pathway are not required.

The next section gives more detail in how that is accomplished.

DDS-XRCE Architecture Features

Starting with a familiar depiction of a typical usage for DDS-XRCE below, you will see right away that the specification scope is limited to an XRCE Client and an XRCE Agent.

The ability of a very resource constrained environment to fully interact with a resource-rich DDS environment cannot be imposed on such small footprint devices (XRCE Client), so the XRCE Agent is introduced as that intermediary to do the heavy lifting on behalf of each connected client. We will discuss the Agent's integration with systems using DDS-RTPS and then specification details internal to the XRCE Client itself.

Figure 4. Common DDS-XRCE Deployment

Client-Server Protocol for DDS-XRCE

DDS technology supports a Publish/Subscribe integration style. The DDS-XRCE specification introduces a client/server protocol for XRCE adopters to use when connecting to a DDS-RTPS ecosystem. The specification provides reasons why this would be the best approach. It states the following (slightly reworded for clarity here).

When comparing an XRCE Client to any other DDS-RTPS application that would use the DDS API internally, XRCE Clients:

1. Do not have a standard API, so they are not portable across vendor implementations
2. Cannot operate without infrastructure support. They need a XRCE Agent to be reachable to them. This is a necessary consequence of the need for XRCE Clients to undergo deep sleep cycles.
3. Do not communicate directly peer-to-peer. All communications are brokered (relayed) by one or more DDS-XRCE Agents. This is also a necessary consequence of the need for Clients to undergo deep sleep cycles.

So as we see from that description, the XRCE Agent becomes a very important intermediary to allow the disparate and sometimes unpredictable behaviors of an XRCE Client implementation to interact with highly standardized and potentially multi-vendor DDS-RTPS implementations without complicating a DDS-RTPS application lifecycle; thus the name "Agent."

We now look at some internal features of XRCE Client and XRCE Agent interactions to better understand how this behavior is handled.

XRCE Agent: A DDS Proxy for XRCE Clients

In the figure below, we see a typical deployment concept where an XRCE Client would naturally be deployed in a very constrained environment with the XRCE Agent deployed into a more robust environment where it will employ full DDS-RTPS behavior.

Figure 5. XRCE Agent Enables DDS Facade to DDS-RTPS Capability
NOTE: Omitted from this figure is the supported QoS discussed later.

A simplified version of the overall object model for the XRCE Agent is added to Figure 5 above to show what subset of the DDS core capabilities the XRCE Agent supports in order to accomplish its goal of interoperating with DDS-RTPS applications.

An important aspect of the Agent comes with the ProxyClient role. Each XRCE Client connecting to an XRCE Agent will receive a ProxyClient instance that brokers the DDS-RTPS capabilities on its behalf, relieving the XRCE Client of this duty but benefitting from DDS functionality otherwise.

Because the XRCE Agent is within the scope of the DDS-XRCE standard specification, an Agent implemented to that standard can act as an XRCE Agent across multiple vendor implementations of XRCE Clients using the DDS-XRCE protocol. The Agent serves as an extension of DDS vendor interoperability. Device vendors that choose to employ an XRCE Client compliant implementation then benefit in being able to deploy into multi-vendor DDS ecosystems as well.

The bottom line is: The XRCE Agent is a very important piece to this puzzle.

Static or Dynamic DDS Application Creation

When developing an XRCE based system, a few questions may arise as to what defines the DDS functionality an XRCE Client can ultimately invoke in the DDS-RTPS ecosystem. The good news is that the DDS-XRCE specification allows for multiple ways to accomplish this, giving implementers the flexibility to address different architectural factors.

Here are some driving questions you may have:

1. Will my devices that contain XRCE Clients be in bounded facilities and/or in the hands of trusted individuals?
2. Does the role of any given XRCE Client need to be constrained to a limited subset of the full DDS application operating in the DDS-RTPS environment?
3. Does the existence of localized configuration on the XRCE Client impose unwelcome footprint pressure?
4. How likely is it that QoS and Security configuration of your application will remain constant in the lifespan of the system?
5. How maintainable are the XRCE Client environments from a centralized system/location?

Within the DDS-XRCE specification, the concept of an XRCE Object is used to describe the overall scope that an XRCE Client can/will use while communicating with an XRCE Agent and ultimately with the DDS-RTPS environment. This object is instantiated in the XRCE Agent for the purpose of scoping what the XRCE Client can do when making contact and referencing that XRCE Object as its "application" to use. So, an XRCE Object can be thought of as a facade to a DDS application in the DDS-RTPS environment to interact with.

An XRCE Object is codified in XML and, in particular, the DDS-XML schema standard that is part of the DDS family of standards. You can find details of DDS-XML here: https://www.omg.org/spec/DDS-XML/

An XRCE Agent uses a DDS-XML formatted configuration to configure a new XRCE Object that is subsequently registered in the XRCE Agent for use by XRCE Clients. The configuration can be read in at startup or sent remotely from a process using the DDS-XRCE protocol. This is where we can leverage multiple ways to define XRCE Client scope within the XRCE Agent to better fit our needed architecture strategy.

Figure 6. Static and Dynamic Configuration of an XRCE Agent

In the prior figure, the two methods are depicted starting with a simple file-based configuration method. The DDS-XML format used in this file allows for a full library of DDS application configurations to be loaded at once, so naturally configuration management of this detail is beneficial with this approach and useful for applications that can withstand periodic restarts of XRCE Agent processes.

The second approach uses a special kind of XRCE Client (XRCE ConfigurationClient) defined in the specification that is limited to create, update, delete, and get-info (read) calls on XRCE Objects. It is not allowed to make read or write calls on application data (e.g., Topic data).

Certainly this approach has its advantages in dynamic management of XRCE Agents using the DDS-XRCE protocol if your solution requirements demand it. This capability opens the possibility of centralized management facilities to manage and monitor a widely distributed fleet of XRCE Agents.

Deployment strategies of XRCE components is the subject of the next section.

Agent Discovery By XRCE Clients

Working further into our survey of the DDS-XRCE specification, one might be curious as to how discovery of XRCE Agents is accomplished from an XRCE Client.

First, just some facts:

1. Agent discovery may be limited by the type of transport being used to make the call. TCP and UDP unicast style calls, for instance, will require static information about Agents to find them. Multicast, however, may find multiple Agents, since they would be listening on a common port and base address.
2. Since multiple Agents can be discovered by a Client, the specification gives the ability to further discriminate to decide which the Client wants to use if only one is desired. Agents return a payload of properties about themselves that the Client can interrogate for making this decision.
3. A Client can be pre-configured with a TransportLocator object representing how to find an Agent on the network. This works well with well-controlled environments where networks are laid out with pre-defined topology and remain generally static.

Following is Figure 10 from the DDS-XRCE specification, which gives us a better visualization of the interactions taking place to discover Agents.

Figure 7. Targeted and Dynamic Agent Discovery

Our first two calls clearly show a GET_INFO call targeting each of XRCE Agent and XRCE Agent2, so this might be more representative of statically defined targets where the XRCE Client has been configured to do so. Multicast equivalent could achieve the same with a single call with each Agent responding.

The OBJECTID_AGENT parameter in the request is a well-known constant defined by the specification. The last 4 bits of that object id represent a defined object type of OBJK_AGENT, which tells each Agent receiving such a call to return its own Agent info and a status of STATUS_OK. The INFO( ) return from each Agent also contains the Agent_Representation, an object that contains a variety of properties about the XRCE version, vendor, and definable properties to give the flexibility to the XRCE Client to make decisions as to whether that Agent should be its ongoing target of client calls—especially helpful when more than one Agent could respond in larger network deployments.

This is a deeper dive into the internals of discovery, but the flexibility provided gives system designers more options to select discovery strategies based on driving system requirements where dynamic, static, or a combination will satisfy that demand.

Transport Abstraction

The DDS-XRCE standard design follows its older siblings in the DDS family by abstracting the network transport from the XRCE Protocol. This is an important differentiator with other small footprint protocols, such as MQTT and CoAP, where a single transport is expected. The DDS-XRCE messaging protocol imposes minimal requirements on the transport layer because it:

1. Does not require ordering
2. Does not require reliable delivery
3. Provides its own means of authentication
4. Performs its own data fragmentation and reassembly

As a result, most transport implementations can simply include the XRCE message as a payload without including additional envelopes.

Figure 8. Transport Abstraction for DDS-XRCE Solutions

A TransportLocator object in the specification model provides for mapping to transports that have been implemented by the XRCE solution provider, so this leaves a lot of architectural flexibility to adapt to the needs of a solution. The specification opens this to custom transports as well, so adding to this selection is available.

The XRCE target space would obviously make this attractive to use where transports such as ZigBee, Bluetooth, 6LoWPAN, or IEEE 802.15.4 might be needed.

Quality of Service (QoS)

When discussing QoS in the context of DDS-XRCE, the notion of QoS is applied between the XRCE Agent and the DDS-RTPS application space. XRCE Clients can, however, direct the XRCE Agent session instantiated on its behalf to use QoS behaviors in two ways:

1. When using the CREATE ( ) call to the Agent to "create" a DDS Application, it can pass a DDS Application configuration for the XRCE Agent to use (see Listing 1 for a sample).
2. Referring to an already instantiated DDS Application in the XRCE Agent that has had QoS applied to it at startup

In Figure 6, we see these two paths at work when a DDS Application needs to be started up and invoked on behalf of an XRCE Client. In Listing 1 at the end of this article, you will find a sample configuration file with which an XRCE Agent can configure itself and allow XRCE Clients to invoke proxied calls into and out of DDS-RTPS environments.

That sample listing is the now-famous DDS ShapesDemo application that is used to demonstrate interoperability among DDS vendor implementations. In that configuration is a section dedicated to QoS configuration. You will see it with the tag <qos_library name="MyQosLibrary">.

When DataReaders/Writers are defined for an application, they can refer to a member of this section to apply QoS to their instance of the DDS Participant (Reader/Writer). When an XRCE Client creates a new session with the XRCE Agent, the Agent will, of course, interact with the DDS-RTPS environment with those QoS policies enforced. This includes DDS Security QoS as well.

A complete discussion of DDS Security is outside the scope of this article, but next we discuss how XRCE Clients and Agents assure their secure communication.

Security in DDS-XRCE

DDS-XRCE transports were discussed earlier and, depending on the transport you choose in your solution, the security options will vary as well. DDS-XRCE does not impose security on the transport layer, so that is the responsibility of the implementor. Transport Layer Security (TLS) and Datagram Transport Layer Security (DTLS) are available for TCP and UDP respectively. Either TCP or UDP can also be used within a VPN connection if available.

Higher in the stack in the XRCE Protocol is where the specification applies further application level security. For each call in the protocol API, the ClientKey object is supplied on the call for the XRCE Client and XRCE Agent to use for authentication and subsequent authorization to invoke a call on the DDS Application instance in the Agent. The CREATE_CLIENT( ) and GET_INFO( ) calls documented in the API, however, do not require this object parameter. The ClientKey is supplied to the XRCE Agent by the XRCE Client with the CREATE_CLIENT( ) call.

The specification makes the following clear points about ClientKey that should help you see how you would prepare to use XRCE Clients:

1. The ClientKey is assigned to each client.
2. The ClientKey uniquely identifies the client to a particular agent.
3. The ClientKey is associated with a set of permissions for the client within the agent.
4. The ClientKey shall be considered secret. It must be configured both on the Client and in the Agent.
5. The creation and configuration are outside the scope of this specification.
6. The ClientKey shall not be interpreted.

So, immediately we see that based on #5 above, the way a ClientKey is created and/or configured initially is left to the vendor implementation of a DDS-XRCE solution. A ClientKey is, however, defined as typedef octet ClientKey[4] in the specification. This is certainly needed for interoperability purposes among interoperating vendor implementations.

When a CREATE_CLIENT( ) call is made from the XRCE Client to an Agent, a ClientKey is provided within the CLIENT_Representation structure given to the Agent. The Agent would create a new ProxyClient object on the XRCE Client's behalf and register that ClientKey with it. Subsequent calls would direct those messages to the appropriate ProxyClient for DDS-RTPS processing.

Security responsibilities for the XRCE Agent and its communication to/from XRCE Clients are also addressed in the specification as it relates to how the Agent is observing traffic with the XRCE Clients. Here is an excerpt:

The Agent shall maintain a counter on the number of times the STATUS_ERR_NOCLIENT was sent on an established connection, and once a certain threshold is crossed, it shall close the connection. The Agent may subsequently refuse or throttle new connections originating from the same client transport endpoint that was previously closed.

This additional protective measure to guard against malicious or misbehaving clients is an important architectural feature that specification-compliant DDS-XRCE implementations will bring to your solution.

Flexible XRCE Deployment Scenarios

Beyond the individual features of the DDS-XRCE capabilities, there are considerations made available to users of DDS-XRCE technology and how different deployment strategies can benefit design requirements. Following are a few of the more common deployment strategies.

One-way Feeder Systems

Common use of DDS-XRCE technology is applied to mass data ingestion where feeder clients are deployed for lightweight dissemination of small data payloads. XRCE Agents would be configured with DDS Applications that contain pertinent DDS publishers that push this data into more robust processing logic.

Consideration for where these XRCE Clients are deployed may help determine if XRCE Client/Agent pairs would need to be enabled with subscription functionality, for example, to supply updates or behavioral changes. Clients deployed in well-controlled facilities that are accessible by personnel and/or where changes are infrequent may not need such capability. Remotely deployed Clients might benefit from it, however, but the dominant purpose of these clients is a one-way data feed function.

Figure 9. Common DDS-XRCE One-Way Feeder Deployment

XRCE Clients Communicate with One Another

There may be times when deployed XRCE Clients need to communicate with each other in addition to upstream DDS-RTPS applications, so naturally XRCE Agent capabilities can be leveraged to do so. Two ways are immediately available.

• Option 1 – Shared XRCE Agent. Multiple XRCE Clients can connect to the same XRCE Agent and each will get its own ProxyClient. Since an Agent supports both the XRCE protocol and DDS-RTPS, and each XRCE Client communicates with its ProxyClient, Client 1 publishing and Client 2 subscribing provides a possible path between the two via DDS-RTPS, which has been instantiated in the Agent. Some implementations might choose to optimize this path rather than using DDS between the two ProxyClients.

Figure 10. Leveraging DDS-XRCE Agent for Client Brokering

• Option 2 – Multiple Clients with Dedicated Agents. In deployments with larger numbers of XRCE Clients, distributed XRCE Agents may be used. The same pattern can be leveraged as Option 1, but across Agents, as long as visibility and permissions of the DDS Applications with which XRCE Clients interact are configured in a coordinated manner.

Figure 11. Using DDS-RTPS Environment for XRCE Client Communication Across Agents

Redundant/Failover Agents

XRCE Clients are allowed to instantiate as many TransportLocator objects as necessary as long as they have available the endpoint data (host/port) for the connection to succeed. This can be used to the advantage of your solution for availability requirements.

Active/Standby configurations may be an option should you have the capacity to do so, but Active/Active may be another approach as long as each Agent is able to carry sufficient load should a failure occur.

Figure 12. Use the Power of Multiple Transport Locators for Redundancy

Federated Agents

As deployments need to be more dispersed, either for network design reasons or from physical geographic requirements, the DDS-XRCE specification provides for the ability of implementations to use XRCE Agents in a very flexible way. An Agent is both an XRCE Client and XRCE Agent due to the fact that the XRCE message protocol and associated transport model is part of its design.

The topology below shows Agents 1 and 3 acting as intermediary “gateway” Agents that would be common in deployments demanding remote geographically dispersed environments (devices). Sensor and gateway suites are commonly used for such use cases, with this way of configuring a DDS-XRCE solution meeting many of the resulting architectural requirements.

An extension of the topology below would be to combine it with redundancy features like that shown in Figure 12, such that Agent 1 and Agent 3 can act as Active/Active failover for each other and their XRCE Client connections if needed.

Figure 13. Extend DDS-XRCE Reach with Federated Agents

Future Considerations

This article has given a survey of the DDS-XRCE specification, a recent addition to the growing family of DDS specifications. It is meant to inspire your thinking toward how you might benefit from such capability in your growing demand for distributed solutions in a very connected world, where you want those solutions to come with reliability, security, and the sound application of architectural qualities that durable solutions demand.

Possible future articles will demonstrate XRCE capability with sample applications and associated discussion. OCI always stands ready to help you pursue your complex target solutions with this and other distributed technologies, so don't hesitate to reach out to us. Also, if you have an idea of what you would like to see regarding this series of articles on DDS-XRCE, let us know that too.

<?xml version="1.0" encoding="UTF-8"?>
<dds xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xmlns="http://www.omg.org/dds"
     xsi:schemaLocation="http://www.omg.org/spec/DDS-XML/20170301/ddsxml_system_example.xsd">
  <types>
    <module name="ShapesDemoTypes" >
      <const name="MAX_COLOR_LEN" type="int32" value="128" />
      <struct name="ShapeType">
        <member name="color" key="true" type="string"
                stringMaxLength="MAX_COLOR_LEN" />
        <member name="x" type="int32" />
        <member name="y" type="int32" />
        <member name="shapesize" type="int32" />
      </struct>
    </module>
  </types>
  <qos_library name="MyQosLibrary">
    <qos_profile name="MyQosProfile">
      <datareader_qos>
        <durability>
          <kind>TRANSIENT_LOCAL_DURABILITY_QOS</kind>
        </durability>
        <reliability>
          <kind>RELIABLE_RELIABILITY_QOS</kind>
        </reliability>
        <history>
          <kind>KEEP_LAST_HISTORY_QOS</kind>
          <depth>6</depth>
        </history>
      </datareader_qos>
      <datawriter_qos>
        <durability>
          <kind>TRANSIENT_LOCAL_DURABILITY_QOS</kind>
        </durability>
        <reliability>
          <kind>RELIABLE_RELIABILITY_QOS</kind>
        </reliability>
        <history>
          <kind>KEEP_LAST_HISTORY_QOS</kind>
          <depth>20</depth>
        </history>
        <lifespan>
          <duration>
            <sec>10</sec>
            <nanosec>0</nanosec>
          </duration>
        </lifespan>
      </datawriter_qos>
    </qos_profile>
  </qos_library>
  <application_library name="MyApplications">
    <application name="ShapesDemoApp">
      <domain_participant name="MyParticipant"
                          domain_ref="ShapesDomainLibrary::ShapesDomain">
        <register_type name="ShapeType" type_ref="ShapeType" />
        <topic name="Square" register_type_ref="ShapeType" />
        <topic name="Circle" register_type_ref="ShapeType" />
        <topic name="Triangle" register_type_ref="ShapeType" />
        <publisher name="MyPublisher">
          <data_writer name="MySquareWriter" topic_ref="Square">
            <datawriter_qos base_name="MyQosLibrary::MyQosProfile"/>
          </data_writer>
          <data_writer name="MyCircleWriter" topic_ref="Circle" />
        </publisher>
        <subscriber name="MySubscriber">
          <data_reader name="MyTriangleRdr" topic_ref="Triangle">
            <datareader_qos base_name="MyQosLibrary::MyQosProfile"/>
          </data_reader>
        </subscriber>
      </domain_participant>
    </application>
  </application_library>
</dds>