Bringing Multicast to the Cloud for Interoperable DDS Applications

Bringing Multicast to the Cloud for Interoperable DDS Applications

By Justin Wilson, OCI Principal Software Engineer

March 2019

Introduction

When the Object Management Group (OMG) released the Data Distribution Specification (DDS) in 2004, the goal of the DDS specification was to define the concepts, interfaces, programming language mappings, etc. necessary for developing distributed applications based on a shared cache model.

Recognizing a need for interoperability among DDS vendors, the OMG released the DDS Interoperability Wire Protocol Specification, which defines the Real-Time Publish Subscribe (RTPS) protocol, in 2008.

Like any specification, standard, or system, DDS and RTPS were designed with certain goals and limitations in mind. OpenDDS, as an implementation of DDS and RTPS, is subject to the same limitations. The authors of these specifications could not have foreseen the rise of cloud computing and the (Industrial) Internet of Things.

To build truly interoperable and standards-based applications, we desire the ability to use RTPS in environments like the cloud and public Internet. The overall goal of RTPS is to distribute samples produced by writers to all associated readers. But before writers exchange data with their associated readers, the readers and writers must discover each other.

In RTPS terminology, a reader or writer is called an endpoint, and a collection of endpoints is called a participant. A locator is an IP address and port. Each endpoint is associated with one or more locators.

RTPS accomplishes discovery in two steps: participant discovery and endpoint discovery.

Participants discover each other using the Simple Participant Discovery Protocol (SPDP).

SPDP messages contain endpoints corresponding to built-in readers and writers that RTPS uses for endpoint discovery. Once a participant is discovered, the built-in readers and writers associate and exchange data about the application-defined endpoints.

The protocol that governs the exchange of application-defined endpoints is called the Simple Endpoint Discovery Protocol (SEDP).

SEDP can use unicast without any difficulty, while SPDP relies upon a mechanism like multicast (or broadcast) for periodic announcements.

Many virtual private cloud networks, including Google Cloud Platform, Microsoft Azure, Amazon Web Services, and the public Internet do not support multicast. Thus, SPDP via multicast cannot be used in these environments.

Both SPDP and SEDP messages contain IP addresses and ports in the form of locators. If an endpoint is behind a firewall or router that is performing network address translation (NAT), which is common in many home, office, and industrial LANs, its locators become useless.

Furthermore, configuring a firewall to support RTPS is a daunting task, given the way ports are assigned. Due to these difficulties, RTPS cannot be used in these environments.

In this article, we address the challenge of running OpenDDS in a cloud environment. Future articles will address the challenge of running OpenDDS over the public Internet.

While the focus of this article is the difficulties of using RTPS in the cloud, similar problems occur for other protocols and applications that rely on multicast. The solution we present below is not tied to RTPS and DDS in any way and can be used for other applications.

The Multicast Repeater

The main problem to overcome when running OpenDDS in a cloud environment is facilitating the discovery of other participants. If a participant can discover the other participants, it can use unicast messages to communicate. Thus, our solution focuses on providing the same functionality as SPDP without using multicast.

The first observation driving our solution is that multicast is often supported internally on a virtual machine (VM) running the cloud. That is, the host operating system will most likely support multicast, even though the underlying virtual network may not.

Thus, we can imagine a daemon running in the VM that repeats the SPDP multicast traffic by forwarding it (via unicast) to a set of hosts. A daemon receiving an SPDP message on its unicast port repeats the message for the local multicast group. This design has the advantage of being completely independent of DDS and RTPS.

The next issue is forming the list of hosts to which a multicast repeater forwards messages. A good starting point is a static list of hosts provided via the command-line or a configuration file.

A more dynamic implementation may include an API that can be used to manipulate the list at runtime. To support OpenDDS in the cloud, we focused on populating the list of hosts from the information exposed via API for various cloud providers. That is, the multicast repeater periodically queries a cloud API to determine the set of hosts that comprise the virtual multicast group.

A multicast repeater that periodically queries a cloud API to determine the set of hosts that comprise the virtual multicast group.

The preceding figure shows a deployment consisting of two applications using multicast repeaters.

  1. The application on the left sends an SPDP message via multicast to its local repeater (1).
  2. The repeater on the left sends the SPDP message via unicast to its peer repeater on the right (2).
  3. Finally, the repeater on the right sends the SPDP message to its local application via multicast (3).

The list of peer repeaters is determined by polling a Cloud API.

Tutorials

This section contains tutorials that demonstrate how to run a simple OpenDDS application that uses RTPS for discovery on various cloud platforms. A publisher will run in one VM, while the subscriber will run in a different VM.

The tutorials are written using the bash scripting language. For expediency, we will use a Docker image with a pre-built OpenDDS application and multicast repeater.

The different tutorials assume a basic familiarity with docker and the CLI tools (gcloud, az, and aws) for each cloud environment, respectively. Furthermore, the account used to invoke the cloud APIs must have sufficient privilege to carry out the operations described.

Google Cloud Platform (GCP)

  1. Create two terminals; one for the publisher and one for the subscriber.
  2. In both terminals, define your project id and number.
        PROJECT_ID=xxxxxxxxx
        PROJECT_NUMBER=XXXXXXXXXXXX
  1. In either terminal
    • create an instance group
            gcloud compute --project=${PROJECT_ID} instance-groups unmanaged create opendds-multicast-repeater-test-group --zone=us-central1-c
    • create VMs for the publisher and subscriber
            gcloud beta compute --project=${PROJECT_ID} instances create publisher --zone=us-central1-c --machine-type=n1-standard-1 --subnet=default --network-tier=PREMIUM --maintenance-policy=MIGRATE --service-account=${PROJECT_NUMBER}-compute@developer.gserviceaccount.com --scopes=https://www.googleapis.com/auth/compute.readonly,https://www.googleapis.com/auth/servicecontrol,https://www.googleapis.com/auth/service.management.readonly,https://www.googleapis.com/auth/logging.write,https://www.googleapis.com/auth/monitoring.write,https://www.googleapis.com/auth/trace.append,https://www.googleapis.com/auth/devstorage.read_only --image=cos-stable-71-11151-71-0 --image-project=cos-cloud --boot-disk-size=100GB --boot-disk-type=pd-standard --boot-disk-device-name=publisher
            gcloud beta compute --project=${PROJECT_ID} instances create subscriber --zone=us-central1-c --machine-type=n1-standard-1 --subnet=default --network-tier=PREMIUM --maintenance-policy=MIGRATE --service-account=${PROJECT_NUMBER}-compute@developer.gserviceaccount.com --scopes=https://www.googleapis.com/auth/compute.readonly,https://www.googleapis.com/auth/servicecontrol,https://www.googleapis.com/auth/service.management.readonly,https://www.googleapis.com/auth/logging.write,https://www.googleapis.com/auth/monitoring.write,https://www.googleapis.com/auth/trace.append,https://www.googleapis.com/auth/devstorage.read_only --image=cos-stable-71-11151-71-0 --image-project=cos-cloud --boot-disk-size=100GB --boot-disk-type=pd-standard --boot-disk-device-name=subscriber
    • add the publisher and subscriber VMs to the instance group
           gcloud compute --project=${PROJECT_ID} instance-groups unmanaged add-instances opendds-multicast-repeater-test-group --instances=publisher,subscriber --zone=us-central1-c
  1. In the publisher terminal
    • SSH into the publisher VM
            gcloud compute --project=${PROJECT_ID} ssh --zone=us-central1-c publisher
    • reset the firewall
            sudo iptables -P INPUT ACCEPT; sudo iptables -P OUTPUT ACCEPT; sudo iptables -P FORWARD ACCEPT; sudo iptables -F
    • start the multicast repeater
            docker run -d --name=repeater --rm --net=host objectcomputing/opendds:repeater /opt/OpenDDS/tools/repeater/repeater.js --group 239.255.0.1:8400 --uport 5000 --gcp us-central1-c:opendds-multicast-repeater-test-group
    • start the publisher
            docker run --name=publisher --rm --net=host -w /opt/OpenDDS/tests/DCPS/Messenger -e LD_LIBRARY_PATH=. objectcomputing/opendds:repeater publisher -DCPSConfigFile rtps.ini
 
  1. In the subscriber terminal
    • SSH into the subscriber VM
           gcloud compute --project=${PROJECT_ID} ssh --zone=us-central1-c subscriber
    • reset the firewall
           sudo iptables -P INPUT ACCEPT; sudo iptables -P OUTPUT ACCEPT; sudo iptables -P FORWARD ACCEPT; sudo iptables -F
    • start the multicast repeater
            docker run -d --name=repeater --rm --net=host objectcomputing/opendds:repeater /opt/OpenDDS/tools/repeater/repeater.js --group 239.255.0.1:8400 --uport 5000 --gcp us-central1-c:opendds-multicast-repeater-test-group
    • start the subscriber
            docker run --name=subscriber --rm --net=host -w /opt/OpenDDS/tests/DCPS/Messenger -e LD_LIBRARY_PATH=. objectcomputing/opendds:repeater subscriber -DCPSConfigFile rtps.ini
  1. Verify the expected output
    • publisher
            Starting publisher
            Starting publisher with 1 args
            Reliable DataWriter
            Creating Writer
            Starting Writer
            Writer finished
            Writer wait small time
            deleting DW
            deleting contained entities
            deleting participant
            shutdown
 
    • subscriber
            Transport is RELIABLE
            Reliable DataReader
            /opt/OpenDDS/tests/DCPS/Messenger/DataReaderListener.cpp:146: INFO: on_subscription_matched()
            /opt/OpenDDS/tests/DCPS/Messenger/DataReaderListener.cpp:139: INFO: on_liveliness_changed()
            SampleInfo.sample_rank = 0
            SampleInfo.instance_state = 1
            Message: subject    = Review
                    subject_id = 99
                    from       = Comic Book Guy
                    count      = 0
                    text       = Worst. Movie. Ever.
 
            ...
 
            SampleInfo.sample_rank = 0
            SampleInfo.instance_state = 1
            Message: subject    = Review
                    subject_id = 99
                    from       = Comic Book Guy
                    count      = 39
                    text       = Worst. Movie. Ever.
            SampleInfo.sample_rank = 0
            SampleInfo.instance_state = 2
            /opt/OpenDDS/tests/DCPS/Messenger/DataReaderListener.cpp:94: INFO: instance is disposed
            /opt/OpenDDS/tests/DCPS/Messenger/DataReaderListener.cpp:139: INFO: on_liveliness_changed()
            /opt/OpenDDS/tests/DCPS/Messenger/DataReaderListener.cpp:146: INFO: on_subscription_matched()
  1. In each terminal, close the SSH session.
        exit
  1. Clean up the cloud resources.
        gcloud compute instances delete publisher subscriber --zone=us-central1-c
        gcloud compute --project=${PROJECT_ID} instance-groups unmanaged delete opendds-multicast-repeater-test-group --zone=us-central1-c

Azure

  1. Create two cloud shells (if running in the browser) or two terminals with Azure CLI tools in the PATH (if running locally). One is for the publisher and the other is for the subscriber.
  2. In both terminals, define your subscription id and path to a public SSH key. When running in a cloud shell openssh should be available. Generate a key by running ssh-keygen -t rsa -b 2048 and answering the prompts. The subscription ID can be found by searching for subscriptions in the Azure portal search bar.
        AZURE_SUBSCRIPTION_ID=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
        KEY_PATH=/PATH/TO/PUBLIC_KEY
  1. In either terminal, create a resource group.
        az group create --subscription=${AZURE_SUBSCRIPTION_ID} --location=westus2 --name=opendds-multicast-repeater-test-group
 
  1. In the publisher terminal
    • create the publisher VM and note its public IP address from the JSON output "publicIpAddress": "XXX.XXX.XXX.XXX"
            az vm create --subscription=${AZURE_SUBSCRIPTION_ID} --location=westus2 --name=publisher --resource-group=opendds-multicast-repeater-test-group --image=UbuntuLTS --size=Standard_B1s --assign-identity --role=Reader --scope /subscriptions/${AZURE_SUBSCRIPTION_ID}/resourceGroups/opendds-multicast-repeater-test-group --ssh-key-value=@${KEY_PATH}
            PUBLISHER_IP=XXX.XXX.XXX.XXX
    • SSH into the publisher VM; if a custom SSH key location was used (e.g., not ~/.ssh/id_rsa), then this needs to be provided using ssh -i /path/to/id_rsa (assuming an RSA key was created using the ssh-keygen command above)
            ssh $PUBLISHER_IP
    • install Docker
            sudo apt-get update -y
            sudo apt-get install docker.io
    • reset the firewall
            sudo iptables -P INPUT ACCEPT; sudo iptables -P OUTPUT ACCEPT; sudo iptables -P FORWARD ACCEPT; sudo iptables -F
    • start the multicast repeater
            AZURE_SUBSCRIPTION_ID=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
            sudo docker run -d --name=repeater --rm -e AZURE_SUBSCRIPTION_ID=${AZURE_SUBSCRIPTION_ID} --net=host objectcomputing/opendds:repeater /opt/OpenDDS/tools/repeater/repeater.js --group 239.255.0.1:8400 --uport 5000 --azure opendds-multicast-repeater-test-group
    • start the publisher
            sudo docker run --name=publisher --rm --net=host -w /opt/OpenDDS/tests/DCPS/Messenger -e LD_LIBRARY_PATH=. objectcomputing/opendds:repeater publisher -DCPSConfigFile rtps.ini
  1. In the subscriber terminal
    • create the subscriber VM and note its public IP address from the JSON output
            az vm create --subscription=${AZURE_SUBSCRIPTION_ID} --location=westus2 --name=subscriber --resource-group=opendds-multicast-repeater-test-group --image=UbuntuLTS --size=Standard_B1s --assign-identity --role=Reader --scope /subscriptions/${AZURE_SUBSCRIPTION_ID}/resourceGroups/opendds-multicast-repeater-test-group --ssh-key-value=@${KEY_PATH}
            SUBSCRIBER_IP=XXX.XXX.XXX.XXX
    • SSH into the subscriber VM in the same way as the publisher (see the note above)
            ssh $SUBSCRIBER_IP
    • install Docker
            sudo apt-get update -y
            sudo apt-get install docker.io
    • reset the firewall
            sudo iptables -P INPUT ACCEPT; sudo iptables -P OUTPUT ACCEPT; sudo iptables -P FORWARD ACCEPT; sudo iptables -F
    • start the multicast repeater
            AZURE_SUBSCRIPTION_ID=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
            sudo docker run -d --name=repeater --rm -e AZURE_SUBSCRIPTION_ID=${AZURE_SUBSCRIPTION_ID} --net=host objectcomputing/opendds:repeater /opt/OpenDDS/tools/repeater/repeater.js --group 239.255.0.1:8400 --uport 5000 --azure opendds-multicast-repeater-test-group
    • start the subscriber
            sudo docker run --name=subscriber --rm --net=host -w /opt/OpenDDS/tests/DCPS/Messenger -e LD_LIBRARY_PATH=. objectcomputing/opendds:repeater subscriber -DCPSConfigFile rtps.ini
 
  1. Verify the expected output
    • publisher
            Starting publisher
            Starting publisher with 1 args
            Reliable DataWriter
            Creating Writer
            Starting Writer
            Writer finished
            Writer wait small time
            deleting DW
            deleting contained entities
            deleting participant
            shutdown
 
    • subscriber
            Transport is RELIABLE
            Reliable DataReader
            /opt/OpenDDS/tests/DCPS/Messenger/DataReaderListener.cpp:146: INFO: on_subscription_matched()
            /opt/OpenDDS/tests/DCPS/Messenger/DataReaderListener.cpp:139: INFO: on_liveliness_changed()
            SampleInfo.sample_rank = 0
            SampleInfo.instance_state = 1
            Message: subject    = Review
                    subject_id = 99
                    from       = Comic Book Guy
                    count      = 0
                    text       = Worst. Movie. Ever.
 
            ...
 
            SampleInfo.sample_rank = 0
            SampleInfo.instance_state = 1
            Message: subject    = Review
                    subject_id = 99
                    from       = Comic Book Guy
                    count      = 39
                    text       = Worst. Movie. Ever.
            SampleInfo.sample_rank = 0
            SampleInfo.instance_state = 2
            /opt/OpenDDS/tests/DCPS/Messenger/DataReaderListener.cpp:94: INFO: instance is disposed
            /opt/OpenDDS/tests/DCPS/Messenger/DataReaderListener.cpp:139: INFO: on_liveliness_changed()
            /opt/OpenDDS/tests/DCPS/Messenger/DataReaderListener.cpp:146: INFO: on_subscription_matched()
  1. In each terminal, close the SSH session.
        exit
  1. Clean up the cloud resources.
        az vm delete --subscription=${AZURE_SUBSCRIPTION_ID} --name=publisher --resource-group=opendds-multicast-repeater-test-group
        az vm delete --subscription=${AZURE_SUBSCRIPTION_ID} --name=subscriber --resource-group=opendds-multicast-repeater-test-group
        az group delete --subscription=${AZURE_SUBSCRIPTION_ID} --name=opendds-multicast-repeater-test-group

Amazon Web Services (AWS)

Before starting this tutorial, determine the default region in which VMs will be created. That region will be used when starting the repeater.

  1. Create two terminals; one for the publisher and one for the subscriber.
  2. In either terminal
    • create a security group
            aws ec2 create-security-group --group-name opendds-sg --description "OpenDDS Multicast Repeater Demo" --query GroupId
            SG_ID=sg-XXXXXXXXXXXXXXXXX
 
    • allow SSH access and UDP messages in the security group
            aws ec2 authorize-security-group-ingress --group-id $SG_ID --protocol tcp --port 22 --cidr 0.0.0.0/0
            aws ec2 authorize-security-group-ingress --group-id $SG_ID --protocol udp --port 0-65535 --source-group $SG_ID
    • create a key pair
            aws ec2 create-key-pair --key-name OpenDDSKeyPair --query KeyMaterial --output text > OpenDDSPrivateKey.pem
            chmod 400 OpenDDSPrivateKey.pem
    • describe the subnets and pick one
            aws ec2 describe-subnets --query Subnets[0].SubnetId
            SUBNET_ID=subnet-XXXXXXXX
    • create ec2-role-trust-policy.json with the following content
            {
              "Version": "2012-10-17",
              "Statement": [
                {
                  "Effect": "Allow",
                  "Principal": { "Service": "ec2.amazonaws.com"},
                  "Action": "sts:AssumeRole"
                }
              ]
            }
    • create ec2-role-access-policy.json with the following content
            {
              "Version": "2012-10-17",
              "Statement": [
                {
                  "Effect": "Allow",
                  "Action": ["ec2:DescribeInstances"],
                  "Resource": ["*"]
                }
              ]
            }
    • create an IAM instance profile
            aws iam create-role --role-name OpenDDS --assume-role-policy-document file://ec2-role-trust-policy.json
            aws iam put-role-policy --role-name OpenDDS --policy-name EC2-Permissions --policy-document file://ec2-role-access-policy.json
            aws iam create-instance-profile --instance-profile-name OpenDDS-ec2access-profile
            aws iam add-role-to-instance-profile --instance-profile-name OpenDDS-ec2access-profile --role-name OpenDDS
 
    • create the publisher, tag it with opendds, and get its public IP address
            aws ec2 run-instances --image-id ami-0f65671a86f061fcd --count 1 --instance-type t2.micro --key-name OpenDDSKeyPair --security-group-ids $SG_ID --iam-instance-profile Name=OpenDDS-ec2access-profile --subnet-id $SUBNET_ID --query Instances[0].InstanceId
            PUBLISHER_ID=i-XXXXXXXXXXXXXXXXX
            aws ec2 create-tags --resources $PUBLISHER_ID --tags Key=opendds,Value=
            aws ec2 describe-instances --instance-ids $PUBLISHER_ID --query Reservations[0].Instances[0].PublicIpAddress
 
    • create the subscriber, tag it with opendds, and get its public IP address
            aws ec2 run-instances --image-id ami-0f65671a86f061fcd --count 1 --instance-type t2.micro --key-name OpenDDSKeyPair --security-group-ids $SG_ID --iam-instance-profile Name=OpenDDS-ec2access-profile --subnet-id $SUBNET_ID --query Instances[0].InstanceId
            SUBSCRIBER_ID=i-XXXXXXXXXXXXXXXXX
            aws ec2 create-tags --resources $SUBSCRIBER_ID --tags Key=opendds,Value=
            aws ec2 describe-instances --instance-ids $SUBSCRIBER_ID --query Reservations[0].Instances[0].PublicIpAddress
  1. In the publisher terminal
    • SSH into the publisher
            ssh -i OpenDDSPrivateKey.pem ubuntu@XXX.XXX.XXX.XXX
    • install Docker
            sudo apt update
            sudo apt install docker.io
    • reset the firewall
            sudo iptables -P INPUT ACCEPT; sudo iptables -P OUTPUT ACCEPT; sudo iptables -P FORWARD ACCEPT; sudo iptables -F
    • start the multicast repeater
            AWS_REGION=XX-XXXX-X
            sudo docker run -d --name=repeater --rm -e AWS_REGION=$AWS_REGION --net=host objectcomputing/opendds:repeater /opt/OpenDDS/tools/repeater/repeater.js --group 239.255.0.1:8400 --uport 5000 --aws opendds
 
    • start the publisher
            sudo docker run --name=publisher --rm --net=host -w /opt/OpenDDS/tests/DCPS/Messenger -e LD_LIBRARY_PATH=. objectcomputing/opendds:repeater publisher -DCPSConfigFile rtps.ini
  1. In the subscriber terminal
    • SSH into the subscriber VM
            ssh -i OpenDDSKeyPair.pem ubuntu@XXX.XXX.XXX.XXX
    • install Docker
            sudo apt update
            sudo apt install docker.io
    • reset the firewall
            sudo iptables -P INPUT ACCEPT; sudo iptables -P OUTPUT ACCEPT; sudo iptables -P FORWARD ACCEPT; sudo iptables -F
    • start the multicast repeater
            AWS_REGION=XX-XXXX-X
            sudo docker run -d --name=repeater --rm -e AWS_REGION=$AWS_REGION --net=host objectcomputing/opendds:repeater /opt/OpenDDS/tools/repeater/repeater.js --group 239.255.0.1:8400 --uport 5000 --aws opendds
    • start the subscriber
            sudo docker run --name=subscriber --rm --net=host -w /opt/OpenDDS/tests/DCPS/Messenger -e LD_LIBRARY_PATH=. objectcomputing/opendds:repeater subscriber -DCPSConfigFile rtps.ini
  1. Verify the expected output
    • publisher
            Starting publisher
            Starting publisher with 1 args
            Reliable DataWriter
            Creating Writer
            Starting Writer
            Writer finished
            Writer wait small time
            deleting DW
            deleting contained entities
            deleting participant
            shutdown
 
    • subscriber
            Transport is RELIABLE
            Reliable DataReader
            /opt/OpenDDS/tests/DCPS/Messenger/DataReaderListener.cpp:146: INFO: on_subscription_matched()
            /opt/OpenDDS/tests/DCPS/Messenger/DataReaderListener.cpp:139: INFO: on_liveliness_changed()
            SampleInfo.sample_rank = 0
            SampleInfo.instance_state = 1
            Message: subject    = Review
                    subject_id = 99
                    from       = Comic Book Guy
                    count      = 0
                    text       = Worst. Movie. Ever.
 
            ...
 
            SampleInfo.sample_rank = 0
            SampleInfo.instance_state = 1
            Message: subject    = Review
                    subject_id = 99
                    from       = Comic Book Guy
                    count      = 39
                    text       = Worst. Movie. Ever.
            SampleInfo.sample_rank = 0
            SampleInfo.instance_state = 2
            /opt/OpenDDS/tests/DCPS/Messenger/DataReaderListener.cpp:94: INFO: instance is disposed
            /opt/OpenDDS/tests/DCPS/Messenger/DataReaderListener.cpp:139: INFO: on_liveliness_changed()
            /opt/OpenDDS/tests/DCPS/Messenger/DataReaderListener.cpp:146: INFO: on_subscription_matched()
  1. In each terminal, close the SSH session.
        exit
  1. Clean up the cloud resources.
        aws ec2 terminate-instances --instance-ids $PUBLISHER_ID $SUBSCRIBER_ID
        aws iam remove-role-from-instance-profile --instance-profile-name OpenDDS-ec2access-profile --role-name OpenDDS
        aws iam delete-instance-profile --instance-profile-name OpenDDS-ec2access-profile
        aws iam delete-role-policy --role-name OpenDDS --policy-name EC2-Permissions
        aws iam delete-role --role-name OpenDDS
        aws ec2 delete-key-pair --key-name OpenDDSKeyPair
        aws ec2 delete-security-group --group-name opendds-sg

Discussion

The Docker containers in the tutorials are started with --net=host, which causes the containers to use the host's network stack. Consequently, the locators determined by the publisher and subscriber are not subject to the network address translation performed by Docker.

This is similar to the network address translation problem introduced by firewalls mentioned in the introduction. Since the publisher and subscriber are using the host's network stack, it is easiest to use the host's network stack for the repeater as well.

The multicast group and port used by the repeater correspond to the default SPDP multicast group and port. The port number can be altered via different configuration parameters. The RTPS specification defines the procedure for computing the port number from these parameters.

In any case, the multicast group and port used by the repeater must match the group and port used by the local DDS participant(s). The multicast group and port are specified with the --group option, e.g., --group 239.255.0.1:8400. The unicast port used by the repeater is specified with the --uport option.

The publisher and subscriber use the same configuration file (rtps.ini).

    [common]
    DCPSGlobalTransportConfig=$file
 
    [domain/4]
    DiscoveryConfig=rtps
 
    [rtps_discovery/rtps]
    SedpMulticast=0
 
    [transport/the_rtps_transport]
    transport_type=rtps_udp
    use_multicast=0
 

In this configuration, multicast is disabled for SEDP and the data transport. This causes OpenDDS to use unicast, which is supported in the virtual networks, for these messages. If one desired to use multicast for SEDP and the data, then one would need to run two additional repeaters.

As illustrated by the tutorials, support for different cloud providers can be activated by passing an argument (--gcp, --aws, or --azure) to the repeater.

The AWS plugin requires the AWS_REGION environment variable to be set, and the Azure plugin requires the AZURE_SUBSCRIPTION_ID environment variable to be set.

The plugins for the different cloud providers show two different approaches to gathering a list of forwarding addresses for the repeater. The GCP and Azure plugin attempt to get the IP addresses of all of the VMs in the group indicated by the --gcp or --azure argument, respectively, while the AWS plugin attempts to get the IP addresses of all of the VMs bearing the tag indicated by the --aws argument. (We could have used the security group concept in AWS, but we wanted to experiment with tags.) The tagging approach is more flexible, and we anticipate converting the GCP and Azure plugins to use tags in the future.

One limitation of the cloud plugins for the repeater is the assumption that all hosts that form the virtual multicast group are in the same region. This is built into many of the cloud APIs that assume a default region when performing certain operations like querying for VMs. The plugins may be improved in the future to overcome this limitation.

An alternate approach is to use a combination of static and dynamic configurations where each region has a well-known repeater that is statically configured by the other well-known repeaters. In this approach, region-to-region traffic always passes through the well-known repeaters.

The multicast repeater is not the only solution for enabling multicast in environments that don't support it. For example, this solution uses VPN technology to build an overlay network that supports multicast. (Before selecting a solution for enabling multicast, we recommend making sure you review and understand the security implications of that solution.)

Conclusion

Modern environments like the cloud and modern techniques like IoT and IIoT challenge some of the assumptions made by the designers of DDS, RTPS, and implementations of the same. In this article, we looked at the problem of running OpenDDS with RTPS in the cloud, where multicast is not supported.

The solution was to introduce an external process that repeats participant discovery (SPDP) multicast traffic to other OpenDDS participants using unicast. In a future article, we will look at an IoT scenario in which OpenDDS is run with RTPS over the public Internet.

Software Engineering Tech Trends (SETT) is a regular publication featuring emerging trends in software engineering.

Related Articles

Distributing Data Securely with OpenDDS

Interoperable Internet-Enabled DDS Applications

OMG DDS for eXtremely Resource-Constrained Environments (DDS-XRCE)

Using Reliable Multicast for Data Distribution with OpenDDS