Not receiving /fmu/out topics on pixhawk (uXRCE-DDS)

PX4 Autopilot
1

Hi all,

I’m having some trouble getting my ROS2 PX4 topics visualised on my Raspberry Pi 4. The RPi acts as the agent and should subscribe to topics published by my Pixhawk 4 (FMUv5). They are connected via a UART connection, which seems to initialise properly as shown below. No datawriter is created which seems odd.

MicroXRCEAgent serial --dev /dev/ttyAMA0 -b 921600
[1763144323.619658] info     | TermiosAgentLinux.cpp | init                     | running…             | fd: 3
[1763144323.620398] info     | Root.cpp           | set_verbose_level        | logger setup           | verbose_level: 4
[1763144323.781953] info     | Root.cpp           | create_client            | create                 | client_key: 0x00000001, session_id: 0x81
[1763144323.782134] info     | SessionManager.hpp | establish_session        | session established    | client_key: 0x00000001, address: 1
[1763144323.902477] info     | ProxyClient.cpp    | create_participant       | participant created    | client_key: 0x00000001, participant_id: 0x001(1)
[1763144324.006475] info     | ProxyClient.cpp    | create_topic             | topic created          | client_key: 0x00000001, topic_id: 0x800(2), participant_id: 0x001(1)
[1763144324.006991] info     | ProxyClient.cpp    | create_subscriber        | subscriber created     | client_key: 0x00000001, subscriber_id: 0x800(4), participant_id: 0x001(1)
[1763144324.017953] info     | ProxyClient.cpp    | create_datareader        | datareader created     | client_key: 0x00000001, datareader_id: 0x800(6), subscriber_id: 0x800(4)
[1763144324.123388] info     | ProxyClient.cpp    | create_topic             | topic created          | client_key: 0x00000001, topic_id: 0x801(2), participant_id: 0x001(1)
[1763144324.123697] info     | ProxyClient.cpp    | create_subscriber        | subscriber created     | client_key: 0x00000001, subscriber_id: 0x801(4), participant_id: 0x001(1)
[1763144324.126135] info     | ProxyClient.cpp    | create_datareader        | datareader created     | client_key: 0x00000001, datareader_id: 0x801(6), subscriber_id: 0x801(4)
[1763144324.129711] info     | ProxyClient.cpp    | create_topic             | topic created          | client_key: 0x00000001, topic_id: 0x802(2), participant_id: 0x001(1)
[1763144324.129934] info     | ProxyClient.cpp    | create_subscriber        | subscriber created     | client_key: 0x00000001, subscriber_id: 0x802(4), participant_id: 0x001(1)
[1763144324.131839] info     | ProxyClient.cpp    | create_datareader        | datareader created     | client_key: 0x00000001, datareader_id: 0x802(6), subscriber_id: 0x802(4)
[1763144324.237123] info     | ProxyClient.cpp    | create_topic             | topic created          | client_key: 0x00000001, topic_id: 0x803(2), participant_id: 0x001(1)
[1763144324.237412] info     | ProxyClient.cpp    | create_subscriber        | subscriber created     | client_key: 0x00000001, subscriber_id: 0x803(4), participant_id: 0x001(1)
[1763144324.239826] info     | ProxyClient.cpp    | create_datareader        | datareader created     | client_key: 0x00000001, datareader_id: 0x803(6), subscriber_id: 0x803(4)
[1763144324.343571] info     | ProxyClient.cpp    | create_topic             | topic created          | client_key: 0x00000001, topic_id: 0x804(2), participant_id: 0x001(1)
[1763144324.343771] info     | ProxyClient.cpp    | create_subscriber        | subscriber created     | client_key: 0x00000001, subscriber_id: 0x804(4), participant_id: 0x001(1)
[1763144324.345889] info     | ProxyClient.cpp    | create_datareader        | datareader created     | client_key: 0x00000001, datareader_id: 0x804(6), subscriber_id: 0x804(4)
[1763144324.449775] info     | ProxyClient.cpp    | create_topic             | topic created          | client_key: 0x00000001, topic_id: 0x805(2), participant_id: 0x001(1)
[1763144324.450152] info     | ProxyClient.cpp    | create_subscriber        | subscriber created     | client_key: 0x00000001, subscriber_id: 0x805(4), participant_id: 0x001(1)
[1763144324.452642] info     | ProxyClient.cpp    | create_datareader        | datareader created     | client_key: 0x00000001, datareader_id: 0x805(6), subscriber_id: 0x805(4)
[1763144324.557653] info     | ProxyClient.cpp    | create_topic             | topic created          | client_key: 0x00000001, topic_id: 0x806(2), participant_id: 0x001(1)
[1763144324.558006] info     | ProxyClient.cpp    | create_subscriber        | subscriber created     | client_key: 0x00000001, subscriber_id: 0x806(4), participant_id: 0x001(1)
[1763144324.560397] info     | ProxyClient.cpp    | create_datareader        | datareader created     | client_key: 0x00000001, datareader_id: 0x806(6), subscriber_id: 0x806(4)
[1763144324.666426] info     | ProxyClient.cpp    | create_topic             | topic created          | client_key: 0x00000001, topic_id: 0x807(2), participant_id: 0x001(1)
[1763144324.666772] info     | ProxyClient.cpp    | create_subscriber        | subscriber created     | client_key: 0x00000001, subscriber_id: 0x807(4), participant_id: 0x001(1)
[1763144324.669192] info     | ProxyClient.cpp    | create_datareader        | datareader created     | client_key: 0x00000001, datareader_id: 0x807(6), subscriber_id: 0x807(4)
[1763144324.774361] info     | ProxyClient.cpp    | create_topic             | topic created          | client_key: 0x00000001, topic_id: 0x808(2), participant_id: 0x001(1)
[1763144324.774711] info     | ProxyClient.cpp    | create_subscriber        | subscriber created     | client_key: 0x00000001, subscriber_id: 0x808(4), participant_id: 0x001(1)
[1763144324.777242] info     | ProxyClient.cpp    | create_datareader        | datareader created     | client_key: 0x00000001, datareader_id: 0x808(6), subscriber_id: 0x808(4)
[1763144324.882876] info     | ProxyClient.cpp    | create_topic             | topic created          | client_key: 0x00000001, topic_id: 0x809(2), participant_id: 0x001(1)

The MAVLink console in QGC outputs the following:

nsh> uxrce_dds_client status
INFO  [uxrce_dds_client] Running, connected
INFO  [uxrce_dds_client] Using transport:     serial
INFO  [uxrce_dds_client] Payload tx:          0 B/s
INFO  [uxrce_dds_client] Payload rx:          0 B/s
INFO  [uxrce_dds_client] timesync converged: false
uxrce_dds_client: cycle: 0 events, 0us elapsed, 0.00us avg, min 0us max 0us 0.000us rms
uxrce_dds_client: cycle interval: 0 events, 0.00us avg, min 0us max 0us 0.000us rms

“ros2 topic list” shows some /fmu/in topics, but no /fmu/out topics:

nsh> ros2 topic list
/fmu/in/actuator_motors
/fmu/in/actuator_servos
/fmu/in/arming_check_reply
/fmu/in/aux_global_position
/fmu/in/config_control_setpoints
/fmu/in/config_overrides_request
/fmu/in/differential_drive_setpoint
/fmu/in/goto_setpoint
/fmu/in/manual_control_input
/fmu/in/message_format_request
/fmu/in/mode_completed
/fmu/in/obstacle_distance
/fmu/in/offboard_control_mode
/fmu/in/onboard_computer_status
/fmu/in/register_ext_component_request
/fmu/in/sensor_optical_flow
/fmu/in/telemetry_status
/fmu/in/trajectory_setpoint
/fmu/in/unregister_ext_component
/fmu/in/vehicle_attitude_setpoint
/fmu/in/vehicle_command
/fmu/in/vehicle_command_mode_executor
/fmu/in/vehicle_mocap_odometry
/fmu/in/vehicle_rates_setpoint
/fmu/in/vehicle_thrust_setpoint
/fmu/in/vehicle_torque_setpoint
/fmu/in/vehicle_trajectory_bezier
/fmu/in/vehicle_trajectory_waypoint
/fmu/in/vehicle_visual_odometry
/parameter_events
/rosout

When trying to echo a topic, there is no output (blank). I have also connected the Pixhawk to my RPi with a usb to serial connection; both TXD and RXD LEDs are turned on, which indicates that data is both being sent and received.

The software versions I am using are the following:

  • ROS2 Humble
  • Ubuntu 22.04
  • PX4 v1.15.4 (have also tried 1.15.0 and 1.16) on my pixhawk only
  • px4_msgs and px4_ros_com: release/1.15
  • uXRCE-DDS Agent version: 2.4.2 with the fastdds version set to 2.13/2.13.6 as seen here

I have also set the following parameters in QGroundControl:

MAV_1_CONFIG = 0 (Disabled)
UXRCE_DDS_CFG = 102 (TELEM2)
SER_TEL2_BAUD = 921600

I suspect it is some message/software version mismatch, but I have tried several software combinations and have not found a solution yet. Any help would be greatly appreciated!

2

Hi @Dimitri , the agent seems indeed to partially connect but the worrying part is the output of uxrce_dds_client status.
Could you try the following:

  1. start up everything, check if the agent seems to start as you reported above.
  2. run uxrce_dds_client status and confirm that it shows disconnected
  3. stop the client with uxrce_dds_client stop and report if it throws any error
  4. try to restart the client with uxrce_dds_client start uxrce_dds_client start -t serial -d /dev/ttyS2 -b 921600 (refer to Holybro Pixhawk 4 | PX4 Guide (main) to get the right /dev/tty*). Report any output of that command please.
3

Hi @Benja , thanks for your reply. The output I get is:

nsh: sysinit: fopen failed: No such file or directory
NuttShell (NSH) NuttX-11.0.0
nsh> uxrce_dds_client status
INFO  [uxrce_dds_client] Running, disconnected
INFO  [uxrce_dds_client] Using transport:     serial
INFO  [uxrce_dds_client] timesync converged: false
uxrce_dds_client: cycle: 0 events, 0us elapsed, 0.00us avg, min 0us max 0us 0.000us rms
uxrce_dds_client: cycle interval: 0 events, 0.00us avg, min 0us max 0us 0.000us rms
nsh> uxrce_dds_client stop
nsh> uxrce_dds_client start -t serial -d /dev/ttyS2 -b 921600
INFO  [uxrce_dds_client] init serial /dev/ttyS2 @ 921600 baud
nsh> uxrce_dds_client status
INFO  [uxrce_dds_client] Running, disconnected
INFO  [uxrce_dds_client] Using transport:     serial
INFO  [uxrce_dds_client] timesync converged: false
uxrce_dds_client: cycle: 0 events, 0us elapsed, 0.00us avg, min 0us max 0us 0.000us rms
uxrce_dds_client: cycle interval: 0 events, 0.00us avg, min 0us max 0us 0.000us rms
nsh> WARN  [timesync] RTT too high for timesync: 12 ms

The timesync warning appears after I start the agent on my Raspberry Pi. I’ve tried the same method earlier today as well. When it was still connected, I got the following error:

nsh> uxrce_dds_client stop

ERROR [uxrce_dds_client] timeout, forcing stop

When trying to restart the client, QGC crashed. It seems to work fine when the client is disconnected before stopping.

When I unplug the UART connection while the agent and client are running, uxrce_dds_client status will still display “running, connected” for some reason.

4

I saw the same behaviour before but I did not have time to properly investigate it.

Could you try main branch of PX4?

5

I pulled the main branch from github, made px4 using make px4_fmu-v5_default and uploaded this to my Pixhawk 4 in QGC. I also reinstalled px4_msgs to ensure message compatibility (now also main). As a result, no topics are created at all (not even /fmu/in topics) and the data link seems to get stuck earlier on than before. Below I’ll post the output of my RPi terminal and MAVLink terminal:

MicroXRCEAgent serial --dev /dev/ttyAMA0 -b 921600
[1763637060.901813] info     | TermiosAgentLinux.cpp | init                     | running...             | fd: 3
[1763637060.902615] info     | Root.cpp           | set_verbose_level        | logger setup           | verbose_level: 4
[1763637062.385012] info     | Root.cpp           | create_client            | create                 | client_key: 0x00000001, session_id: 0x81
[1763637062.385214] info     | SessionManager.hpp | establish_session        | session established    | client_key: 0x00000001, address: 1
[1763637062.506150] info     | ProxyClient.cpp    | create_participant       | participant created    | client_key: 0x00000001, participant_id: 0x001(1)

MAVLink console:

nsh> uxrce_dds_client stop
ERROR [uxrce_dds_client] timeout, forcing stop
nsh> uxrce_dds_client start -t serial -d /dev/ttyS2 -b 921600
INFO  [uxrce_dds_client] init serial /dev/ttyS2 @ 921600 baud
nsh> ERROR [uxrce_dds_client] timeout during time synchronization
ERROR [uxrce_dds_client] session setup failed, will retry now
INFO  [uxrce_dds_client] init serial /dev/ttyS2 @ 921600 baud
ERROR [uxrce_dds_client] timeout during time synchronization
ERROR [uxrce_dds_client] session setup failed, will retry now
INFO  [uxrce_dds_client] init serial /dev/ttyS2 @ 921600 baud
nsh> uxrce_dds_client status
INFO  [uxrce_dds_client] Running, disconnected
INFO  [uxrce_dds_client] Using transport:     serial
INFO  [uxrce_dds_client] timesync converged: false
uxrce_dds_client: cycle: 0 events, 0us elapsed, 0.00us avg, min 0us max 0us 0.000us rms
uxrce_dds_client: cycle interval: 0 events, 0.00us avg, min 0us max 0us 0.000us rms
6

After some troubleshooting I found something that might be useful. Using the main versions of both px4_msgs and PX4, I initially got the same output as my last message.

I then opened a new terminal and started a new agent (while the existing one was running, stuck at “create participant”). I let it run for a while and then killed the second (newer) agent. A few seconds after this, I got an output in which the /fmu/out topics were created. I could echo these topics in a new terminal and got the correct data flowing.

It seems that the original agent stops (the dataflow stagnates when starting the second agent), and correctly restarts once the second agent is terminated, as if it triggers some process. The data I get when echoing the topics seems to buffer a lot as well (extreme fluctuations in data rate).

Just to be clear, my process to get to this point is:

  1. Start the agent
  2. Open a new terminal
  3. While agent one is running, start a new agent
  4. The old agent will freeze
  5. Stop the new agent
  6. The old agent will unfreeze and suddenly work

The output from both the MAVLink console and the RPi terminal looks fine to me, let me know if you’d like me to include it.

1 Like
7

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.