I am trying to get multiple (4+) quads flying on a single network with a single router. Currently, mavros+mavlink is running on the same onboard computer running px4. We have a ground station sending input commands at 20Hz via ROS and a second computer on each drone sending position data over wifi to each quad (also at 20Hz).
Network congestion is getting in our way, big time. We cannot fly more than 3 at once with this setup, and upgrading to a high-end router made no difference.
Our first step, moving everything from the ground control station to the quad except for the joystick interpreter, has not helped at all. I have moved all of our connections from TCP to UDP and it barely makes a difference.
It seems to be an issue router-side (packets are being sent correctly but are being received in clumps or not at all). What else could we try?
We have a ground station sending input commands at 20Hz via ROS and a second computer on each drone sending position data over wifi to each quad (also at 20Hz).
Can you clarify if this is sending ONLY position data between drones?
How are you running your ROS master? Is everything bound to a single ROS master?
If you want to check if it is the wifi problem, you need to monitor the traffic from the router, and see if it is reaching it’s maximum.
Yes, the second computer is sending pose data and nothing else at 20Hz. It’s split into two message types, so we’ll call it 40Hz.
Everything is using the same ros master. It’s running on the computer sending 20Hz input commands.
The router is not hitting its maximum traffic level but it seems to be having trouble delivering things as they are produced. I wiresharked the data and I will see a 20Hz stream being produced on one machine being received in “clumps” of 10-20 messages at once on another. This is only observed when more than 3 quads are connected.
They’re being sent via UDP as a custom message type. It’s split between a position and an attitude message, of 122 and 134 bytes respectively (including message headers). Both messages are published simultaneously. We have tried the same setup with a nav_msgs/odometry message and had the same results.
edit: It was pointed out to me that we’re experiencing packet loss and network troubles with things like ssh and ping so I’m skeptical of it being a problem with roscore.
I am having similar problem with indoor setup of multiple drones too. Have you figured out the bottleneck of this issue?
Are you using 5GHz or 2.4? Static/dynamic IP?
We have not figured out the bottleneck. We are using static IPs on 5GHz.
In your case, it’s possible that it’s a simultaneous transmission issue. As I understand it, 802.11ac uses something called MU-MIMO to handle beamforming but it has a limited number of devices to which it can simultaneously transmit. You need to be careful about simultaneous sending of packets.
