PX4 + ROS 2 + Foxglove multi-vehicle camera streaming
I recently tested Foxglove as a live viewer for PteroSim camera streams over ROS 2.
Demo:
We flew four x500 quadcopters, each running its own PX4 instance, in a 2.5 m orbit around a shared center. The vehicles were pointing inward, so each gimbal camera could see the other three.
The cameras were running at 768×480 and 30 FPS. PteroSim encoded the video to H.264 using NVENC and sent it over RTP/UDP.
ROS 2 Humble was running in WSL2 on Ubuntu 22.04 with mirrored networking. A small Python bridge forwarded the H.264 stream without decoding it and published it as foxglove_msgs/CompressedVideo. Vehicle poses were published through /tf, and foxglove_bridge exposed everything to Foxglove on Windows.
We initially tried sending raw rgb8 images through DDS. At this resolution, each frame was around 1.1 MB, and with the default Linux UDP buffers we were losing around 40–50% of the messages.
H.264 worked much better for this setup.
We also ran into a DDS discovery issue. Fast DDS, which is the default in ROS 2 Humble, did not discover the nodes correctly with WSL2 mirrored networking. Switching to Cyclone DDS fixed it.
With all four video streams running at the same time, we were getting around 27 FPS per camera on average, compared with the 30 FPS target.
The test was done on:
Intel i9-14900HX
RTX 4090 Laptop GPU 16 GB
64 GB RAM
Windows 11 + WSL2
ROS 2 Humble
Has anyone else tried a similar multi-vehicle camera setup with PX4 and ROS 2?
Interesting results especially the raw DDS vs. H.264 comparison.
One thing I’d be interested in measuring in this setup is the end-to-end temporal behaviour across all four vehicles: camera capture timestamp → encoding → ROS 2 publication → Foxglove reception, correlated with the corresponding PX4 vehicle state.
At ~27 FPS per stream, average throughput looks good, but it would be useful to see whether latency/jitter and frame loss remain evenly distributed or whether individual streams diverge under load.
Did you preserve the original capture timestamps through the H.264 bridge, or are the CompressedVideo timestamps generated when the packets are published into ROS 2?
That distinction could become important if the video is later correlated with PX4 /tf, estimator state, or other sensor data.
Also interesting that Cyclone DDS resolved the WSL2 mirrored-networking discovery issue. A reproducible comparison of Fast DDS vs. Cyclone DDS under the same four-vehicle workload would be useful.
@yigitcan-ozturk The timestamps are generated at publish. ffmpeg in the simulator stamps each frame with the wall clock when it reads it (-use_wallclock_as_timestamps 1), and the bridge ignores the RTP timestamp anyway: CompressedVideo.timestamp is now() at publish, and /tf is stamped the same way. Video and /tf share a clock, but each frame is late by the pipeline latency, which I haven’t measured end to end. The stamp should be sim time at capture, carried through the stream. Under lockstep that’s PX4’s clock too. I haven’t done that yet.
Thanks that clarifies it. In that case the streams are clock-consistent at publication, but not necessarily temporally aligned to the physical/simulated observation.
I agree that carrying the capture-time sim timestamp through the pipeline would be the cleaner model, especially under lockstep where it can be related directly to the PX4 state timeline.
Before changing the bridge, it might be useful to measure the existing pipeline once:
Even a small sample would give you the latency distribution and jitter for each of the four streams rather than just average FPS.
Then, after propagating capture timestamps, you could compare the same run again. That would separate two different properties:
transport latency — how late the frame arrives temporal correctness — what instant the frame actually represents
That distinction becomes especially useful when correlating a visible event in the camera with PX4 state or /tf.
If you end up capturing such a trace, I’d be very interested in comparing the four streams and looking for the first point where their timing behaviour diverges under load.
@Benja Nothing else, the Agent and client are at defaults.
The cause turned out to be WSL2 mirrored networking: UDP to 127.0.0.1 goes through Windows with a 1500 MTU, so anything over 1472 bytes is dropped. Fast DDS sends same-host SEDP to 127.0.0.1 in ~4 KB datagrams, so discovery never completed, while Cyclone uses the interface address. Setting <maxMessageSize>1400</maxMessageSize> on the Fast DDS UDPv4 transport fixes it, and the setup now runs on Fast DDS too.
Here’s the trace you asked for. We had to add two things to the sim first: every frame carries its capture time (wall clock and sim time) in an H.264 SEI, and the stream bitrate is a camera setting. Measuring turned up a few bugs on our side, which are fixed now. ffmpeg dropped the first frame, so every stamp was one frame off. RTP timestamps were rounded to 40 ms and some frames ended up sharing one; they’re the capture time as well now. The encoder also overshot the bitrate by about 10%. On the ROS side, the bridge puts the capture time in the message header instead of the arrival time.
Setup: 4 x500 on PX4 SITL, 768x480 at 30 fps, NVENC at 4 Mbit/s, ROS 2 and foxglove_bridge in WSL2. Capture to the Foxglove client is about 140 ms median, p95 around 220, jitter under 30 ms. Nearly all of it is inside the sim. Render plus GPU readback takes ~105 ms (about three engine frames), encoding ~5 ms, and getting to the bridge another ~30. DDS and Foxglove add 1-3 ms. The four streams stay within half a millisecond of each other, and no frames were dropped.
This is an excellent trace thanks for taking the time to instrument it.
The result is particularly interesting because it separates two things that were previously mixed together: timestamp correctness and transport latency.
With capture time now carried through the H.264 SEI into the ROS message header, the ~140 ms median latency is no longer hidden behind a publish-time timestamp. And the breakdown makes the bottleneck quite clear:
capture/render/readback (~105 ms) → encode (~5 ms) → transport to bridge (~30 ms) → DDS/Foxglove (1–3 ms)
So in this setup, ROS 2/DDS/Foxglove is contributing very little to the end-to-end delay; most of it is already present before the frame reaches that part of the pipeline.
The fact that all four streams remain within ~0.5 ms of each other with no drops is also a useful result for multi-vehicle correlation.
One measurement I would preserve if you continue testing is both timestamps for every frame:
capture sim-time + wall-clock arrival/presentation time
That gives you two independent observables:
event time — when the simulated observation actually happened delivery time — when downstream software could act on it
Then if the simulator load changes, you can distinguish a correctly timestamped but increasingly stale observation from an actual synchronization failure.
Also, finding the one-frame ffmpeg offset and RTP timestamp collision while instrumenting the pipeline is a nice example of why carrying provenance from capture is useful.
If you publish the trace format or instrumentation changes, please link them here — I’d be interested in looking at them.
