IMU-camera calibration time syncing

#1

Hi all! I am trying to properly calibrate my camera-imu setup in order to run visual odometry algorithms using a Pixracer R15 and a Pointgrey Blackfly Bfly-U3-03S2M-CS camera. The pixracer is properly triggering the camera through the gpio pins, and all of the trigger parameters were set in Qgroundcontrol.
I followed this guide: https://docs.px4.io/en/peripherals/camera.html to set those parameters. The pixracer is connected to a NanoPC via USB link at 921600 baud rate (http://wiki.friendlyarm.com/wiki/index.php/NanoPC-T4).

I also am using a modified ROS pointgrey camera driver that modifies the image timestamps using the mavros cam_imu_sync message (https://github.com/ProjectArtemis/pointgrey_camera_driver), and I am using mavros from the master branch at https://github.com/ProjectArtemis/mavros/tree/master.
The nanoPC has ROS kinetic on Ubuntu 16.04.

The imu is outputting data through /mavros/imu/data_raw topic at 200 hz and the camera is being triggered at 30 hz to the topic /camera/image_raw.

I am trying to perform calibration using Kalibr (https://github.com/ethz-asl/kalibr) but keep ending up with very high offset between the camera and imu. When the mavros timesync feature is on, the time shift offset (given by Kalibr) was around 110 ms. I modified the px4-config.yaml to turn the mavros timesync off so that the pixracer timestamps are sent through by using timesync mode PASSTHROUGH, and then re-recorded the images and imu data. This reduced the time offset to about 64 ms, which is still much too high. Previously I had been using the same camera-IMU setup, but running mavros and the camera driver on a Dell Precision 7510 running Ubuntu 14.04 and ROS Indigo, with the same pointgrey driver but an older version of mavros, and the time delay was a bit lower, around 34 ms.

At this point, I’m at a loss as to what I can do to properly sync the camera and IMU and reduce this offset to something under 10 or so ms. I greatly appreciate the help!
-Tina

2 Likes
#2

Hey there.

Is the offset consistent over multiple calibrations/bags?
Maybe you could try to add a static offset when performing the time-stamping in the cameras driver.
Just use the time-calibration output of kalibr and add/substract this to the stamp.
Suppose that the stamping works and the offset is deterministic this could do the job