Good Afternoon Folks,
I was wondering if anybody on the forum had any experience using cuVSLAM as a library for Visual Inertial Odometry (VIO)? My current lab team is attempting to go this route for position estimation and has had limited success with this method. I performed all of the original methology, setup, and troubleshooting that the authors of the paper recommend for setting up the camera which involved calibration, making sure you had clean feature point generation and Lucas-Kanade tracking frame by frame. On the bench I did some ground tests of checking orientation through one of their many provided examples by moving ~20cm in each direction (N,S,E,W, Up, Down) and had reasonable looking transformation frames.
Once I established what I thought were good bench tests, I (with the help of Claude) wrapped up the cuVSLAM library into a ROS 2 node, followed the PX4 guide for integrating vision sensors as a form of odometry data, and tried to run some position hold flights. On Thursday night, I ran some tests in the lab and it performed wonderfully. Since then, it’s been incredibly finicky with odd takeoff interactions, weird ovular oscillatory behavior in the XZ-plane of all places (tries to lock a position, goes up, goes down and forward, more down and back, up, and further back, then back forward and higher up if that makes sense), and loss of direction.
For the drone, I’m using a stereo camera in the OAK-D S2 which is mounted about 7cm from the floor at a 45 degree angle which is seemingly the best angle of mounting for VIO. When I fire up the drone though I get very clean position tracking at (0,0,0) +/- 1/1000 of a meter, and I’ve been able to leave it sit for a couple minutes with no real position change, which is a good thing that the LOCAL_POSITION_NED doesn’t skew off. But when I go into the air, I get massive changes in position almost as if the scaling is wrong. When I looking back at the ROS bags though, there are tons of feature points. The only other possible error I thought was occuring was loss of too many frames, but I ran isolated tests with cuVSLAM the core process running, maximum power, didn’t lose any frames (thus would’ve been relying on IMU data solely), and it still had problems.
For what it’s worth, I’ll try to post an update if I ever find a solution to my problems and add the copious notes I’ve been taking on the whole process, just wanted to see if anybody else on the forum had attempted this.
Specs
Our current setup:
- Flight Controller: OrangeCube+
- Flight Stack: PX4-Autopilot
- Companion Computer: Jetson Orin Nano Devkit
- Propellers: 13 inch
- Battery: 6S
- PX4 Version: v1.16.0
- Jetpack Version: 6.2
- ROS 2 Version: Humble
- GPS-Denied: True
Logs
Here you can find parameters, logs, and rosbags linked in the google drive if it assists anybody in the future. LMK if you don’t have viewing permission and would like it, it should be that anyone with a link can view:
