No global position failsafe with GPS lock


Today we ran twice into a Failsafe: no global position situation, and this even though the log shows continuous GPS lock with 17 (or 16) satellites and accuracy < 1m.

On the first occurrence, it happened during the VTOL Takeoff ascent. I linked it to a bad Accels calibration as just after launching the mission QGC showed a “Preflight advice: Check Accel Cal” message, but the drone started the mission anyways.
Flight review :
EKF : 08_00_02.ulg.pdf (152.3 KB)

After a recalibration, the mission was executed successfully:
Flight review :
EKF log : 08_12_24.ulg.pdf (281.8 KB)

Relaunching the same mission after letting the motors cool down, the failsafe was triggered shortly after the front transition.
Flight review :
EKF log : 08_22_18.ulg.pdf (268.2 KB)

Launching the mission a fourth time worked well again.

From the EKF pdfs I figured out that it was the diverging Velocity and Position innovations that triggered the failsafe, but I’m wondering how this could happen with a good GPS signal. Any ideas?

Running v1.9.0 on a Cube in a VTOL drone. I have no memories of this happening with v1.8.2 with which we have approx. 10h flight time.

Thanks for any help,

Update: just ran into the same issue again yesterday.

Flight review:
EKF log: 15_08_21.ulg.pdf (230.8 KB)

There is a logged message about resetting yaw. Can a misalignment in yaw cause the failure in position estimation? If yes, how do we make sure it doesn’t happen again?

I could really need some help here. Losing global position seemingly out of nowhere even though we have full GPS is quite problematic for flying automatic missions.

@Paul_Riseborough judging from your talk at the dev summit you’re an expert on EKF, would you have any ideas on the root cause of the diverging innovations?

Thanks a lot!

You seem to have quite high vibrations. I would check the mounting of the IMU and especially the magnetometer, the amplitude of the oscillation in the norm of the magnetic field is very large.