No global position failsafe with GPS lock

PX4 Autopilot
1

Hi,

Today we ran twice into a Failsafe: no global position situation, and this even though the log shows continuous GPS lock with 17 (or 16) satellites and accuracy < 1m.

On the first occurrence, it happened during the VTOL Takeoff ascent. I linked it to a bad Accels calibration as just after launching the mission QGC showed a “Preflight advice: Check Accel Cal” message, but the drone started the mission anyways.
Flight review : https://logs.px4.io/plot_app?log=0c90788c-04a1-4e44-8659-a47d0bc4e488
EKF : 08_00_02.ulg.pdf (152.3 KB)

After a recalibration, the mission was executed successfully:
Flight review : https://logs.px4.io/plot_app?log=68795cb1-97e0-4ca1-a516-c8a4f07c9c3f
EKF log : 08_12_24.ulg.pdf (281.8 KB)

Relaunching the same mission after letting the motors cool down, the failsafe was triggered shortly after the front transition.
Flight review : https://logs.px4.io/plot_app?log=9603c8e7-9301-418b-9c08-2f9d7fa293d0
EKF log : 08_22_18.ulg.pdf (268.2 KB)

Launching the mission a fourth time worked well again.

From the EKF pdfs I figured out that it was the diverging Velocity and Position innovations that triggered the failsafe, but I’m wondering how this could happen with a good GPS signal. Any ideas?

Running v1.9.0 on a Cube in a VTOL drone. I have no memories of this happening with v1.8.2 with which we have approx. 10h flight time.

Thanks for any help,
Thomas

2

Update: just ran into the same issue again yesterday.

Flight review: https://logs.px4.io/plot_app?log=7c1a4be9-8e6b-41e2-b7fa-dd29cd270795
EKF log: 15_08_21.ulg.pdf (230.8 KB)

There is a logged message about resetting yaw. Can a misalignment in yaw cause the failure in position estimation? If yes, how do we make sure it doesn’t happen again?

I could really need some help here. Losing global position seemingly out of nowhere even though we have full GPS is quite problematic for flying automatic missions.

@Paul_Riseborough judging from your talk at the dev summit you’re an expert on EKF, would you have any ideas on the root cause of the diverging innovations?

Thanks a lot!

3

Hi,
You seem to have quite high vibrations. I would check the mounting of the IMU and especially the magnetometer, the amplitude of the oscillation in the norm of the magnetic field is very large.