Uncommanded Yaw when switching EKF2_MAG_TYPE mode in 1.7.3

We are occasionally seeing an uncommanded yaw motion of our aircraft shortly after takeoff. It seems to happen at about 1.5m in the air, which is the trigger height for the EKF2_MAG_TYPE=0 switch from magnetometer->heading to magnetometer->3D vector.

There are two logs showing this behavior:
https://logs.px4.io/plot_app?log=51cbc618-7bbb-47fd-9091-1eaa92a1baf4
https://logs.px4.io/plot_app?log=11c49798-526a-4fe7-b8f9-648265bd34c5

You can see that right at the beginning of the flight, there is a step change in both the Yaw setpoint and the yaw estimate. However the yaw estimate then diverges and then comes back down to the setpoint. On the aircraft, this resulted in a ~30 deg yaw of the aircraft.

It looks to me like the heading estimate was updated because of the switch to using the full 3D magnetometer vector, and the setpoint was changed too to prevent a yaw error signal from being generated. However something appears to cause an overshoot which commands a yaw change.

There is also a pitch and roll bump in the estimate at the same time. It seems like their is a discontinuity when the EKF2_MAG_TYPE mode changes. I can’t explain why it only happens some flights and not others. Could calibration be a problem? Or the EKF isn’t fully “aligned” when this happens, so there is a transient?

Platform:
Octorotor with custom PX4 version branched at 1.7.3, running on a Pixhawk 2.1 cube with HERE2 GPS antenna.

*NOTE this has happened on two different aircraft.

We are aware that 1.7.3 does not support the external magnetometer in the HERE2, and we are using the internal magnetometers. In one of the logs above, we have CAL_MAG0_EN set to 0 to force use of the second magnetometer, because the MPU9250 which is MAG0 shows periodic spikes in its readings. MAG1 does not show these spikes but this did not seem to affect the results.

Any insight on what causes this discontinuity when switching from heading to 3D would be helpful. Thanks!

@Paul_Riseborough you probably immediately know what this is about :smile:. Thanks.

@Paul_Riseborough Any chance you could point us in the right direction about what is causing this?

We found this PR which appears related https://github.com/PX4/Firmware/pull/10848. We’ve tested with a fixed declination and it seems to improve. Can you confirm the behavior described above is likely caused by this issue?

Thanks!

There have been a number of fixes for yaw reset behaviour since that release. The is also a pending change that could be relevant: https://github.com/PX4/ecl/pull/570

Edit: The yaw reset is being triggered at 1.5 metres, so that commit is unlikely to be the issue.

The way yaw resets were handled internally were significantly modified by this PR: https://github.com/PX4/ecl/pull/556

Looking at the data, the mag data is likely going to be too noisy for the single point sample method used during the yaw reset, eg look at this plot of yaw innovation and mag field length.

We would need to modify the code to use a multipoint sample method to handle this type of data if an inflight reset using measured field is to be performed.

If it helps, we’re working on fixing our inoperative external magnetometer (HERE2) which should help with the mag noise.

Based on what you see in the log, does it seem likely that the yaw reset is the cause of our uncommanded yaw motions? The described behavior in the PR seems to be consistent with what we’re seeing, but given the enormous complexity of PX4 I can’t rule out other causes yet.

Thanks!