Multi EKF disaster

We were flying a quadcopter with a Cube Orange FMU. PX4 detected redundant IMUs (3) and Magnetometers (2) and started multiple EKF instances (6).
During a flight one of the instances had the IMU with values totally inconsistent (for example estimating a roll of 180 deg).
Despite that PX4 decided to switch to that instance making the quadcopter flip and ruinously fall down.
How is it possible that PX4 decided to use an EKF instance that was completely far from the other 4 (that were insted aligned)?
How is it possible that PX4, while in stable hovering, decided to switch to an EKF instance that was assuming the drone was upside down?!?
I am stunned…there is still a huge gap from being a reliable system.
If there is no control over several EKF instances probably this functionality should be disabled by default and with clear warnings.

I’m sorry for the crash. That doesn’t sound good at all indeed.

Can you tell me which PX4 version this was and please upload the log of the flight to logs.px4.io, and share here (or DM if need be). I can then have a closer look

Hi Julian, thanks for your answer.
I just uploaded log here

https://logs.px4.io/plot_app?log=e79075a4-d192-4128-a392-cc492cb06ef4

Firmware version was 1.13.3

Thanks for the log. This does indeed look a bit strange in terms of multiple EKFs. I don’t quite understand why it switched multiple times like this.

I’m tagging @bresch hoping that he can have a quick look at the log. He might also be aware of fixes that have been made since v1.13.3 to address this.