EV data causing the drone to crash when in stabilize mode

I am flying PX4 v1.11 and v1.13 on the ModalAI Flight core with their VOXL companion computer that gives VIO data - and we experience a bug in PX4 that results in crashes. It’s something that should not happen, and normally does not happen but when it does we need to emergency land or if possible switch to acro mode and then safely make an emergency landing.

So what happens: In stabilize mode when flying the drone, after flying for some time without any troubles, the drone suddenly pitches or rolls in one direction to its maximum angle. Even counter-steering the drone in the other direction only works to some degree, it will still go in that direction. When changing to acro, we can still safely control and land the drone.

Here we have a Log of one instance. You can see the drone is only in stabilize mode, and at the end of the flight, the pilot is at full pitch and rolls to make a safe emergency landing. I have a video of this, and this just happens when hovering, the drone is not at any cruising speed or something. The drone is just scanning something an object from top to bottom.
https://logs.px4.io/plot_app?log=cde7b479-812b-4d3f-8035-c8c79fc4ca3e

The way we solve this problem is when we fly in stabilize mode we change the EKF2_AID_MASK from 280 to 0.

So what we think happens is that the VIO data becomes bad (due to low features or whatever), and think the drone is going in the wrong direction. So the drone wants to counteract this and rolls and pitches in the opposite direction the VIO thinks it’s going to. This is something that would happen in position-hold mode. But here comes the bug, this also happens in modes that “don’t” use the VIO data. We see this happening in modes like stabilize, altitude hold mode, and position failsafe.

How is this possible? And can we switch this off? Or is this a bug that needs to be fixed?