EKF snap to zero with LeddarOne

Flight Log

I am testing my LeddarOne sensor with optical flow on the Aero RTF. On the bench, with the rotors off, I arm the drone and transition to Offboard mode and give a command to takeoff to a height of ~0.2m. All of that completes successfully, except for the actual takeoff since the rotors are off. I wanted to make sure that the local position indication would work before I attached the rotors, so I manually pick up the drone to see what the response is. Initially, it seems that X and Y move, but then seem to exponentially drift, then both snap to zero. Meanwhile, Z pretty much follows the movement.
I’m assuming that the EKF2 estimate integrates up some error, up to a point, then rejects optical flow. Is that what is happening?
Is there a better test to ensure that the local positioning works before attaching the rotors and having the drone fly off in some direction thinking that it’s at (0,0) the whole time?