EKF2 diverging of position with optical flow only

I am using ekf2 associated with v1.7.3 px4 firmware.
My setup includes ACC, GYRO, MAG, sonar, optical flow.
I put my drone static on a shelf with distance of around 0.7m above the ground so that the optical flow camera could work through the holes of the shelf.
The following graph is the estimates from the EKF2.
The flow data is proved to be correct. ekf2_lp_pos is the local position output, while the lp_valid_xy_z is xy and z valid.

Here is some params I think be related. EKF2_ACC_NOISE 0.05; EKF2_BARO_NOISE 1; EKF2_HGT_MODE 0; EKF2_AID_MASK 3; EKF2_OF_N_MIN 0.01; EKF2_OF_N_MAX 0.1; EKF2_MAG_TYPE 1; EKF2_OF_GATE 5.
I have difficulties in understanding related code since some of them are generated by MATLAB syms.
So is there anyone have some advice?
Thank you.

I am also facing the exact same issue. Can anyone tell a reason and correction for this?

Thanks in advance!