PX4Flow false distance measurements

I have been trying to get my pixhawk 4 based drone to maintain xyz position using with the PX4flow.
The drone drifts as soon as it is lifted off the ground in most cases.

I’ve checked camera image and feature quality as said here: https://docs.px4.io/en/sensor/px4flow.html#developer_guide
My pixhawk4 uses EKF2_AID_mask 3 (gps + optical flow) as proposed.

When looking at the sonar range OPTICAL_FLOW_RAD.distance message in QGC I’ve noticed that the distance reading is normally zero except when an obstacle is under 50cm from the sensor. I’ve seen this happen with two different modules, both indoors and outdoors.

Has anyone else experienced these difficulties?

I think the usual recommendation is to use a separate lidar sensor instead of what comes with PX4Flow. Here are a few listed: https://docs.px4.io/en/sensor/rangefinders.html