Drifting altitude when hovring with Offboard mode

Hi I am using offboard with via raw MAVLink connection with SET_POSITION_TARGET_LOCAL_NED and SET_POSITION_TARGET_GLOBAL_INT.
when I am trying to hover with vx 0, vy 0 and vz 0 with SET_POSITION_TARGET_LOCAL_NED or with SET_POSITION_TARGET_GLOBAL_INT the drone is drifting between 5 meters.
I am using px4_sitl in gazebo classic.
thanks for the help

If you are sending zero velocity, you wont be able to control the position, since it is not controlling it. The tracking error of velocity would accumulate resulting in a drift. If you want the vehicle to keep position, you should control position, not velocity

So if I want to hover I need to set position.
Can I set position relative to the drone frame? (I want to set 0,0,0)
I tried to use SET_POSITION_TARGET_LOCAL_NED but It only supports position that relative to the home of the drone.

No, because then it will be always moving (wherever the drone is it is 0, 0, 0) and will drift.

Ok thanks for the help I will use global frame (lat, long) to hold position.