Local_planner and stability in Avoidance

When using the avoidance ros package I get very unstable flight. The drone drifts erratically with up to a radius of 10 meters, which seems correspond to the output of the local_position/pose mavros topic.
When I take off with Qgroundcontrol though, the flight is relatively stable even though the local_position/pose is still drifting around.
Does anyone else have a similar issue, or can this be resolved with a different set ekf parameters or something of the sort?

The Local Planner does not need retuning of the EKF. If the drone flies stable in position control, than that should be good enough for the avoidance system. Are you trying to fly in OFFBOARD mode or MISSION mode? I could try to help you debugging your system, if you could provide a rosbag containing the most important ROS topics (e.g. /goal_position /current_setpoint /complete_tree /histogram_image /local_planner_node/parameter_descriptions /local_planner_node/parameter_updates /mavros/companion_process/status /local_pointcloud /cost_image /tf /tf_static /tree_path )