Pixhawk 4 position estimation with ultra-wide band system problems

Hi all,
I’m trying to fly a drone with position control using Ultra-Wideband devices as a position source. However, I’m having trouble getting the position estimation done by the EKF2 algorithm right. Right now, it looks like the flight controller is relying too much on the accelerometer and too little on the ultra-wideband, based on the video. In this video, I use Rviz (Robot Operating System visualization) to show the estimated position from the flight controller itself, which is the topic /mavros/local_position/pose, and is the red arrow in the video, and the estimated position by ultra-wideband, the yellow arrow.The ultra-wideband positioning (yellow arrow) is very jumpy, but is quite correct on average, while the flight controller positioning is much more smooth, but behaves really odd. Here is a list of the related parameter settings in QGroundControl:
EKF2_AID_MASK = 24 (vision position fusion & vision yaw fusion)
EKF2_HGT_MODE = range sensor (I’m also using lidar for height estimation. This works fine)I hope someone will be able to shed some light on this situation. If more information is needed, please don’t hesitate to ask!

Here’s a link to the video I mention: https://drive.google.com/file/d/1xu__f5OmADQAlR3Xl7VJGMmZhevouuoF/view?usp=sharing