High IMU noise affect VICON odometry

The log file: https://review.px4.io/plot_app?log=322128d3-6a02-49c7-a038-6bd04a0d266a

I am using a quadcopter in indoor environment without GPS. Localization is done using a VICON system.
I am having problems about the EKF2 estimate of the local position of the drone: sometimes the estimate differs a lot from the VICON measurement.

I noticed that the vibration metrics are quite high and when there are very high spikes of noise the local position estimate is affected a lot, like in this graph representing axis Z:

(the reason for this spike is unknown, the drone was in flight for the entire time without high speed manuvers)

I already tried to reduce the vibrations with little success, there is a way to rely more on the visual_odometry measurement instead of the IMU? I am having this problem only on Z axis. I already tried to modify parameters about the accelerometer noise but there is no change.