Hight Position Drift Issue using LIO(lidar inertial odometry)/VIO

I am working on indoor UAV SLAM using a Pixhawk 6C running PX4 v1.15.2 paired with a Jetson Orin Nano as the companion computer. For localization, I am using a Livox Mid-360 LiDAR and running FAST-LIVO2 for local position estimation.

After launching FAST-LIVO2, it publishes the estimated pose on /mavros/vision_pose/pose. I have configured the PX4 parameters according to the Using Vision or Motion Capture Systems for Position Estimation guide. Initially, in QGroundControl (QGC), I observe a no valid local position warning, but this disappears once FAST-LIVO2 starts running, indicating that the system is receiving position estimates.

However, when switching to Position mode, the drone successfully takes off, but it experiences significant drift without any manual input. The drift is highly unstable, making it difficult to control the UAV.

I have attached multiple test flight logs for reference:
:pushpin: Test-flight1
:pushpin: Test-flight2
:pushpin: Test-flight3
:pushpin: Test-flight4

I would appreciate any suggestions on how to resolve the position drift issue

Any help or guidance would be greatly appreciated. Thanks in advance!