Ok, so I looked at the very first log and I can see that you first fly in stabilized mode which seems to work ok. Then you switch to position control.
Now, the attitude still seems to be following the attitude setpoint fine, however the velocity and position diverges rapidly. This let's me believe that something about the coordinate system must be wrong. Have you checked that the heading is correct with North defined in your position system and that all the signs are correct? It is likely that the mag is off indoor where you have a lot of magnetic interference.
Edit: Also, it seems that your magnetometer is also disturbed by the motors and or power cables. You can see the correlation between thrust (= more current) and the magnetic field here: https://logs.px4.io/plot_app?log=cd0fb154-cd8c-4eb5-b885-438d103d3ae8#Nav-Thrust-and-Magnetic-Field
I suggest that you use an external magnetometer like the ones on the GPS and move it far away from wires and motors. And also make sure to twist all cables that have DC current on them such as battery to power module and power module to ESCs, this also reduces the electromagnetic field.
Another idea would be to use the optitrack for the yaw angle instead of the magnetometer but I don't know how or if that can be done.