Drone flies away during RTL

Firmware version: 1.9.2 with slight modifications (but nothing RTL related)

We have observed this behaviour now a couple of times: After triggering RTL the drone flies over its destination point, but breaks out a few meters to north-east. This behaviour can be seen in the local position x and local position y graphs in https://review.px4.io/plot_app?log=a52c8e3e-0ce8-4bba-a19e-ed40d4ba0490 where the estimated (=percepted by us!) deviates from the setpoint.

Another such flight is https://review.px4.io/plot_app?log=ab16ef90-548d-4b0e-9d04-1af1f4f37176

Is there any sensor error responsible for that? Or is there something wrong with RTL?